Autofac

Using Nitride - Markdown

2025-06-10T05:00:00Z

Yesterday, I created an over-engineered program to copy a single file from one directory to another. Now, time to make it less overkill by transforming that Markdown file into simple HTML.

Series

2025-06-07 Using Nitride - Introduction - I'm going to start a long, erratic journey to update my publisher website. Along the way, I'm going to document how to create a MfGames.Nitride website from the ground up, along with a lot of little asides about reasoning or purpose.
2025-06-08 Using Nitride - Pipelines - The first major concept of Nitride is the idea of a "pipeline" which are the largest units of working in the system. This shows a relatively complex setup of pipelines that will be used to generate the website.
2025-06-09 Using Nitride - Entities - The continued adventures of creating the Typewriter website using MfGames.Nitride, a C# static site generator. The topic of today is the Entity class and how Nitride uses an ECS (Entity-Component-System) to generate pages.
2025-06-10 Using Nitride - Markdown - Examples and explanations of converting Markdown to HTML using MfGames.Nitride and MarkDig.
2025-06-12 Using Nitride - Front Matter - How to use Nitride to take the front matter from pages and add them as a component into the Entity class.

MarkDig

I don't like reinventing the wheel. I mean, I seem to keep doing it but I don't enjoy it. That was one reason why I tried another static site generators before working on MfGames.Nitride. However, when it comes to redoing a Markdown parser, even I'm not that foolish when there is already MarkDig, an excellent library for turning Markdown into HTML and extendable enough that I could also turn Markdown into Gemini (a later post).

In these cases, we need to tell Nitride how to do anything with Markdown since I didn't make it part of the core library. To do that, we need to pull in the NuGet package. While we're at it, we're also going to add the HTML processing library.

$ cd src/dotnet
$ dotnet add package MfGames.Nitride.Markdown
$ dotnet add package MfGames.Nitride.Html

Once we have the packages installed, we need to add those modules into the system. This is where Autofac came in helpful since I just have to add a module for the package it will handle the registration of any operations, components, and systems that we need to use.

// In //src/dotnet/Program.cs
var builder = new NitrideBuilder(args)
    .UseIO(rootDirectory)
    .UseMarkdown()
    .UseHtml()
    .UseModule<WebsiteModule>();

As you can see, I'm trying to follow the generic host pattern for the setup.

Identifying Markdown

While it may be obvious to convert any entity class that ends in .md into .html, we break this apart into separate steps. First, is that we identify a file as a Markdown file. This does two things, it adds the MfGames.Nitride.Markdown.IsMarkdown as a component, and then treats the contents as text instead of binary.

If you remember previously, we had this output:

[00:41:57 INF] <PagesPipeline> Read in 1 files from /src/pages
[00:41:57 INF] <PagesPipeline> Entity: Path /index.md, Components ["MfGames.Nitride.Contents.IBinaryContent","Zio.UPath"]
[00:41:57 INF] <SimplifiedMarkdownPipeline> Reading 1 entities
[00:41:57 INF] <StyleHtmlPipeline> Reading 1 entities
[00:41:57 INF] <OutputHtmlPipeline> Writing out 1 files
[00:41:57 INF] <OutputHtmlPipeline> Entity: Path /build/typewriter/html/index.md, Components ["MfGames.Nitride.Contents.IBinaryContent","Zio.UPath"]

Now, we're going to use a new operation, MfGames.Nitride.Markdown.IdentifyMarkdownFromPath. This could be put in a central place, such as SimplifiedMarkdownPipeline, but I found it is better to do this earlier than later so I usually put the identify process in the input methods. In this case, PagesPipeline:

// In //src/dotnet/Pipelines/Inputs/PagesPipeline.cs
public PagesPipeline(
    ILogger<PagesPipeline> logger,
    ReadFiles readFiles,
    IdentifyMarkdownFromPath identifyMarkdownFromPath)
{
    _logger = logger;
    _identifyMarkdownFromPath = identifyMarkdownFromPath;

    _readFiles = readFiles
        .WithPattern("/src/pages/typewriter/**/*.md")
        .WithRemovePathPrefix("/src/pages/typewriter");
}

public override IAsyncEnumerable<Entity> RunAsync(
    IEnumerable<Entity> entities,
    CancellationToken cancellationToken = default)
{
    var list = _readFiles
        .Run(cancellationToken)
        .Run(_identifyMarkdownFromPath)
        .ToList();

This operation doesn't take any parameters because it attempts to “do the right thing” with a minimal amount of effort. Running the code now produces this:

[00:46:43 INF] <PagesPipeline> Entity: Path /index.md, Components ["MfGames.Nitride.Markdown.IsMarkdown","Zio.UPath","MfGames.Nitride.Contents.ITextContent"]

The big things is that we have a new component, IsMarkdown, and the IBinaryContent changed to ITextComponent which gives us some additional extension methods but also indicates that the file is a text file instead of treating it as a simple binary. It still hasn't loaded the file into memory, it just switched how it is handled.

This is a separate step is because sometimes I want to keep a specific file as Markdown and not convert it to HTML. Also, there are times when I construct an Entity directly without having a valid path and I just have to add the IsHtml and ITextContent and it then acts like every other file without having to have any special rules.

Content

Entities with ITextComponent have a number of useful extension methods associated with them. (Technically, you can use these methods against any Entity class and it will try to convert a binary to text if there is a IBinaryContent and we want a ITextContent).

bool hasText = entity.HasTextContent();
string text = entity.GetTextContent();

entity.SetTextContent(stringValue);
entity.SetTextContent(stringBufferValue);

These are also stored as a ITextContent instead of string or StringBuffer. This is because the default interface is to leave the content on the disk and use Zio to retrieve it. However, as soon as SetTextContent is used, then it keeps that value in memory for the rest of the execution. This is the point where memory pressure begins to increase.

In the future, we could easily create a ITextContent implementation that writes large text files to the disk to get them out of memory. However, even my largest chapter of twenty-five thousand words doesn't create too much of a problem so I haven't bothered trying to implement that at this point (but if I did, it would go into //.cache in some manner).

Converting Markdown to HTML

Just identifying a file as Markdown doesn't do anything in itself. To convert it, we use another operation, MfGames.Nitride.Markdown.ConvertMarkdownToHtml. This easily goes into the StyleHtmlPipeline to handle the conversion and styling. It allows any MarkDig extension or plugin to be called as part of the setup, allow one to customize exactly how the output is generated.

We also want to change the extension from .md to .html. I realize I should have baked that logic into the ConvertMarkdownToHtml since it is a common operation, which I will do, but for now, we also need to use the MfGames.Nitride.IO.Paths.ChangePathExtension to do that.

public StyleHtmlPipeline(
    ILogger<StyleHtmlPipeline> logger,
    SimplifiedMarkdownPipeline simplifiedMarkdownPipeline,
    ConvertMarkdownToHtml convertMarkdownToHtml,
    ChangePathExtension changePathExtension)
{
    _logger = logger;
    _changePathExtension = changePathExtension.WithExtension(".html");

    _convertMarkdownToHtml = convertMarkdownToHtml
        .WithConfigureMarkdown(builder =>
        {
            SmartyPantOptions smartyPantOptions = new();
            SmartyPantsExtension smartyPants = new(smartyPantOptions);

            builder
                .Use<GenericAttributesExtension>()
                .Use(smartyPants);
        });

    AddDependency(simplifiedMarkdownPipeline);
}

public override IAsyncEnumerable<Entity> RunAsync(
    IEnumerable<Entity> entities,
    CancellationToken cancellationToken = default)
{
    var list = entities
        .Run(_convertMarkdownToHtml, cancellationToken)
        .Run(_changePathExtension, cancellationToken)
        .ToList();

    _logger.LogInformation("Reading {Count:N0} entities", list.Count);

    return list.ToAsyncEnumerable();
}

Running this gets us:

$ just build
[01:00:43 INF] <PagesPipeline> Read in 1 files from /src/pages
[01:00:43 INF] <PagesPipeline> Entity: Path /index.md, Components ["MfGames.Nitride.Markdown.IsMarkdown","Zio.UPath","MfGames.Nitride.Contents.ITextContent"]
[01:00:43 INF] <SimplifiedMarkdownPipeline> Reading 1 entities
[01:00:44 INF] <StyleHtmlPipeline> Reading 1 entities
[01:00:44 INF] <OutputHtmlPipeline> Writing out 1 files
[01:00:44 INF] <OutputHtmlPipeline> Entity: Path /build/typewriter/html/index.html, Components ["MfGames.Nitride.Html.IsHtml","Zio.UPath","MfGames.Nitride.Contents.ITextContent"]
$ find build -type f
build/typewriter/html/index.html
$ cat src/pages/typewriter/index.md 
# Typewriter Press
$ cat build/typewriter/html/index.html
cat build/typewriter/html/index.html
<h1>Typewriter Press</h1>

And now our over-engineered copy method has duplicated markdown2html for a single file.

Components

You may notice that IsMarkdown has been removed and IsHtml was added. This is part of where I struggled with Statiq and lead me down the path of using components. I don't have to pre-define the different data types, purposes, or even formats of a file. Enums are great, but they don't allow easy extension but with an ECS, it's just a matter of adding and removing components based on the use.

I've used components for a lot of things including identifying pages that should be in the blog archives, special notices, or pages that I want to ignore because they are aliases. I also embed indexes and lists into the pages to allow things like the “next” or “previous” links. If I was doing a web comic, I could have a per-character next/previous system easily implemented via those components.

My other static site generators didn't even have the content type tagging Statiq did, which was a novel concept for me and one that I'm glad I had a chance to use. It simplified a lot of my logic and lead nicely into where I am today.

Planning Ahead

As I'm planning ahead, I'm going to do the following change:

Rename SimplifiedMarkdownPipeline to ContentPipeline
Move the logic I just added into a new pipeline called BareHtmlPipeline and insert it into between ContentPipeline and StyleHtmlPipeline

The reason for this is because RSS/Atom feeds use bare HTML to generate their content, so it makes sense to have that bare pipeline feed both of them while having the ContentPipeline handle a lot of the linking and references that we'll need.

Directory Paths

One last thing for this post: I prefer paths that end in directory slashes instead of files. So, if create a contact page at //src/pages/content.md, we want the HTML to be at https://typewriter.press/content/. This is, creatively enough, another operation: MfGames.Nitride.IO.Paths.MoveToIndexPath.

// In //src/dotnet/Pipelines/Inputs/PagesPipeline.cs
public PagesPipeline(
    ILogger<PagesPipeline> logger,
    ReadFiles readFiles,
    IdentifyMarkdownFromPath identifyMarkdownFromPath,
    MoveToIndexPath moveToIndexPath)
{
    _logger = logger;
    _identifyMarkdownFromPath = identifyMarkdownFromPath;
    _moveToIndexPath = moveToIndexPath;

    _readFiles = readFiles
        .WithPattern("/src/pages/typewriter/**/*.md")
        .WithRemovePathPrefix("/src/pages/typewriter");
}

public override IAsyncEnumerable<Entity> RunAsync(
    IEnumerable<Entity> entities,
    CancellationToken cancellationToken = default)
{
    var list = _readFiles
        .Run(cancellationToken)
        .Run(_identifyMarkdownFromPath)
        .Run(_moveToIndexPath)
        .ToList();

And we add a contact.md page which a run gives us this:

$ just build
[01:12:26 INF] <PagesPipeline> Entity: Path /contact/index.md, Components ["MfGames.Nitride.Contents.ITextContent","MfGames.Nitride.Markdown.IsMarkdown","Zio.UPath"]
[01:12:26 INF] <PagesPipeline> Entity: Path /index.md, Components ["MfGames.Nitride.Contents.ITextContent","MfGames.Nitride.Markdown.IsMarkdown","Zio.UPath"]
[01:12:26 INF] <ContentPipeline> Reading 2 entities
[01:12:26 INF] <BareHtmlPipeline> Reading 2 entities
[01:12:26 INF] <StyleHtmlPipeline> Reading 2 entities
[01:12:26 INF] <OutputHtmlPipeline> Writing out 2 files
[01:12:26 INF] <OutputHtmlPipeline> Entity: Path /build/typewriter/html/contact/index.html, Components ["MfGames.Nitride.Contents.ITextContent","MfGames.Nitride.Html.IsHtml","Zio.UPath"]
[01:12:26 INF] <OutputHtmlPipeline> Entity: Path /build/typewriter/html/index.html, Components ["MfGames.Nitride.Contents.ITextContent","MfGames.Nitride.Html.IsHtml","Zio.UPath"]
$ cat src/pages/typewriter/contact.md 
# Contact Us

Our emails is [contact@typewriter.press](mailto:contact@typewriter.press).
$ cat build/typewriter/html/contact/index.html 
<h1>Contact Us</h1>
<p>Our emails is <a href="mailto:contact@typewriter.press">contact@typewriter.press</a>.</p>

As you can see, the input is //src/pages/contact.md, but through the pipeline, it is written out as //build/typewriter/html/contact/index.html.

What's Next

Next up, handling front matter. We need that for many reasons, but one of the biggest reason is to provide metadata to generate styled HTML output.

2025-06-07 Using Nitride - Introduction - I'm going to start a long, erratic journey to update my publisher website. Along the way, I'm going to document how to create a MfGames.Nitride website from the ground up, along with a lot of little asides about reasoning or purpose.
2025-06-08 Using Nitride - Pipelines - The first major concept of Nitride is the idea of a "pipeline" which are the largest units of working in the system. This shows a relatively complex setup of pipelines that will be used to generate the website.
2025-06-09 Using Nitride - Entities - The continued adventures of creating the Typewriter website using MfGames.Nitride, a C# static site generator. The topic of today is the Entity class and how Nitride uses an ECS (Entity-Component-System) to generate pages.
2025-06-10 Using Nitride - Markdown - Examples and explanations of converting Markdown to HTML using MfGames.Nitride and MarkDig.
2025-06-12 Using Nitride - Front Matter - How to use Nitride to take the front matter from pages and add them as a component into the Entity class.

Using Nitride - Entities

2025-06-09T05:00:00Z

From yesterday's post, there was an dangling topic, the Entity class. Unlike most of the static site generators I've worked with, MfGames.Nitride uses an ECS (Entity Component System) to generate its files.

Series

2025-06-07 Using Nitride - Introduction - I'm going to start a long, erratic journey to update my publisher website. Along the way, I'm going to document how to create a MfGames.Nitride website from the ground up, along with a lot of little asides about reasoning or purpose.
2025-06-08 Using Nitride - Pipelines - The first major concept of Nitride is the idea of a "pipeline" which are the largest units of working in the system. This shows a relatively complex setup of pipelines that will be used to generate the website.
2025-06-09 Using Nitride - Entities - The continued adventures of creating the Typewriter website using MfGames.Nitride, a C# static site generator. The topic of today is the Entity class and how Nitride uses an ECS (Entity-Component-System) to generate pages.
2025-06-10 Using Nitride - Markdown - Examples and explanations of converting Markdown to HTML using MfGames.Nitride and MarkDig.
2025-06-12 Using Nitride - Front Matter - How to use Nitride to take the front matter from pages and add them as a component into the Entity class.

History

I've worked on a number of static site generators over the years, including more than a few adhoc ones until I finally got something stable with CobblestoneJS which carried me for almost a decade of heavy use. A lot of the patterns that Nitride was based on were on how I found I had to build my pages in Cobblestone, including having a separate “gather” step which wrote out temporary files and then the “process” step which formatted them and made everything pretty. I had to do that because I wanted to have cross-linking and references work across multiple Git projects.

A good example is the Fedran website. At the bottom of a chapter, I have a list of characters and locations in the chapter. The links are built from data from the wiki project but the chapter information comes from the story Git repository. The gather step let me combine both of those together to generate those links.

That also meant that I needed to have some supplementary information stored in the gather step (which was written to a //build folder) to help with those linking.

Near the end, once Cobblestone began to break under its abstraction and complexity, I wanted to get away from writing my own and switched to GatsbyJS which made a very pretty website, but it was painful to debug and develop. It only lasted a year or so before I wanted to move (though the Gatsby version of Fedran was probably the prettiest I had ever made). If anything, Gatsby ended up being an ecosystem that needed me to pay attention to update things and my cycle of coming around to update a project was too far for it so I spent days trying to get it working again to build it.

So I tried out Statiq. I never successfully made a website out of it, but it introduced me to some really interesting concepts including pulling in data from other sources (Gatsby also did this, just never really used it). But with the C#, it had some really interesting problems. However, I quickly found that I was trying to do something that was outside of the planned usage, which involved me trying to extend an enumeration which involved basically rewriting large hunks of the system just to handle it. Plus, I really like Autofac and I could never get it to play well with that.

Which lead me into creating Nitride.

Now, I want to say, both Gatsby and Statiq are both good systems. They just aren't good systems for me. These don't do what I want them to do and they are designed around patterns that I'm not comfortable with. Even with the desire not to write my own, it wasn't enough.

Entity Component Systems

Nitride is build on MfGames.Gallium which basically means the formal name of the system Gallium Nitride:

Gallium nitride (GaN) is a binary III/V direct bandgap semiconductor commonly used in blue light-emitting diodes since the 1990s. The compound is a very hard material that has a Wurtzite crystal structure. Its wide band gap of 3.4 eV affords it special properties for applications in optoelectronics, high-power and high-frequency devices. For example, GaN is the substrate that makes violet (405 nm) laser diodes possible, without requiring nonlinear optical frequency doubling.

Gallium is a small ECS library. It doesn't intend to be fast or efficient, but it was written to match the idiomatic patterns of System.Linq. I wanted something that wasn't going to reinvent the wheel, which also meant not making custom collection classes or lookup tables. Instead, I wanted to work with BCL primitives which is why much of the system is based on IEnumerable<Entity>.

There is some high-level documentation for Gallium on its project page and its test, but I'll give the basics here.

The core object is MfGames.Gallium.Entity. This is a small class that has an integer identifier and basically a Dictionary<Type, object> called Components. It has some methods to make it easier, but basically you can set or get components from it:

// This creates an empty entity.
Entity entity1 = new();

// This creates an entity with four components:
//   "System.Int32": 123
//   "System.String": "bob"
//   "System.IO.FileInfo": ...
//   "System.IO.DirectoryInfo": ...
Entity entity2 = new()
    .Set(123)
    .Set("bob")
    .SetAll(
        new FileInfo("/tmp/bob.txt"),
        new DirectoryInfo("/tmp")
    );

Assert.True(entity2.HasComponent<FileInfo>);
Assert.False(entity2.HasComponent<decimal>);

Assert.True(123, entity2.Get<int>());

In effect, the class of the component is the key. There are a number of other functions, the test class is probably the best documentation at this point.

The systems part of the system is the LINQ-inspired collection classes. Most of them work off IEnumerable<Entity> with the test class being the bulk of the documentation:

// Set up a list of entities.
List<Entity> list = ...;

// This gets all the entities that have a FileInfo.
var fileOnlyList = list.WhereEntityHas<FileInfo>().ToList();

// This gets all entities that have both a FileInfo and an int.
var fileAndIntList = list.WhereEntityHasAll<FileInfo, int>().ToList();

// Get a modified list based on components. This looks for all
// entities that have a FileInfo and then adds a marker component,
// IsHtml, into it.
IsHtml isHtml = IsHtml.Instance;

var modifiedList = list
    .SelectEntity<FileInfo>((entity, file) => entity.Set(isHtml))
    .ToList();

As a note, Entity is “mostly” immutable. The Set command will return a new Entity instance with the same ID but with a different set of components. The “mostly” bit comes that the objects that it points to can be mutated.

🛈 The lack of formal documentation for Gallium and Nitride is one of the reasons this is still alpha or beta level code.

Operations

The next concept needed for today's post is an operation which extends MfGames.Nitride.IOperation. These are just easy ways of creating some functionality that works on an IEnumerable<Entity> that can be combined together.

Naturally, there are some extension methods that make it easier to work with operations with an IEnumerable<Entity>:

IEnumerable<Entity> entityList = ...;

IEnumerable<Entity> otherList = someOperation1.Run(entityList);

IEnumerable<Entity> resultList = otherList
    .Run(someOperation2)
    .Run(someOperation3)
    .Run(someOperation4);

This makes it relatively easy to create easy processing through composition of one or more operations. Alternatively, we could easily just work directly off the enumerable.

Zio

There is no question, I'm fond of the Zio library. In this case, Nitride uses two big features with it: to create an abstract view of the project that is independent of the working directory and the UPath class which normalizes paths.

If you remember from the previous posts, we set a root directory in the Program.cs:

public static async Task<int> Main(string[] args)
{
    var rootDirectory = new DirectoryInfo(Environment.CurrentDirectory);

    var builder = new NitrideBuilder(args)
        .UseIO(rootDirectory)
        .UseModule<WebsiteModule>();

    return await builder.RunAsync();
}

This caused an Zio.IFileSystem to be injected in the system which treats that directory as the /. So, in our case where we had our index file in //src/pages/index.md, it has a UPath of /src/pages/index.md. It doesn't seem like a lot, but it means that all paths in the system are always relative to the root of the project and there is no need to pass a root directory throughout the system which means path operations are considerably simplified.

And the UPath ensures a consistent pattern for accessing so this works on Windows, Mac, and Linux without a problem. Not to mention, any issues with .. are handled automatically.

It also simplifies writing tests.

Copy File in Six Files

Put all that together, we can use two operations to create a massively over-engineered copy file operation using Nitride. Admittedly, the complexity will make sense later, but at the moment, we're going to read the file in the PagesPipeline, through SimplifiedMarkdownPipeline, through StyleHtmlPipeline, and finally write it out in OutputHtmlPipeline.

Making Some Noise

To start with, let's put some logging to show how everything flows from pipeline to pipeline. To do that, we're going to inject an ILogger<> into the constructor into all four pipelines and then put a logging message into the last three (the PagesPipeline is going to do something different).

// In //src/dotnet/Pipelines/Content/SimplifiedMarkdownPipeline.cs
public class SimplifiedMarkdownPipeline : PipelineBase
{
    private readonly ILogger _logger;

    public SimplifiedMarkdownPipeline(
        ILogger<SimplifiedMarkdownPipeline> logger,
        PagesPipeline pagesPipeline)
    {
        _logger = logger;

        AddDependency(pagesPipeline);
    }

    public override IAsyncEnumerable<Entity> RunAsync(
        IEnumerable<Entity> entities,
        CancellationToken cancellationToken = default)
    {
        var list = entities.ToList();

        _logger.LogInformation("Reading {Count:N0} entities", list.Count);

        return list.ToAsyncEnumerable();
    }
}

Internally, Nitride uses Serilog, so using {Count:N0} does the right thing even thought we are using a logging abstraction for the actual call. When we run this, we get the following output (I'm not going to show the just build call for these today):

[14:00:46 DBG] <DirectoryService> Setting root directory and filesystem to /home/dmoonfire/src/typewriter-press/typewriter-press-website
[14:00:46 DBG] <BuildCommand> Processing 0 pipeline options
[14:00:46 INF] <BuildCommand> Running pipelines
[14:00:46 INF] <SimplifiedMarkdownPipeline> Reading 0 entities
[14:00:46 INF] <StyleHtmlPipeline> Reading 0 entities
[14:00:46 INF] <OutputHtmlPipeline> Reading 0 entities
[14:00:46 INF] <PipelineManager> Completed in 00:00:00.1079413
[14:00:46 INF] <BuildCommand> Command took 00:00:00.1097142

🛈 I moved NuGet.config to the root level because there is where the Website.sln file was located and it felt right.

Reading Files

Now, to read a file, we are going to use the MfGames.Nitride.IO.ReadFiles operation which reads file from the Zio file system and passes them through. We've already included that NuGet package in a previous step, so nothing to do there.

// In //src/dotnet/Inputs/PagesPipeline.cs
using MfGames.Gallium;
using MfGames.Nitride.IO.Contents;
using MfGames.Nitride.Pipelines;
using Microsoft.Extensions.Logging;

namespace Generator.Pipelines.Inputs;

public class PagesPipeline : PipelineBase
{
    private readonly ILogger _logger;
    private readonly ReadFiles _readFiles;

    public PagesPipeline(
        ILogger<PagesPipeline> logger,
        ReadFiles readFiles)
    {
        _logger = logger;
        _readFiles = readFiles.WithPattern("/src/pages/**/*.md");
    }

    public override IAsyncEnumerable<Entity> RunAsync(
        IEnumerable<Entity> entities,
        CancellationToken cancellationToken = default)
    {
        var list = _readFiles
            .Run(cancellationToken)
            .ToList();

        _logger.LogInformation(
            "Read in {Count:N0} files from /src/pages",
            list.Count);

        foreach (var entity in list)
        {
            _logger.LogInformation("Entity: Path {Path}, Components {ComponentList}",
                entity.Get<UPath>(),
                entity.GetComponentTypes().Select(x => x.FullName));
        }

        return list.ToAsyncEnumerable();
    }
}

This requires a bit of explanation. We are using dependency injection to insert the ReadFiles operation into the constructor. This is not a singleton, so you can have multiple ReadFiles if you need to read from multiple locations (or use a globbing operator to do it in one).

_readFiles = readFiles.WithPattern("/src/pages/**/*.md");

In the constructor, I configure it. I could do this in the RunAsync function but I found it made it easier to parse the files when I configure in the constructor. Like most things in Nitride, this uses a fluent coding style where each object sets the properties and returns itself to make a nice chained list of calls.

There are two properties being set, Pattern and RemovePathPrefix. I use a source generator to automatically create the With... pattern for fluent coding. The first one tells which files to read, relative to the root directory. This happen to use a globbing that allows me to grab all Markdown files in the source directory.

var list = _readFiles
    .Run(cancellationToken)
    .ToList();

This run the _readFiles operation and gets a list of the files it read in. (I'm also going to put this in the OutputHtmlPipeline to show how everything writes out.)

foreach (var entity in list)
{
    _logger.LogInformation(
        "Entity: Path {Path}, Components {ComponentList}",
        entity.Get<UPath>(),
        entity.GetComponentTypes().Select(x => x.FullName));
}

This is just some debugging to show how things are working and how Gallium works. In this case, the read files sets two components: the UPath for the path of the file and a IBinaryContent which is an abstraction to the binary content. Since we haven't done anything to the file, this just points to the file system itself without loading it into memory. That way, we can easily deal with gigabyte-sized files without overloading the system.

return list.ToAsyncEnumerable();

With this, we return the list of files that we just read in. With this change, running the build gets us this:

[14:20:03 DBG] <DirectoryService> Setting root directory and filesystem to /home/dmoonfire/src/typewriter-press/typewriter-press-website
[14:20:03 DBG] <BuildCommand> Processing 0 pipeline options
[14:20:03 INF] <BuildCommand> Running pipelines
[14:20:04 INF] <PagesPipeline> Read in 1 files from /src/pages
[14:20:04 INF] <PagesPipeline> Entity: Path /src/pages/typewriter/index.md, Components ["MfGames.Nitride.Contents.IBinaryContent","Zio.UPath"]
[14:20:04 INF] <SimplifiedMarkdownPipeline> Reading 1 entities
[14:20:04 INF] <StyleHtmlPipeline> Reading 1 entities
[14:20:04 INF] <OutputHtmlPipeline> Writing out 1 files
[14:20:04 INF] <OutputHtmlPipeline> Entity: Path /src/pages/typewriter/index.md, Components ["MfGames.Nitride.Contents.IBinaryContent","Zio.UPath"]
[14:20:04 INF] <PipelineManager> Completed in 00:00:00.9543639
[14:20:04 INF] <BuildCommand> Command took 00:00:00.9559424

As you can see, that single file read in is then passed through the other pipelines because each of them return their input list and then the final pipeline gets those same entities and paths.

Changing Paths

Now the problem with the above example is that we are writing our output to the same location as we are reading. That isn't optional. What we want to do is strip off the /src/pages/typewriter from the beginning of the read in path and add /build/typewriter/html in front of it. This is easily done with two additional operators, RemovePathPrefix and AddPathPrefix.

To start with, let's remove the prefix from the read methods to normalize the paths.

// In //src/dotnet/Inputs/PagesPipeline.cs
public PagesPipeline(
    ILogger<PagesPipeline> logger,
    ReadFiles readFiles,
    RemovePathPrefix removePathPrefix)
{
    _logger = logger;
    _removePathPrefix = removePathPrefix.WithPathPrefix("/src/pages/typewriter");
    _readFiles = readFiles.WithPattern("/src/pages/**/*.md");
}

And then we chain the operation using an extension method on operations:

// In //src/dotnet/Inputs/PagesPipeline.cs
public override IAsyncEnumerable<Entity> RunAsync(
    IEnumerable<Entity> entities,
    CancellationToken cancellationToken = default)
{
    var list = _readFiles
        .Run(cancellationToken)
        .Run(_removePathPrefix)
        .ToList();

Since each operation also takes an IEnumerable<Entity> and removes the same type, they can flow from one operation to another. When we run the build, the output line looks like this:

[14:23:35 INF] <PagesPipeline> Read in 1 files from /src/pages
[14:23:35 INF] <PagesPipeline> Entity: Path /index.md, Components ["MfGames.Nitride.Contents.IBinaryContent","Zio.UPath"]

Adding the output prefix is done on the output step:

// In //src/dotnet/Html/OutputHtmlPipeline.cs
public OutputHtmlPipeline(
    ILogger<OutputHtmlPipeline> logger,
    AddPathPrefix addPathPrefix,
    StyleHtmlPipeline styleHtmlPipeline)
{
    _logger = logger;
    _addPathPrefix = addPathPrefix.WithPathPrefix("/build/typewriter/html");

    AddDependency(styleHtmlPipeline);
}

public override IAsyncEnumerable<Entity> RunAsync(
    IEnumerable<Entity> entities,
    CancellationToken cancellationToken = default)
{
    var list = entities
        .Run(_addPathPrefix)
        .ToList();

Because we already have a list of entities, we can just use the extension method to pipe it through the operation and produce the following output:

[14:28:35 INF] <PagesPipeline> Read in 1 files from /src/pages
[14:28:35 INF] <PagesPipeline> Entity: Path /index.md, Components ["MfGames.Nitride.Contents.IBinaryContent","Zio.UPath"]
[14:28:35 INF] <SimplifiedMarkdownPipeline> Reading 1 entities
[14:28:35 INF] <StyleHtmlPipeline> Reading 1 entities
[14:28:35 INF] <OutputHtmlPipeline> Writing out 1 files
[14:28:35 INF] <OutputHtmlPipeline> Entity: Path /build/typewriter/html/index.md, Components ["MfGames.Nitride.Contents.IBinaryContent","Zio.UPath"]

The reason PagesPipeline doesn't use entities.Run(...) is because that is the first pipeline and the entities is going to be an empty list. Instead, we need to make entities, which is why we call _readFiles.Run() instead.

Writing Files

And the last bit for today's post is to write out files. This, creatively enough, uses the WriteFiles operation.

// In //src/dotnet/Html/OutputHtmlPipeline.cs
public OutputHtmlPipeline(
    ILogger<OutputHtmlPipeline> logger,
    AddPathPrefix addPathPrefix,
    WriteFiles writeFiles,
    StyleHtmlPipeline styleHtmlPipeline)
{
    _logger = logger;
    _writeFiles = writeFiles;
    _addPathPrefix = addPathPrefix.WithPathPrefix("/build/typewriter/html");

    AddDependency(styleHtmlPipeline);
}

public override IAsyncEnumerable<Entity> RunAsync(
    IEnumerable<Entity> entities,
    CancellationToken cancellationToken = default)
{
    var list = entities
        .Run(_addPathPrefix)
        .Run(_writeFiles)
        .ToList();

In this case, we've already done all the fancy operations to adjust the path so it writes out the files in the proper location:

$ find build -type f
build/typewriter/html/index.md

Convenience Methods

Now, adding and removing path prefixes is pretty common, so both the ReadFiles and WriteFiles have methods to do those in a single call.

// In //src/dotnet/Inputs/PagesPipeline.cs
public PagesPipeline(
    ILogger<PagesPipeline> logger,
    ReadFiles readFiles)
{
    _logger = logger;
    _readFiles = readFiles
        .WithPattern("/src/pages/typewriter/**/*.md")
        .WithRemovePathPrefix("/src/pages/typewriter");
}

// In //src/dotnet/Html/OutputHtmlPipeline.cs
public OutputHtmlPipeline(
    ILogger<OutputHtmlPipeline> logger,
    WriteFiles writeFiles,
    StyleHtmlPipeline styleHtmlPipeline)
{
    _logger = logger;
    _writeFiles = writeFiles.WithAddPathPrefix("/build/typewriter/html");

    AddDependency(styleHtmlPipeline);
}

What's Next

And there you go, an over-engineered system for copying a single file from one directory to another. In the next post of this series, we'll turn that Markdown file into an HTML page.

2025-06-07 Using Nitride - Introduction - I'm going to start a long, erratic journey to update my publisher website. Along the way, I'm going to document how to create a MfGames.Nitride website from the ground up, along with a lot of little asides about reasoning or purpose.
2025-06-08 Using Nitride - Pipelines - The first major concept of Nitride is the idea of a "pipeline" which are the largest units of working in the system. This shows a relatively complex setup of pipelines that will be used to generate the website.
2025-06-09 Using Nitride - Entities - The continued adventures of creating the Typewriter website using MfGames.Nitride, a C# static site generator. The topic of today is the Entity class and how Nitride uses an ECS (Entity-Component-System) to generate pages.
2025-06-10 Using Nitride - Markdown - Examples and explanations of converting Markdown to HTML using MfGames.Nitride and MarkDig.
2025-06-12 Using Nitride - Front Matter - How to use Nitride to take the front matter from pages and add them as a component into the Entity class.