Using Nitride - Entities

From yesterday's post, there was an dangling topic, the Entity class. Unlike most of the static site generators I've worked with, MfGames.Nitride uses an ECS (Entity Component System) to generate its files.

Series

  • 2025-06-07 Using Nitride - Introduction - I'm going to start a long, erratic journey to update my publisher website. Along the way, I'm going to document how to create a MfGames.Nitride website from the ground up, along with a lot of little asides about reasoning or purpose.

  • 2025-06-08 Using Nitride - Pipelines - The first major concept of Nitride is the idea of a "pipeline" which are the largest units of working in the system. This shows a relatively complex setup of pipelines that will be used to generate the website.

  • 2025-06-09 Using Nitride - Entities - The continued adventures of creating the Typewriter website using MfGames.Nitride, a C# static site generator. The topic of today is the Entity class and how Nitride uses an ECS (Entity-Component-System) to generate pages.

  • 2025-06-10 Using Nitride - Markdown - Examples and explanations of converting Markdown to HTML using MfGames.Nitride and MarkDig.

  • 2025-06-12 Using Nitride - Front Matter - How to use Nitride to take the front matter from pages and add them as a component into the Entity class.

History

I've worked on a number of static site generators over the years, including more than a few adhoc ones until I finally got something stable with CobblestoneJS which carried me for almost a decade of heavy use. A lot of the patterns that Nitride was based on were on how I found I had to build my pages in Cobblestone, including having a separate “gather” step which wrote out temporary files and then the “process” step which formatted them and made everything pretty. I had to do that because I wanted to have cross-linking and references work across multiple Git projects.

A good example is the Fedran website. At the bottom of a chapter, I have a list of characters and locations in the chapter. The links are built from data from the wiki project but the chapter information comes from the story Git repository. The gather step let me combine both of those together to generate those links.

That also meant that I needed to have some supplementary information stored in the gather step (which was written to a //build folder) to help with those linking.

Near the end, once Cobblestone began to break under its abstraction and complexity, I wanted to get away from writing my own and switched to GatsbyJS which made a very pretty website, but it was painful to debug and develop. It only lasted a year or so before I wanted to move (though the Gatsby version of Fedran was probably the prettiest I had ever made). If anything, Gatsby ended up being an ecosystem that needed me to pay attention to update things and my cycle of coming around to update a project was too far for it so I spent days trying to get it working again to build it.

So I tried out Statiq. I never successfully made a website out of it, but it introduced me to some really interesting concepts including pulling in data from other sources (Gatsby also did this, just never really used it). But with the C#, it had some really interesting problems. However, I quickly found that I was trying to do something that was outside of the planned usage, which involved me trying to extend an enumeration which involved basically rewriting large hunks of the system just to handle it. Plus, I really like Autofac and I could never get it to play well with that.

Which lead me into creating Nitride.

Now, I want to say, both Gatsby and Statiq are both good systems. They just aren't good systems for me. These don't do what I want them to do and they are designed around patterns that I'm not comfortable with. Even with the desire not to write my own, it wasn't enough.

Entity Component Systems

Nitride is build on MfGames.Gallium which basically means the formal name of the system Gallium Nitride:

Gallium nitride (GaN) is a binary III/V direct bandgap semiconductor commonly used in blue light-emitting diodes since the 1990s. The compound is a very hard material that has a Wurtzite crystal structure. Its wide band gap of 3.4 eV affords it special properties for applications in optoelectronics, high-power and high-frequency devices. For example, GaN is the substrate that makes violet (405 nm) laser diodes possible, without requiring nonlinear optical frequency doubling.

Gallium is a small ECS library. It doesn't intend to be fast or efficient, but it was written to match the idiomatic patterns of System.Linq. I wanted something that wasn't going to reinvent the wheel, which also meant not making custom collection classes or lookup tables. Instead, I wanted to work with BCL primitives which is why much of the system is based on IEnumerable<Entity>.

There is some high-level documentation for Gallium on its project page and its test, but I'll give the basics here.

The core object is MfGames.Gallium.Entity. This is a small class that has an integer identifier and basically a Dictionary<Type, object> called Components. It has some methods to make it easier, but basically you can set or get components from it:

// This creates an empty entity.
Entity entity1 = new();

// This creates an entity with four components:
//   "System.Int32": 123
//   "System.String": "bob"
//   "System.IO.FileInfo": ...
//   "System.IO.DirectoryInfo": ...
Entity entity2 = new()
    .Set(123)
    .Set("bob")
    .SetAll(
        new FileInfo("/tmp/bob.txt"),
        new DirectoryInfo("/tmp")
    );

Assert.True(entity2.HasComponent<FileInfo>);
Assert.False(entity2.HasComponent<decimal>);

Assert.True(123, entity2.Get<int>());

In effect, the class of the component is the key. There are a number of other functions, the test class is probably the best documentation at this point.

The systems part of the system is the LINQ-inspired collection classes. Most of them work off IEnumerable<Entity> with the test class being the bulk of the documentation:

// Set up a list of entities.
List<Entity> list = ...;

// This gets all the entities that have a FileInfo.
var fileOnlyList = list.WhereEntityHas<FileInfo>().ToList();

// This gets all entities that have both a FileInfo and an int.
var fileAndIntList = list.WhereEntityHasAll<FileInfo, int>().ToList();

// Get a modified list based on components. This looks for all
// entities that have a FileInfo and then adds a marker component,
// IsHtml, into it.
IsHtml isHtml = IsHtml.Instance;

var modifiedList = list
    .SelectEntity<FileInfo>((entity, file) => entity.Set(isHtml))
    .ToList();

As a note, Entity is “mostly” immutable. The Set command will return a new Entity instance with the same ID but with a different set of components. The “mostly” bit comes that the objects that it points to can be mutated.

🛈 The lack of formal documentation for Gallium and Nitride is one of the reasons this is still alpha or beta level code.

Operations

The next concept needed for today's post is an operation which extends MfGames.Nitride.IOperation. These are just easy ways of creating some functionality that works on an IEnumerable<Entity> that can be combined together.

Naturally, there are some extension methods that make it easier to work with operations with an IEnumerable<Entity>:

IEnumerable<Entity> entityList = ...;

IEnumerable<Entity> otherList = someOperation1.Run(entityList);

IEnumerable<Entity> resultList = otherList
    .Run(someOperation2)
    .Run(someOperation3)
    .Run(someOperation4);

This makes it relatively easy to create easy processing through composition of one or more operations. Alternatively, we could easily just work directly off the enumerable.

Zio

There is no question, I'm fond of the Zio library. In this case, Nitride uses two big features with it: to create an abstract view of the project that is independent of the working directory and the UPath class which normalizes paths.

If you remember from the previous posts, we set a root directory in the Program.cs:

public static async Task<int> Main(string[] args)
{
    var rootDirectory = new DirectoryInfo(Environment.CurrentDirectory);

    var builder = new NitrideBuilder(args)
        .UseIO(rootDirectory)
        .UseModule<WebsiteModule>();

    return await builder.RunAsync();
}

This caused an Zio.IFileSystem to be injected in the system which treats that directory as the /. So, in our case where we had our index file in //src/pages/index.md, it has a UPath of /src/pages/index.md. It doesn't seem like a lot, but it means that all paths in the system are always relative to the root of the project and there is no need to pass a root directory throughout the system which means path operations are considerably simplified.

And the UPath ensures a consistent pattern for accessing so this works on Windows, Mac, and Linux without a problem. Not to mention, any issues with .. are handled automatically.

It also simplifies writing tests.

Copy File in Six Files

Put all that together, we can use two operations to create a massively over-engineered copy file operation using Nitride. Admittedly, the complexity will make sense later, but at the moment, we're going to read the file in the PagesPipeline, through SimplifiedMarkdownPipeline, through StyleHtmlPipeline, and finally write it out in OutputHtmlPipeline.

Making Some Noise

To start with, let's put some logging to show how everything flows from pipeline to pipeline. To do that, we're going to inject an ILogger<> into the constructor into all four pipelines and then put a logging message into the last three (the PagesPipeline is going to do something different).

// In //src/dotnet/Pipelines/Content/SimplifiedMarkdownPipeline.cs
public class SimplifiedMarkdownPipeline : PipelineBase
{
    private readonly ILogger _logger;

    public SimplifiedMarkdownPipeline(
        ILogger<SimplifiedMarkdownPipeline> logger,
        PagesPipeline pagesPipeline)
    {
        _logger = logger;

        AddDependency(pagesPipeline);
    }

    public override IAsyncEnumerable<Entity> RunAsync(
        IEnumerable<Entity> entities,
        CancellationToken cancellationToken = default)
    {
        var list = entities.ToList();

        _logger.LogInformation("Reading {Count:N0} entities", list.Count);

        return list.ToAsyncEnumerable();
    }
}

Internally, Nitride uses Serilog, so using {Count:N0} does the right thing even thought we are using a logging abstraction for the actual call. When we run this, we get the following output (I'm not going to show the just build call for these today):

[14:00:46 DBG] <DirectoryService> Setting root directory and filesystem to /home/dmoonfire/src/typewriter-press/typewriter-press-website
[14:00:46 DBG] <BuildCommand> Processing 0 pipeline options
[14:00:46 INF] <BuildCommand> Running pipelines
[14:00:46 INF] <SimplifiedMarkdownPipeline> Reading 0 entities
[14:00:46 INF] <StyleHtmlPipeline> Reading 0 entities
[14:00:46 INF] <OutputHtmlPipeline> Reading 0 entities
[14:00:46 INF] <PipelineManager> Completed in 00:00:00.1079413
[14:00:46 INF] <BuildCommand> Command took 00:00:00.1097142

🛈 I moved NuGet.config to the root level because there is where the Website.sln file was located and it felt right.

Reading Files

Now, to read a file, we are going to use the MfGames.Nitride.IO.ReadFiles operation which reads file from the Zio file system and passes them through. We've already included that NuGet package in a previous step, so nothing to do there.

// In //src/dotnet/Inputs/PagesPipeline.cs
using MfGames.Gallium;
using MfGames.Nitride.IO.Contents;
using MfGames.Nitride.Pipelines;
using Microsoft.Extensions.Logging;

namespace Generator.Pipelines.Inputs;

public class PagesPipeline : PipelineBase
{
    private readonly ILogger _logger;
    private readonly ReadFiles _readFiles;

    public PagesPipeline(
        ILogger<PagesPipeline> logger,
        ReadFiles readFiles)
    {
        _logger = logger;
        _readFiles = readFiles.WithPattern("/src/pages/**/*.md");
    }

    public override IAsyncEnumerable<Entity> RunAsync(
        IEnumerable<Entity> entities,
        CancellationToken cancellationToken = default)
    {
        var list = _readFiles
            .Run(cancellationToken)
            .ToList();

        _logger.LogInformation(
            "Read in {Count:N0} files from /src/pages",
            list.Count);

        foreach (var entity in list)
        {
            _logger.LogInformation("Entity: Path {Path}, Components {ComponentList}",
                entity.Get<UPath>(),
                entity.GetComponentTypes().Select(x => x.FullName));
        }

        return list.ToAsyncEnumerable();
    }
}

This requires a bit of explanation. We are using dependency injection to insert the ReadFiles operation into the constructor. This is not a singleton, so you can have multiple ReadFiles if you need to read from multiple locations (or use a globbing operator to do it in one).

_readFiles = readFiles.WithPattern("/src/pages/**/*.md");

In the constructor, I configure it. I could do this in the RunAsync function but I found it made it easier to parse the files when I configure in the constructor. Like most things in Nitride, this uses a fluent coding style where each object sets the properties and returns itself to make a nice chained list of calls.

There are two properties being set, Pattern and RemovePathPrefix. I use a source generator to automatically create the With... pattern for fluent coding. The first one tells which files to read, relative to the root directory. This happen to use a globbing that allows me to grab all Markdown files in the source directory.

var list = _readFiles
    .Run(cancellationToken)
    .ToList();

This run the _readFiles operation and gets a list of the files it read in. (I'm also going to put this in the OutputHtmlPipeline to show how everything writes out.)

foreach (var entity in list)
{
    _logger.LogInformation(
        "Entity: Path {Path}, Components {ComponentList}",
        entity.Get<UPath>(),
        entity.GetComponentTypes().Select(x => x.FullName));
}

This is just some debugging to show how things are working and how Gallium works. In this case, the read files sets two components: the UPath for the path of the file and a IBinaryContent which is an abstraction to the binary content. Since we haven't done anything to the file, this just points to the file system itself without loading it into memory. That way, we can easily deal with gigabyte-sized files without overloading the system.

return list.ToAsyncEnumerable();

With this, we return the list of files that we just read in. With this change, running the build gets us this:

[14:20:03 DBG] <DirectoryService> Setting root directory and filesystem to /home/dmoonfire/src/typewriter-press/typewriter-press-website
[14:20:03 DBG] <BuildCommand> Processing 0 pipeline options
[14:20:03 INF] <BuildCommand> Running pipelines
[14:20:04 INF] <PagesPipeline> Read in 1 files from /src/pages
[14:20:04 INF] <PagesPipeline> Entity: Path /src/pages/typewriter/index.md, Components ["MfGames.Nitride.Contents.IBinaryContent","Zio.UPath"]
[14:20:04 INF] <SimplifiedMarkdownPipeline> Reading 1 entities
[14:20:04 INF] <StyleHtmlPipeline> Reading 1 entities
[14:20:04 INF] <OutputHtmlPipeline> Writing out 1 files
[14:20:04 INF] <OutputHtmlPipeline> Entity: Path /src/pages/typewriter/index.md, Components ["MfGames.Nitride.Contents.IBinaryContent","Zio.UPath"]
[14:20:04 INF] <PipelineManager> Completed in 00:00:00.9543639
[14:20:04 INF] <BuildCommand> Command took 00:00:00.9559424

As you can see, that single file read in is then passed through the other pipelines because each of them return their input list and then the final pipeline gets those same entities and paths.

Changing Paths

Now the problem with the above example is that we are writing our output to the same location as we are reading. That isn't optional. What we want to do is strip off the /src/pages/typewriter from the beginning of the read in path and add /build/typewriter/html in front of it. This is easily done with two additional operators, RemovePathPrefix and AddPathPrefix.

To start with, let's remove the prefix from the read methods to normalize the paths.

// In //src/dotnet/Inputs/PagesPipeline.cs
public PagesPipeline(
    ILogger<PagesPipeline> logger,
    ReadFiles readFiles,
    RemovePathPrefix removePathPrefix)
{
    _logger = logger;
    _removePathPrefix = removePathPrefix.WithPathPrefix("/src/pages/typewriter");
    _readFiles = readFiles.WithPattern("/src/pages/**/*.md");
}

And then we chain the operation using an extension method on operations:

// In //src/dotnet/Inputs/PagesPipeline.cs
public override IAsyncEnumerable<Entity> RunAsync(
    IEnumerable<Entity> entities,
    CancellationToken cancellationToken = default)
{
    var list = _readFiles
        .Run(cancellationToken)
        .Run(_removePathPrefix)
        .ToList();

Since each operation also takes an IEnumerable<Entity> and removes the same type, they can flow from one operation to another. When we run the build, the output line looks like this:

[14:23:35 INF] <PagesPipeline> Read in 1 files from /src/pages
[14:23:35 INF] <PagesPipeline> Entity: Path /index.md, Components ["MfGames.Nitride.Contents.IBinaryContent","Zio.UPath"]

Adding the output prefix is done on the output step:

// In //src/dotnet/Html/OutputHtmlPipeline.cs
public OutputHtmlPipeline(
    ILogger<OutputHtmlPipeline> logger,
    AddPathPrefix addPathPrefix,
    StyleHtmlPipeline styleHtmlPipeline)
{
    _logger = logger;
    _addPathPrefix = addPathPrefix.WithPathPrefix("/build/typewriter/html");

    AddDependency(styleHtmlPipeline);
}

public override IAsyncEnumerable<Entity> RunAsync(
    IEnumerable<Entity> entities,
    CancellationToken cancellationToken = default)
{
    var list = entities
        .Run(_addPathPrefix)
        .ToList();

Because we already have a list of entities, we can just use the extension method to pipe it through the operation and produce the following output:

[14:28:35 INF] <PagesPipeline> Read in 1 files from /src/pages
[14:28:35 INF] <PagesPipeline> Entity: Path /index.md, Components ["MfGames.Nitride.Contents.IBinaryContent","Zio.UPath"]
[14:28:35 INF] <SimplifiedMarkdownPipeline> Reading 1 entities
[14:28:35 INF] <StyleHtmlPipeline> Reading 1 entities
[14:28:35 INF] <OutputHtmlPipeline> Writing out 1 files
[14:28:35 INF] <OutputHtmlPipeline> Entity: Path /build/typewriter/html/index.md, Components ["MfGames.Nitride.Contents.IBinaryContent","Zio.UPath"]

The reason PagesPipeline doesn't use entities.Run(...) is because that is the first pipeline and the entities is going to be an empty list. Instead, we need to make entities, which is why we call _readFiles.Run() instead.

Writing Files

And the last bit for today's post is to write out files. This, creatively enough, uses the WriteFiles operation.

// In //src/dotnet/Html/OutputHtmlPipeline.cs
public OutputHtmlPipeline(
    ILogger<OutputHtmlPipeline> logger,
    AddPathPrefix addPathPrefix,
    WriteFiles writeFiles,
    StyleHtmlPipeline styleHtmlPipeline)
{
    _logger = logger;
    _writeFiles = writeFiles;
    _addPathPrefix = addPathPrefix.WithPathPrefix("/build/typewriter/html");

    AddDependency(styleHtmlPipeline);
}

public override IAsyncEnumerable<Entity> RunAsync(
    IEnumerable<Entity> entities,
    CancellationToken cancellationToken = default)
{
    var list = entities
        .Run(_addPathPrefix)
        .Run(_writeFiles)
        .ToList();

In this case, we've already done all the fancy operations to adjust the path so it writes out the files in the proper location:

$ find build -type f
build/typewriter/html/index.md

Convenience Methods

Now, adding and removing path prefixes is pretty common, so both the ReadFiles and WriteFiles have methods to do those in a single call.

// In //src/dotnet/Inputs/PagesPipeline.cs
public PagesPipeline(
    ILogger<PagesPipeline> logger,
    ReadFiles readFiles)
{
    _logger = logger;
    _readFiles = readFiles
        .WithPattern("/src/pages/typewriter/**/*.md")
        .WithRemovePathPrefix("/src/pages/typewriter");
}

// In //src/dotnet/Html/OutputHtmlPipeline.cs
public OutputHtmlPipeline(
    ILogger<OutputHtmlPipeline> logger,
    WriteFiles writeFiles,
    StyleHtmlPipeline styleHtmlPipeline)
{
    _logger = logger;
    _writeFiles = writeFiles.WithAddPathPrefix("/build/typewriter/html");

    AddDependency(styleHtmlPipeline);
}

What's Next

And there you go, an over-engineered system for copying a single file from one directory to another. In the next post of this series, we'll turn that Markdown file into an HTML page.

  • 2025-06-07 Using Nitride - Introduction - I'm going to start a long, erratic journey to update my publisher website. Along the way, I'm going to document how to create a MfGames.Nitride website from the ground up, along with a lot of little asides about reasoning or purpose.

  • 2025-06-08 Using Nitride - Pipelines - The first major concept of Nitride is the idea of a "pipeline" which are the largest units of working in the system. This shows a relatively complex setup of pipelines that will be used to generate the website.

  • 2025-06-09 Using Nitride - Entities - The continued adventures of creating the Typewriter website using MfGames.Nitride, a C# static site generator. The topic of today is the Entity class and how Nitride uses an ECS (Entity-Component-System) to generate pages.

  • 2025-06-10 Using Nitride - Markdown - Examples and explanations of converting Markdown to HTML using MfGames.Nitride and MarkDig.

  • 2025-06-12 Using Nitride - Front Matter - How to use Nitride to take the front matter from pages and add them as a component into the Entity class.

Metadata

Categories:

Tags: