D. Moonfire

Leicmin

2026-04-04T05:00:00Z

I haven't posted in three months. That isn't to nothing has been happening, only that I was so focused on the “now” that I didn't really have a chance for any retrospection (which is effectively what a lot of my posts are) or idle thoughts (which cover most of the rest). Part of that is how my thoughts work, I withdraw from everything when I'm struggling with problems.

There is always the usual family crises (we lost a family member last week and there is a good chance we're going to lose a close one “soon”), work pressure, and the difficulties of being a parent.

Age Verification

On top of that, the various news about age verification laws, the casual tossing around of felonies and crippling monetary fines, and the rise of privacy-busting legislation were hitting my communities. Too many legislation were salivating at the openings that the Texas got with the Supreme Court ruling, the idea of forcing all cases to be dragged into the state with the most restrictions, and generally finding Machiavellian ways to force creators from closing up shop without explicitly saying that porn was illegal (because it isn't).

I know Fedran is “mostly” excluded from most of those age verification laws. There are a million words on the site and less than 10% of them are what I would consider “adult” or graphic. I could easily say it isn't pornographic, but then Tennessee's Protect Tennessee Minors Act got put into effect and it appears to apply to my fiction.

It was that law that triggered a “fight or flight” response. I seriously considered just closing down the shop and giving up writing. I was already in the middle of the burnout from my 300 consecutive week challenge and my struggles after Covid. It felt like an easy escape, just cut off that part of my life.

But then I didn't. I'm terrible at giving up, I mean that was pretty much the major thread throughout Sand and Blood, Sand and Ash, and Sand and Bone. That part is part of me. I might take a while to get there, but I don't give up easily. I also saw other creators who were also struggling with the same problems, but they were clearly in the arenas the laws were targeting. Creators who were writing things that were more erotic (it's never about violence, which I universally consider more disgusting than sexuality).

So the flight (giving up) turned into fight (doing something). I already knew that I wasn't going to be great about anything I wrote, but it was something I could do. I have to be honest, I didn't come out of Covid unscathed and I'm painfully aware of how much I struggle with things that I found pleasurable less than ten years ago.

Also, quite a few years before all this, I had an idea of creating a self-hosted site that could coordinate various subscription/patron services. Basically, taking the supporter badges from forums, the ability to have Patreon or Ko-Fi supporters (or for a while, cryptocurrency), and be able to control access to sites. Ideally, in a manner that would not require exposing personal information (because no one needs to know those details). And no gamification or social interactions, just something between a creator and someone who likes their work.

That also would give me the ability to integrate age verification systems or even geo-blocking or VPN-blocking (there were a few bills that required periodic verification of age or the need to ban all VPN usage) in a central place.

Would it be helpful? Maybe. But if anything, I could implement something to protect my ass and maybe it would help others.

Leicmin

And that ended up being Leicmin (more detail on the development site and the forge).

Well, multiple iterations of Leicmin. I started with trying to implement a C# version since I thought I was really good at the language and it would be “easy” to bang up something.

I didn't do it because it was easy. I did it because I thought it was easy.

After a few months, I realized that I had too many things rattling in my head, and the ideas that I toyed with for years thinking they would make a great system (event sourcing, automated auditing, API-first) ended up crumbling once I started to do significant things. I spent more time fighting libraries and tools than I was moving forward.

Somewhere in December, I realized it wasn't going to work out so I decided to burn it down and start from scratch. This time, I jettisoned most of the fancy ideas and went with something simple: a website that required no Javascript, had no API, and was about as plain and simple as I could make it. I also decided to go with Rust, but avoided the fancier front ends and went with straight, old-school templating.

There were some things that I wasn't going to get rid of: there are type-safe identifiers everywhere, almost everything is CLI first, it is based on Postgres, and there are almost no general solutions anywhere.

And I got progress. I could see things moving forward and I didn't feel like I was writing myself into a corner. True, the C# attempt taught me a lot, but it was really nice seeing slow but steady progress to something that works. I felt bad that it wasn't “fancy” or more capable but I was happy that I was creating something I thought could benefits others.

My goal was the end of February, but it ended up being the end of March, that I got something with a very simple functionality.

Folks could log in and change passwords
They could connect Patreon and Subscribe Star (a feature requested online)
They could set up a password that could be used basic authentication
The creator could export a .htpasswd file, hook it up to a static site, and be able to authenticate

Through that, I could get a “basic” age verification since Patreon and Subscribe Star both do age verification on their sides. That means I could mark my patreon as “mature” (I haven't done that yet), wire up my site in MfGames.Nitride to use it, and have what I need to quell that “fight” response in the back of my head.

And someone (the person who asked for Subscribe Star) is already using it. And reporting bugs and defects as they onboard their subscribers onto it. Which is exciting, because I rarely see someone actually using my tools (Nitride and MfGames Writing mostly).

Going Slow

Some of the painful parts is that progress was slow. I've been getting a lot of pressure from work to embrace large language models but I didn't want to do that with Leicmin. I want to lay down the code, to come up with the ideas, and work through them because I cherish those little insights when I find some pattern I never thought about before.

Protecting Minors

I'm not against the idea of limiting minors from accessing content. What I don't like is how law makers was going about it, since little of what they are producing is about protecting children, but very clearly attacking something they consider immoral.

(I also think it should be something parents should decide on, not law makers making decisions for everyone.)

Even listening to them talk about it, it was just another attack on people they don't like, they just use minors as their shield to defend their own biases.

That also means as I have the bandwidth, I'll find alternatives to handing over personal information to for-profit companies. Or find a way not to need a subscription service. I read about and bookmarked a couple promising alternatives, but they aren't quite there yet.

Going Forward

There are a ton of things that still need to be done before I consider it “done” in terms of user experience and usability.

Light theme was the first thing I was asked about
Documentation
Better authentication options
Emails
More age verification options
More geo blocking options
Auditing compliance (okay, that one is me)
Many more

On my personal side:

Set up an instance of my own
Go through my Fedran stories and mark the “adult” ones
Set up the instance to put a password on the adult stories

And, keep slowly moving forward. It was almost immediately after I was able to lean back and say “I got the MVP” that I was able to relax. Even not setting it up myself (which I will do), the knowledge that I had a clear path forward to handle that fight or flight was a major relief.

Just keep swimming.

—Dory, Finding Nemo

Allegro

Speaking of writing, I'm slowly submitting Allegro through the writing group. Of the alpha readers, about half of them came back with some good feedback and half of them never finished. It probably means that it will end up as well as Flight of the Scions, but I'm still going to try publishing the best book I can.

Other Smarter People

2026-01-04T06:00:00Z

I don't like reinventing the wheel. At the same time, I'll look for something and not find it, go through the effort to create something of my own, only to find out that my search skills weren't high enough and someone has already figured that out.

In a matter of twenty-four hours, I found that two such situations had happened. Which means, I can close the topic on identifiers and simply point to other folk's work and build the little quirk I want on to of those.

Series

2024-05-30 Generating and Using Identifiers - A little discussion on unique identifiers, including formatting and presentation. A small database lesson on page splitting.
2024-06-29 Generating and Using Identifiers (Part 2) - A few expanded ideas about identifiers after discussing it with others.
2026-01-04 Other Smarter People - I found another specification for representing identifiers that is better supported, has more attention, and covers most of the basics I need to.

Identifiers

Let's start with the last six months (it's been that last since the last of this series… actually, of most of my posts): I actually used my ideas to see if they were viable in practice as they were in my head.

I found that when I elided the identifiers, which means only showing the last group of the identifier along with the prefix, it didn't give the benefit I thought it would. For example, group__01_hz613s22_k8nr6w6g_rv1y5c52 being elided to group__rv1y5c52. When seen on screen, the group__ was still the bulk of the screen estate and the content was sufficient to make it unimportant.

Then while I was working on Leicmin, I stumbled onto a series of Rust libraries that implemented a standard that was almost completely the same as mine, including using Crockford32 encoding: TypeID (or my local tag).

TypeID uses the underscore separator, so double-click and selection works. It defaults to UUIDv7 but can also be used with other UUIDs. It only has one between the prefix and identifier block and no grouping. Which means the above example would be group_01hz613s22k8nr6w6grv1y5c52.

I lose the grouping and there no rules for eliding, but those are easy to handle. Just take the left X characters and not worry about anything else. So, in C#, eliding would simply be identifier.ToString()[^8..] and getting rv1y5c52.

Good enough.

Fortunately, there are a lot of implementations of TypeID. None of them work the way I want them too, or they are quite there, but it's a much better start and works with a standard that someone else has already established.

Package URLs

Speaking of finding standard, when I was giving my thoughts on Bakfu, I was trying to come up with a useful approach for referencing packages across different ecosystems.

While learning about TypeID, I noticed that the crates.io packages has a PURL link on the sidebar. I had never seen that before, but it basically lead me down to learning about a “mostly universal” URL format for packages so… other people were trying to solve the same thing and they had some interest behind their own (as opposed to none behind mine).

Some examples (from their website):

pkg:deb/debian/curl@7.50.3-1?arch=i386&distro=jessie
pkg:docker/cassandra@sha256:244fd47e07d1004f0aed9c
pkg:gem/jruby-launcher@1.1.2?platform=java
pkg:golang/google.golang.org/genproto#googleapis/api/annotations
pkg:maven/org.apache.xmlgraphics/batik-anim@1.9.1?repository_url=repo.spring.io%2Frelease&packaging=sources
pkg:npm/%40angular/animation@12.3.1
pkg:nuget/EnterpriseLibrary.Common@6.0.1304
pkg:pypi/django@1.11.1
pkg:rpm/fedora/curl@7.50.3-1.fc25?arch=i386&distro=fedora-25
pkg:rpm/opensuse/curl@7.56.1-1.1?arch=i386&distro=opensuse-tumbleweed

Well, that covers pretty much the same scope I was trying to implement, so another thing I can move to an accepted standard and stop working on my own.

Thoughts

A lot of my personal work is in isolation. I've always struggled to join communities and I have a relatively small “footprint” when it comes to social networks. And, because of my fragmented nature, I'm rarely in touch of people who are in the “know” who can tell me that someone else is already working on, or has solved, a problem I'm working on.

That all said, I'm glad I still went down the rabbit hole of trying to solve it. More so when my solution is only trivially different from the more popular one. It tells me I'm in the right direction.

I also think as a thought exercise, it gives me more appreciation for the work that goes behind these efforts and frequently makes me a fan or even advocate because I'm emotional behind the reasons.

So, time not wasted but also relief that someone else figured it out instead.

Git For Authors (v0.0.10)

2025-08-08T23:41:54Z

🛈 This isn't done. Not even close. As you can see from the front page, a lot of the bullet items aren't leading to links, which I full intend to flesh out into more detailed sections. Plus I'm still in the process of dumping information and the organization isn't there. If this was the 90s, I would have a “Under Construction” GIF.

I enjoy writing as much as I like coding. While I used entirely separate tools to do either, over the years, those two processes have converge to my writing projects look like my coding projects. They use pipelines to generate the final output, source control, automated versioning, and the same suite of tools. In many cases, these tools are designed to automate tedious parts of publishing or coordinate work with others, something that can be difficult with existing tools.

The inspiration of this is Fast Trip, a story by James White.

Organization

I don't like just saying “this is how I do things” because this isn't, nor will it ever be, the One True Way™ of writing with Git. Instead, I've attempted to organize the topics from high-level ones, to global concepts, and then into advanced topics.

There is a lot of cross-linking of topics within the individual pages for relevance, but you can easily choose to ignore those links since if you follow this index page, you'll get to them in “discovery” order.

Meta Topics

Target Audience - Who this plot is aimed for and what skills are useful to take advantage of it
Conventions describes the notations used
Model-View-Controller is the high-level concepts of separating the content (model) from its appearance (view)

File Formats

While the goal of this plot is to use a program—Git in this case—to manage writing, the first part is talking about “what” is being managed. In this case, it is the words you want to put into a novel or story. There are other things that you might want tracked, such as character notes, research, various pin boards and themes. These are all going to be “tracked” as part of the project.

Text Files is a discussion of why writing should be done in plain text files
Top Matter are the details about a given chapter or story
Binary Files talks about why Git doesn't play well with binary files

Because of limitations in text files for doing extra formatting, I use markup to tell when to make something italic, bold, or write an epigraph.

Markup is how to decorate a text file with some of the formatting rules that writers need
Markdown is the most popular form of markup

Organization

Organization also becomes important with bigger projects. While you can throw everything into the project directory in a glorious mess, I have established some conventions that work well for me.

Project Layout is how I organize my files and directories for writing projects.

Source Control

The entire reason for this plot is to use a source control program to manage writing. These programs keep a historical record as you write, allow to review changes, but also avoid accidentally overwriting changes. Programs like Git also allow you to handle feedback from editors and reviewers even when they come in over a period of time and after you've already made changes.

Source Control discusses the systems in general and my history with them
Git is the source control system I mainly use while writing
Repositories are copies of the writing project
Commits are how changes are saved to a repository (and a branch)
Tags can be used to identify points in time where changes are made
Branches are used to coordinate with readers and editor, or to redo chapters

Text Editors

One advantage of using text files is that almost any editor or IDE can be used to write. They have different capabilities or plugins that help.

Visual Studio Code is a lightweight and flexible text editor

Transforming

A critical component of using text formats is how to transform them into other formats that editors and readers need.

Transforming talks about transforming source files into the desired output
MfGames Writing is a publishing tool for making EPUB and PDF files

Source Forges

Building on top of source control, Git in specific, are code forges which give a web view of your projects, limit others from seeing them, but also gathering feedback and edits.

Code Forges is about online systems for managing Git repositories, pipelines, and issues
Pipelines are automated tasks that run
Issues are a task tracking for a project
Pull Requests are for getting changes from others

Each forge is different from the others in almost every way. That includes setting them up, how to run the pipelines, or even how issues are tracked.

Forgejo
SourceHut
GitLab
GitHub

Versioning

One way of dealing with the “final23” files

Versioning describes while creating a version for a story is important and how to use semantic versioning
Conventional Commits describes a common formatting for Git commit messages to automate versioning

Other Tools

Beyond the format of the files, how a project is organized is important to keeping everything separate and also to make some of the tools more effective.

Querying talks about using tools to work with the source files
Markdowny is a tool used for querying the top matter

Setting Up the Environment

I've been writing novels and stories for many decades. While most of the time, I just write it and then produce the resulting PDF or EPUB, occasionally I want to go back only to find that the libraries have “bit rotted” or I based on them on a global package I forgot to migrate to my new machine or that I had modified to handle a newer book that broke the old one.

Inevitability, this means days or even weeks of bringing the project up to date just to get a PDF out of it.

Along the way, I got frustrated with that and have come up with various tools or frameworks for producing a consistent environment years or even decades later.

Using NixOS describes setting up NixOS to make it easier to write

About This

Finally, this is a living document. Writing it as a digital plot and versioning it will hopefully allow me to evolve as we go.

License

This book is distributed under a Creative Commons Attribution-ShareAlike 4.0 International license. More info can be found at https://creativecommons.org/licenses/by-sa/4.0/

The preferred attribution for this novel is:

“Git For Authors” by D. Moonfire is licensed under CC BY-SA 4.0

In the above attribution, use the following links:

Git For Authors: https://d.moonfire.us/garden/project-layout/
D. Moonfire: https://d.moonfire.us/
CC BY-SA 4.0: https://creativecommons.org/licenses/by-sa/4.0/

Source

The source of this project can be found on the Moonfire Games forge. Feel free to report any issues, requests for expansion or clarifications. Alternately, you can contact me directly.

Using Nitride - Front Matter

2025-06-12T05:00:00Z

Being able to associated metadata for a page is a very useful thing. It can be used to group pages into categories, control how it is styled, or to simply provide internal notes. To do that, we use something called a YAML front matter to describe the details.

---
title: Name of the Page
summary: Summary of page
image: /path/to/image.png
---

This is the front page.

This is a major functionality of the Markdown + YAML pages that I use in my technical, fantasy, and even blog posts. I use the categories and tags heavily on this page, not to mention giving a summary details for links. It can also be used to provide OpenGraph schemas.

Series

2025-06-07 Using Nitride - Introduction - I'm going to start a long, erratic journey to update my publisher website. Along the way, I'm going to document how to create a MfGames.Nitride website from the ground up, along with a lot of little asides about reasoning or purpose.
2025-06-08 Using Nitride - Pipelines - The first major concept of Nitride is the idea of a "pipeline" which are the largest units of working in the system. This shows a relatively complex setup of pipelines that will be used to generate the website.
2025-06-09 Using Nitride - Entities - The continued adventures of creating the Typewriter website using MfGames.Nitride, a C# static site generator. The topic of today is the Entity class and how Nitride uses an ECS (Entity-Component-System) to generate pages.
2025-06-10 Using Nitride - Markdown - Examples and explanations of converting Markdown to HTML using MfGames.Nitride and MarkDig.
2025-06-12 Using Nitride - Front Matter - How to use Nitride to take the front matter from pages and add them as a component into the Entity class.

Defining a Model

Because C# is a static language, we want to take advantage of a schema so we can have type-safe. Historically, I've called this PageModel because I wrap that in a TemplateModel when I get to the styling.

// In //src/dotnet/Models/PageModel.cs
namespace Generator.Models;

public class PageModel
{
    /// <summary>
    ///     Gets or sets the optional list of categories associated with the page.
    /// </summary>
    public List<string>? Categories { get; set; }

    /// <summary>
    ///     Gets or sets the URL of the image associated with the page.
    /// </summary>
    public string? Image { get; set; }

    /// <summary>
    ///     Gets or sets the summary for the page.
    /// </summary>
    public string? Summary { get; set; }

    /// <summary>
    ///     Gets or sets the title of the page.
    /// </summary>
    public string? Title { get; set; }
}

Naturally, this can and will get a lot more complicated. Because we are using YAML, we can have nested objects, tags, and references that are appropriate for our page. In the above example, we are defining an optional title, summary, an image, and a list of categories.

It may come to a surprise to you, but this will eventually become a component in the page Entity class that is passed through the pipelines.

Including the YAML module.

Like the others MfGames.Nitride modules, we have to add it as a NuGet reference and tell the system to use it.

cd src/dotnet
dotnet add package MfGames.Nitride.Yaml

Adding it to the system is just a matter of modifying Program.cs to include it.

// In //src/dotnet/Program.cs
var builder = new NitrideBuilder(args)
    .UseIO(rootDirectory)
    .UseMarkdown()
    .UseHtml()
    .UseYaml()
    .UseModule<WebsiteModule>();

Parsing Front Matter

Parsing the front matter uses an operation, MfGames.Nitride.Yaml.ParseYamlHeader to parse the text content, pull off the front matter, wrap it in the model call, and then replace ITextContent with the page without the YAML header.

In effect, this:

---
title: Name of the Page
summary: Summary of page
image: /path/to/image.png
---

This is the front page.

... becomes “This is the front page.” in the text content with a PageModel component.

Adding the operation is relatively simple but this operation uses a generic parameter to identify the model to parse the YAML as.

// In //src/dotnet/Pipelines/Inputs/PagesModel.cs
public PagesPipeline(
    ILogger<PagesPipeline> logger,
    ReadFiles readFiles,
    IdentifyMarkdownFromPath identifyMarkdownFromPath,
    MoveToIndexPath moveToIndexPath,
    ParseYamlHeader<PageModel> parseYamlHeader)
{
    _logger = logger;
    _identifyMarkdownFromPath = identifyMarkdownFromPath;
    _moveToIndexPath = moveToIndexPath;
    _parseYamlHeader = parseYamlHeader;

And, like the others, it's just a matter of adding the operation into the pipeline.

// In //src/dotnet/Pipelines/Inputs/PagesModel.cs
var list = _readFiles
    .Run(cancellationToken)
    .Run(_identifyMarkdownFromPath)
    .Run(_parseYamlHeader)
    .Run(_moveToIndexPath)
    .ToList();

Once we run it, we get the following output:

[01:44:05 INF] <PagesPipeline> Entity: Path /contact/index.md, Components ["Zio.UPath","Generator.Models.PageModel","MfGames.Nitride.Contents.ITextContent","MfGames.Nitride.Markdown.IsMarkdown"]
[01:44:05 INF] <PagesPipeline> Entity: Path /index.md, Components ["Zio.UPath","Generator.Models.PageModel","MfGames.Nitride.Contents.ITextContent","MfGames.Nitride.Markdown.IsMarkdown"]

When we look at the HTML output, you'll notice it has a simplified version.

$ cat build/typewriter/html/index.html 
<h1>Typewriter Press</h1>

Retrieving the component is code just requires the Get<PageModel>() method or to use the various LINQ-based calls from MfGames.Gallium.

var page = entity.Get<PageModel>();

var list = entities
    .SelectEntity<PageModel>(OnlyRunOnEntitiesWithPageModel)
    .ToList();

public Entity OnlyRunOnEntitiesWithPageModel(Entity entity, PageModel page)
{
    return entity.Set(somePageSpecificVariable);
}

Gallium Methods

In the above case, the SelectEntity call will skip calling the lambda for entities that don't have a PageModel, but will still return it into the list. This is the “systems” part of the ECS system. The following will strip out the entities that don't have the appropriate models.

.SelectEntity<PageModel>((entity, page) => entity, includeEntitiesWithoutComponents: false)

So far, Gallium is set up to allow up to four components. I haven't had a need to do more, but those are easy to add. This makes it useful when doing some model processing on files that have a future date.

.WhereEntity<PageModel, UPage, Instant>(FilterOutFuturePages)

The SelectEntity is rather powerful because it takes an Entity and returns one. It can be the same entity, or it can be one that has zero or more components added or changed inside it. I found this really useful when I'm parsing out categories or I'm trying to build up a list of some specialized functionality that the styling will need.

public Entity OnSelectEntity(
    Entity entity,
    PageModel page,
    UPath path,
    ITextContent textContent)
{
    return entity
        .Set(nextPreviousModel)
        .SetAll(parentPage, parentUrl)
        .Set(modifiedPageModel);
}

What's Next

We have the minimum number of components and systems needed to setup styling. Next time, I'll use Handlebars to style the page and put a little chrome around the output.

2025-06-07 Using Nitride - Introduction - I'm going to start a long, erratic journey to update my publisher website. Along the way, I'm going to document how to create a MfGames.Nitride website from the ground up, along with a lot of little asides about reasoning or purpose.
2025-06-08 Using Nitride - Pipelines - The first major concept of Nitride is the idea of a "pipeline" which are the largest units of working in the system. This shows a relatively complex setup of pipelines that will be used to generate the website.
2025-06-09 Using Nitride - Entities - The continued adventures of creating the Typewriter website using MfGames.Nitride, a C# static site generator. The topic of today is the Entity class and how Nitride uses an ECS (Entity-Component-System) to generate pages.
2025-06-10 Using Nitride - Markdown - Examples and explanations of converting Markdown to HTML using MfGames.Nitride and MarkDig.
2025-06-12 Using Nitride - Front Matter - How to use Nitride to take the front matter from pages and add them as a component into the Entity class.

Using Nitride - Markdown

2025-06-10T05:00:00Z

Yesterday, I created an over-engineered program to copy a single file from one directory to another. Now, time to make it less overkill by transforming that Markdown file into simple HTML.

Series

2025-06-07 Using Nitride - Introduction - I'm going to start a long, erratic journey to update my publisher website. Along the way, I'm going to document how to create a MfGames.Nitride website from the ground up, along with a lot of little asides about reasoning or purpose.
2025-06-08 Using Nitride - Pipelines - The first major concept of Nitride is the idea of a "pipeline" which are the largest units of working in the system. This shows a relatively complex setup of pipelines that will be used to generate the website.
2025-06-09 Using Nitride - Entities - The continued adventures of creating the Typewriter website using MfGames.Nitride, a C# static site generator. The topic of today is the Entity class and how Nitride uses an ECS (Entity-Component-System) to generate pages.
2025-06-10 Using Nitride - Markdown - Examples and explanations of converting Markdown to HTML using MfGames.Nitride and MarkDig.
2025-06-12 Using Nitride - Front Matter - How to use Nitride to take the front matter from pages and add them as a component into the Entity class.

MarkDig

I don't like reinventing the wheel. I mean, I seem to keep doing it but I don't enjoy it. That was one reason why I tried another static site generators before working on MfGames.Nitride. However, when it comes to redoing a Markdown parser, even I'm not that foolish when there is already MarkDig, an excellent library for turning Markdown into HTML and extendable enough that I could also turn Markdown into Gemini (a later post).

In these cases, we need to tell Nitride how to do anything with Markdown since I didn't make it part of the core library. To do that, we need to pull in the NuGet package. While we're at it, we're also going to add the HTML processing library.

$ cd src/dotnet
$ dotnet add package MfGames.Nitride.Markdown
$ dotnet add package MfGames.Nitride.Html

Once we have the packages installed, we need to add those modules into the system. This is where Autofac came in helpful since I just have to add a module for the package it will handle the registration of any operations, components, and systems that we need to use.

// In //src/dotnet/Program.cs
var builder = new NitrideBuilder(args)
    .UseIO(rootDirectory)
    .UseMarkdown()
    .UseHtml()
    .UseModule<WebsiteModule>();

As you can see, I'm trying to follow the generic host pattern for the setup.

Identifying Markdown

While it may be obvious to convert any entity class that ends in .md into .html, we break this apart into separate steps. First, is that we identify a file as a Markdown file. This does two things, it adds the MfGames.Nitride.Markdown.IsMarkdown as a component, and then treats the contents as text instead of binary.

If you remember previously, we had this output:

[00:41:57 INF] <PagesPipeline> Read in 1 files from /src/pages
[00:41:57 INF] <PagesPipeline> Entity: Path /index.md, Components ["MfGames.Nitride.Contents.IBinaryContent","Zio.UPath"]
[00:41:57 INF] <SimplifiedMarkdownPipeline> Reading 1 entities
[00:41:57 INF] <StyleHtmlPipeline> Reading 1 entities
[00:41:57 INF] <OutputHtmlPipeline> Writing out 1 files
[00:41:57 INF] <OutputHtmlPipeline> Entity: Path /build/typewriter/html/index.md, Components ["MfGames.Nitride.Contents.IBinaryContent","Zio.UPath"]

Now, we're going to use a new operation, MfGames.Nitride.Markdown.IdentifyMarkdownFromPath. This could be put in a central place, such as SimplifiedMarkdownPipeline, but I found it is better to do this earlier than later so I usually put the identify process in the input methods. In this case, PagesPipeline:

// In //src/dotnet/Pipelines/Inputs/PagesPipeline.cs
public PagesPipeline(
    ILogger<PagesPipeline> logger,
    ReadFiles readFiles,
    IdentifyMarkdownFromPath identifyMarkdownFromPath)
{
    _logger = logger;
    _identifyMarkdownFromPath = identifyMarkdownFromPath;

    _readFiles = readFiles
        .WithPattern("/src/pages/typewriter/**/*.md")
        .WithRemovePathPrefix("/src/pages/typewriter");
}

public override IAsyncEnumerable<Entity> RunAsync(
    IEnumerable<Entity> entities,
    CancellationToken cancellationToken = default)
{
    var list = _readFiles
        .Run(cancellationToken)
        .Run(_identifyMarkdownFromPath)
        .ToList();

This operation doesn't take any parameters because it attempts to “do the right thing” with a minimal amount of effort. Running the code now produces this:

[00:46:43 INF] <PagesPipeline> Entity: Path /index.md, Components ["MfGames.Nitride.Markdown.IsMarkdown","Zio.UPath","MfGames.Nitride.Contents.ITextContent"]

The big things is that we have a new component, IsMarkdown, and the IBinaryContent changed to ITextComponent which gives us some additional extension methods but also indicates that the file is a text file instead of treating it as a simple binary. It still hasn't loaded the file into memory, it just switched how it is handled.

This is a separate step is because sometimes I want to keep a specific file as Markdown and not convert it to HTML. Also, there are times when I construct an Entity directly without having a valid path and I just have to add the IsHtml and ITextContent and it then acts like every other file without having to have any special rules.

Content

Entities with ITextComponent have a number of useful extension methods associated with them. (Technically, you can use these methods against any Entity class and it will try to convert a binary to text if there is a IBinaryContent and we want a ITextContent).

bool hasText = entity.HasTextContent();
string text = entity.GetTextContent();

entity.SetTextContent(stringValue);
entity.SetTextContent(stringBufferValue);

These are also stored as a ITextContent instead of string or StringBuffer. This is because the default interface is to leave the content on the disk and use Zio to retrieve it. However, as soon as SetTextContent is used, then it keeps that value in memory for the rest of the execution. This is the point where memory pressure begins to increase.

In the future, we could easily create a ITextContent implementation that writes large text files to the disk to get them out of memory. However, even my largest chapter of twenty-five thousand words doesn't create too much of a problem so I haven't bothered trying to implement that at this point (but if I did, it would go into //.cache in some manner).

Converting Markdown to HTML

Just identifying a file as Markdown doesn't do anything in itself. To convert it, we use another operation, MfGames.Nitride.Markdown.ConvertMarkdownToHtml. This easily goes into the StyleHtmlPipeline to handle the conversion and styling. It allows any MarkDig extension or plugin to be called as part of the setup, allow one to customize exactly how the output is generated.

We also want to change the extension from .md to .html. I realize I should have baked that logic into the ConvertMarkdownToHtml since it is a common operation, which I will do, but for now, we also need to use the MfGames.Nitride.IO.Paths.ChangePathExtension to do that.

public StyleHtmlPipeline(
    ILogger<StyleHtmlPipeline> logger,
    SimplifiedMarkdownPipeline simplifiedMarkdownPipeline,
    ConvertMarkdownToHtml convertMarkdownToHtml,
    ChangePathExtension changePathExtension)
{
    _logger = logger;
    _changePathExtension = changePathExtension.WithExtension(".html");

    _convertMarkdownToHtml = convertMarkdownToHtml
        .WithConfigureMarkdown(builder =>
        {
            SmartyPantOptions smartyPantOptions = new();
            SmartyPantsExtension smartyPants = new(smartyPantOptions);

            builder
                .Use<GenericAttributesExtension>()
                .Use(smartyPants);
        });

    AddDependency(simplifiedMarkdownPipeline);
}

public override IAsyncEnumerable<Entity> RunAsync(
    IEnumerable<Entity> entities,
    CancellationToken cancellationToken = default)
{
    var list = entities
        .Run(_convertMarkdownToHtml, cancellationToken)
        .Run(_changePathExtension, cancellationToken)
        .ToList();

    _logger.LogInformation("Reading {Count:N0} entities", list.Count);

    return list.ToAsyncEnumerable();
}

Running this gets us:

$ just build
[01:00:43 INF] <PagesPipeline> Read in 1 files from /src/pages
[01:00:43 INF] <PagesPipeline> Entity: Path /index.md, Components ["MfGames.Nitride.Markdown.IsMarkdown","Zio.UPath","MfGames.Nitride.Contents.ITextContent"]
[01:00:43 INF] <SimplifiedMarkdownPipeline> Reading 1 entities
[01:00:44 INF] <StyleHtmlPipeline> Reading 1 entities
[01:00:44 INF] <OutputHtmlPipeline> Writing out 1 files
[01:00:44 INF] <OutputHtmlPipeline> Entity: Path /build/typewriter/html/index.html, Components ["MfGames.Nitride.Html.IsHtml","Zio.UPath","MfGames.Nitride.Contents.ITextContent"]
$ find build -type f
build/typewriter/html/index.html
$ cat src/pages/typewriter/index.md 
# Typewriter Press
$ cat build/typewriter/html/index.html
cat build/typewriter/html/index.html
<h1>Typewriter Press</h1>

And now our over-engineered copy method has duplicated markdown2html for a single file.

Components

You may notice that IsMarkdown has been removed and IsHtml was added. This is part of where I struggled with Statiq and lead me down the path of using components. I don't have to pre-define the different data types, purposes, or even formats of a file. Enums are great, but they don't allow easy extension but with an ECS, it's just a matter of adding and removing components based on the use.

I've used components for a lot of things including identifying pages that should be in the blog archives, special notices, or pages that I want to ignore because they are aliases. I also embed indexes and lists into the pages to allow things like the “next” or “previous” links. If I was doing a web comic, I could have a per-character next/previous system easily implemented via those components.

My other static site generators didn't even have the content type tagging Statiq did, which was a novel concept for me and one that I'm glad I had a chance to use. It simplified a lot of my logic and lead nicely into where I am today.

Planning Ahead

As I'm planning ahead, I'm going to do the following change:

Rename SimplifiedMarkdownPipeline to ContentPipeline
Move the logic I just added into a new pipeline called BareHtmlPipeline and insert it into between ContentPipeline and StyleHtmlPipeline

The reason for this is because RSS/Atom feeds use bare HTML to generate their content, so it makes sense to have that bare pipeline feed both of them while having the ContentPipeline handle a lot of the linking and references that we'll need.

Directory Paths

One last thing for this post: I prefer paths that end in directory slashes instead of files. So, if create a contact page at //src/pages/content.md, we want the HTML to be at https://typewriter.press/content/. This is, creatively enough, another operation: MfGames.Nitride.IO.Paths.MoveToIndexPath.

// In //src/dotnet/Pipelines/Inputs/PagesPipeline.cs
public PagesPipeline(
    ILogger<PagesPipeline> logger,
    ReadFiles readFiles,
    IdentifyMarkdownFromPath identifyMarkdownFromPath,
    MoveToIndexPath moveToIndexPath)
{
    _logger = logger;
    _identifyMarkdownFromPath = identifyMarkdownFromPath;
    _moveToIndexPath = moveToIndexPath;

    _readFiles = readFiles
        .WithPattern("/src/pages/typewriter/**/*.md")
        .WithRemovePathPrefix("/src/pages/typewriter");
}

public override IAsyncEnumerable<Entity> RunAsync(
    IEnumerable<Entity> entities,
    CancellationToken cancellationToken = default)
{
    var list = _readFiles
        .Run(cancellationToken)
        .Run(_identifyMarkdownFromPath)
        .Run(_moveToIndexPath)
        .ToList();

And we add a contact.md page which a run gives us this:

$ just build
[01:12:26 INF] <PagesPipeline> Entity: Path /contact/index.md, Components ["MfGames.Nitride.Contents.ITextContent","MfGames.Nitride.Markdown.IsMarkdown","Zio.UPath"]
[01:12:26 INF] <PagesPipeline> Entity: Path /index.md, Components ["MfGames.Nitride.Contents.ITextContent","MfGames.Nitride.Markdown.IsMarkdown","Zio.UPath"]
[01:12:26 INF] <ContentPipeline> Reading 2 entities
[01:12:26 INF] <BareHtmlPipeline> Reading 2 entities
[01:12:26 INF] <StyleHtmlPipeline> Reading 2 entities
[01:12:26 INF] <OutputHtmlPipeline> Writing out 2 files
[01:12:26 INF] <OutputHtmlPipeline> Entity: Path /build/typewriter/html/contact/index.html, Components ["MfGames.Nitride.Contents.ITextContent","MfGames.Nitride.Html.IsHtml","Zio.UPath"]
[01:12:26 INF] <OutputHtmlPipeline> Entity: Path /build/typewriter/html/index.html, Components ["MfGames.Nitride.Contents.ITextContent","MfGames.Nitride.Html.IsHtml","Zio.UPath"]
$ cat src/pages/typewriter/contact.md 
# Contact Us

Our emails is [contact@typewriter.press](mailto:contact@typewriter.press).
$ cat build/typewriter/html/contact/index.html 
<h1>Contact Us</h1>
<p>Our emails is <a href="mailto:contact@typewriter.press">contact@typewriter.press</a>.</p>

As you can see, the input is //src/pages/contact.md, but through the pipeline, it is written out as //build/typewriter/html/contact/index.html.

What's Next

Next up, handling front matter. We need that for many reasons, but one of the biggest reason is to provide metadata to generate styled HTML output.

2025-06-07 Using Nitride - Introduction - I'm going to start a long, erratic journey to update my publisher website. Along the way, I'm going to document how to create a MfGames.Nitride website from the ground up, along with a lot of little asides about reasoning or purpose.
2025-06-08 Using Nitride - Pipelines - The first major concept of Nitride is the idea of a "pipeline" which are the largest units of working in the system. This shows a relatively complex setup of pipelines that will be used to generate the website.
2025-06-09 Using Nitride - Entities - The continued adventures of creating the Typewriter website using MfGames.Nitride, a C# static site generator. The topic of today is the Entity class and how Nitride uses an ECS (Entity-Component-System) to generate pages.
2025-06-10 Using Nitride - Markdown - Examples and explanations of converting Markdown to HTML using MfGames.Nitride and MarkDig.
2025-06-12 Using Nitride - Front Matter - How to use Nitride to take the front matter from pages and add them as a component into the Entity class.

Using Nitride - Entities

2025-06-09T05:00:00Z

From yesterday's post, there was an dangling topic, the Entity class. Unlike most of the static site generators I've worked with, MfGames.Nitride uses an ECS (Entity Component System) to generate its files.

Series

2025-06-07 Using Nitride - Introduction - I'm going to start a long, erratic journey to update my publisher website. Along the way, I'm going to document how to create a MfGames.Nitride website from the ground up, along with a lot of little asides about reasoning or purpose.
2025-06-08 Using Nitride - Pipelines - The first major concept of Nitride is the idea of a "pipeline" which are the largest units of working in the system. This shows a relatively complex setup of pipelines that will be used to generate the website.
2025-06-09 Using Nitride - Entities - The continued adventures of creating the Typewriter website using MfGames.Nitride, a C# static site generator. The topic of today is the Entity class and how Nitride uses an ECS (Entity-Component-System) to generate pages.
2025-06-10 Using Nitride - Markdown - Examples and explanations of converting Markdown to HTML using MfGames.Nitride and MarkDig.
2025-06-12 Using Nitride - Front Matter - How to use Nitride to take the front matter from pages and add them as a component into the Entity class.

History

I've worked on a number of static site generators over the years, including more than a few adhoc ones until I finally got something stable with CobblestoneJS which carried me for almost a decade of heavy use. A lot of the patterns that Nitride was based on were on how I found I had to build my pages in Cobblestone, including having a separate “gather” step which wrote out temporary files and then the “process” step which formatted them and made everything pretty. I had to do that because I wanted to have cross-linking and references work across multiple Git projects.

A good example is the Fedran website. At the bottom of a chapter, I have a list of characters and locations in the chapter. The links are built from data from the wiki project but the chapter information comes from the story Git repository. The gather step let me combine both of those together to generate those links.

That also meant that I needed to have some supplementary information stored in the gather step (which was written to a //build folder) to help with those linking.

Near the end, once Cobblestone began to break under its abstraction and complexity, I wanted to get away from writing my own and switched to GatsbyJS which made a very pretty website, but it was painful to debug and develop. It only lasted a year or so before I wanted to move (though the Gatsby version of Fedran was probably the prettiest I had ever made). If anything, Gatsby ended up being an ecosystem that needed me to pay attention to update things and my cycle of coming around to update a project was too far for it so I spent days trying to get it working again to build it.

So I tried out Statiq. I never successfully made a website out of it, but it introduced me to some really interesting concepts including pulling in data from other sources (Gatsby also did this, just never really used it). But with the C#, it had some really interesting problems. However, I quickly found that I was trying to do something that was outside of the planned usage, which involved me trying to extend an enumeration which involved basically rewriting large hunks of the system just to handle it. Plus, I really like Autofac and I could never get it to play well with that.

Which lead me into creating Nitride.

Now, I want to say, both Gatsby and Statiq are both good systems. They just aren't good systems for me. These don't do what I want them to do and they are designed around patterns that I'm not comfortable with. Even with the desire not to write my own, it wasn't enough.

Entity Component Systems

Nitride is build on MfGames.Gallium which basically means the formal name of the system Gallium Nitride:

Gallium nitride (GaN) is a binary III/V direct bandgap semiconductor commonly used in blue light-emitting diodes since the 1990s. The compound is a very hard material that has a Wurtzite crystal structure. Its wide band gap of 3.4 eV affords it special properties for applications in optoelectronics, high-power and high-frequency devices. For example, GaN is the substrate that makes violet (405 nm) laser diodes possible, without requiring nonlinear optical frequency doubling.

Gallium is a small ECS library. It doesn't intend to be fast or efficient, but it was written to match the idiomatic patterns of System.Linq. I wanted something that wasn't going to reinvent the wheel, which also meant not making custom collection classes or lookup tables. Instead, I wanted to work with BCL primitives which is why much of the system is based on IEnumerable<Entity>.

There is some high-level documentation for Gallium on its project page and its test, but I'll give the basics here.

The core object is MfGames.Gallium.Entity. This is a small class that has an integer identifier and basically a Dictionary<Type, object> called Components. It has some methods to make it easier, but basically you can set or get components from it:

// This creates an empty entity.
Entity entity1 = new();

// This creates an entity with four components:
//   "System.Int32": 123
//   "System.String": "bob"
//   "System.IO.FileInfo": ...
//   "System.IO.DirectoryInfo": ...
Entity entity2 = new()
    .Set(123)
    .Set("bob")
    .SetAll(
        new FileInfo("/tmp/bob.txt"),
        new DirectoryInfo("/tmp")
    );

Assert.True(entity2.HasComponent<FileInfo>);
Assert.False(entity2.HasComponent<decimal>);

Assert.True(123, entity2.Get<int>());

In effect, the class of the component is the key. There are a number of other functions, the test class is probably the best documentation at this point.

The systems part of the system is the LINQ-inspired collection classes. Most of them work off IEnumerable<Entity> with the test class being the bulk of the documentation:

// Set up a list of entities.
List<Entity> list = ...;

// This gets all the entities that have a FileInfo.
var fileOnlyList = list.WhereEntityHas<FileInfo>().ToList();

// This gets all entities that have both a FileInfo and an int.
var fileAndIntList = list.WhereEntityHasAll<FileInfo, int>().ToList();

// Get a modified list based on components. This looks for all
// entities that have a FileInfo and then adds a marker component,
// IsHtml, into it.
IsHtml isHtml = IsHtml.Instance;

var modifiedList = list
    .SelectEntity<FileInfo>((entity, file) => entity.Set(isHtml))
    .ToList();

As a note, Entity is “mostly” immutable. The Set command will return a new Entity instance with the same ID but with a different set of components. The “mostly” bit comes that the objects that it points to can be mutated.

🛈 The lack of formal documentation for Gallium and Nitride is one of the reasons this is still alpha or beta level code.

Operations

The next concept needed for today's post is an operation which extends MfGames.Nitride.IOperation. These are just easy ways of creating some functionality that works on an IEnumerable<Entity> that can be combined together.

Naturally, there are some extension methods that make it easier to work with operations with an IEnumerable<Entity>:

IEnumerable<Entity> entityList = ...;

IEnumerable<Entity> otherList = someOperation1.Run(entityList);

IEnumerable<Entity> resultList = otherList
    .Run(someOperation2)
    .Run(someOperation3)
    .Run(someOperation4);

This makes it relatively easy to create easy processing through composition of one or more operations. Alternatively, we could easily just work directly off the enumerable.

Zio

There is no question, I'm fond of the Zio library. In this case, Nitride uses two big features with it: to create an abstract view of the project that is independent of the working directory and the UPath class which normalizes paths.

If you remember from the previous posts, we set a root directory in the Program.cs:

public static async Task<int> Main(string[] args)
{
    var rootDirectory = new DirectoryInfo(Environment.CurrentDirectory);

    var builder = new NitrideBuilder(args)
        .UseIO(rootDirectory)
        .UseModule<WebsiteModule>();

    return await builder.RunAsync();
}

This caused an Zio.IFileSystem to be injected in the system which treats that directory as the /. So, in our case where we had our index file in //src/pages/index.md, it has a UPath of /src/pages/index.md. It doesn't seem like a lot, but it means that all paths in the system are always relative to the root of the project and there is no need to pass a root directory throughout the system which means path operations are considerably simplified.

And the UPath ensures a consistent pattern for accessing so this works on Windows, Mac, and Linux without a problem. Not to mention, any issues with .. are handled automatically.

It also simplifies writing tests.

Copy File in Six Files

Put all that together, we can use two operations to create a massively over-engineered copy file operation using Nitride. Admittedly, the complexity will make sense later, but at the moment, we're going to read the file in the PagesPipeline, through SimplifiedMarkdownPipeline, through StyleHtmlPipeline, and finally write it out in OutputHtmlPipeline.

Making Some Noise

To start with, let's put some logging to show how everything flows from pipeline to pipeline. To do that, we're going to inject an ILogger<> into the constructor into all four pipelines and then put a logging message into the last three (the PagesPipeline is going to do something different).

// In //src/dotnet/Pipelines/Content/SimplifiedMarkdownPipeline.cs
public class SimplifiedMarkdownPipeline : PipelineBase
{
    private readonly ILogger _logger;

    public SimplifiedMarkdownPipeline(
        ILogger<SimplifiedMarkdownPipeline> logger,
        PagesPipeline pagesPipeline)
    {
        _logger = logger;

        AddDependency(pagesPipeline);
    }

    public override IAsyncEnumerable<Entity> RunAsync(
        IEnumerable<Entity> entities,
        CancellationToken cancellationToken = default)
    {
        var list = entities.ToList();

        _logger.LogInformation("Reading {Count:N0} entities", list.Count);

        return list.ToAsyncEnumerable();
    }
}

Internally, Nitride uses Serilog, so using {Count:N0} does the right thing even thought we are using a logging abstraction for the actual call. When we run this, we get the following output (I'm not going to show the just build call for these today):

[14:00:46 DBG] <DirectoryService> Setting root directory and filesystem to /home/dmoonfire/src/typewriter-press/typewriter-press-website
[14:00:46 DBG] <BuildCommand> Processing 0 pipeline options
[14:00:46 INF] <BuildCommand> Running pipelines
[14:00:46 INF] <SimplifiedMarkdownPipeline> Reading 0 entities
[14:00:46 INF] <StyleHtmlPipeline> Reading 0 entities
[14:00:46 INF] <OutputHtmlPipeline> Reading 0 entities
[14:00:46 INF] <PipelineManager> Completed in 00:00:00.1079413
[14:00:46 INF] <BuildCommand> Command took 00:00:00.1097142

🛈 I moved NuGet.config to the root level because there is where the Website.sln file was located and it felt right.

Reading Files

Now, to read a file, we are going to use the MfGames.Nitride.IO.ReadFiles operation which reads file from the Zio file system and passes them through. We've already included that NuGet package in a previous step, so nothing to do there.

// In //src/dotnet/Inputs/PagesPipeline.cs
using MfGames.Gallium;
using MfGames.Nitride.IO.Contents;
using MfGames.Nitride.Pipelines;
using Microsoft.Extensions.Logging;

namespace Generator.Pipelines.Inputs;

public class PagesPipeline : PipelineBase
{
    private readonly ILogger _logger;
    private readonly ReadFiles _readFiles;

    public PagesPipeline(
        ILogger<PagesPipeline> logger,
        ReadFiles readFiles)
    {
        _logger = logger;
        _readFiles = readFiles.WithPattern("/src/pages/**/*.md");
    }

    public override IAsyncEnumerable<Entity> RunAsync(
        IEnumerable<Entity> entities,
        CancellationToken cancellationToken = default)
    {
        var list = _readFiles
            .Run(cancellationToken)
            .ToList();

        _logger.LogInformation(
            "Read in {Count:N0} files from /src/pages",
            list.Count);

        foreach (var entity in list)
        {
            _logger.LogInformation("Entity: Path {Path}, Components {ComponentList}",
                entity.Get<UPath>(),
                entity.GetComponentTypes().Select(x => x.FullName));
        }

        return list.ToAsyncEnumerable();
    }
}

This requires a bit of explanation. We are using dependency injection to insert the ReadFiles operation into the constructor. This is not a singleton, so you can have multiple ReadFiles if you need to read from multiple locations (or use a globbing operator to do it in one).

_readFiles = readFiles.WithPattern("/src/pages/**/*.md");

In the constructor, I configure it. I could do this in the RunAsync function but I found it made it easier to parse the files when I configure in the constructor. Like most things in Nitride, this uses a fluent coding style where each object sets the properties and returns itself to make a nice chained list of calls.

There are two properties being set, Pattern and RemovePathPrefix. I use a source generator to automatically create the With... pattern for fluent coding. The first one tells which files to read, relative to the root directory. This happen to use a globbing that allows me to grab all Markdown files in the source directory.

var list = _readFiles
    .Run(cancellationToken)
    .ToList();

This run the _readFiles operation and gets a list of the files it read in. (I'm also going to put this in the OutputHtmlPipeline to show how everything writes out.)

foreach (var entity in list)
{
    _logger.LogInformation(
        "Entity: Path {Path}, Components {ComponentList}",
        entity.Get<UPath>(),
        entity.GetComponentTypes().Select(x => x.FullName));
}

This is just some debugging to show how things are working and how Gallium works. In this case, the read files sets two components: the UPath for the path of the file and a IBinaryContent which is an abstraction to the binary content. Since we haven't done anything to the file, this just points to the file system itself without loading it into memory. That way, we can easily deal with gigabyte-sized files without overloading the system.

return list.ToAsyncEnumerable();

With this, we return the list of files that we just read in. With this change, running the build gets us this:

[14:20:03 DBG] <DirectoryService> Setting root directory and filesystem to /home/dmoonfire/src/typewriter-press/typewriter-press-website
[14:20:03 DBG] <BuildCommand> Processing 0 pipeline options
[14:20:03 INF] <BuildCommand> Running pipelines
[14:20:04 INF] <PagesPipeline> Read in 1 files from /src/pages
[14:20:04 INF] <PagesPipeline> Entity: Path /src/pages/typewriter/index.md, Components ["MfGames.Nitride.Contents.IBinaryContent","Zio.UPath"]
[14:20:04 INF] <SimplifiedMarkdownPipeline> Reading 1 entities
[14:20:04 INF] <StyleHtmlPipeline> Reading 1 entities
[14:20:04 INF] <OutputHtmlPipeline> Writing out 1 files
[14:20:04 INF] <OutputHtmlPipeline> Entity: Path /src/pages/typewriter/index.md, Components ["MfGames.Nitride.Contents.IBinaryContent","Zio.UPath"]
[14:20:04 INF] <PipelineManager> Completed in 00:00:00.9543639
[14:20:04 INF] <BuildCommand> Command took 00:00:00.9559424

As you can see, that single file read in is then passed through the other pipelines because each of them return their input list and then the final pipeline gets those same entities and paths.

Changing Paths

Now the problem with the above example is that we are writing our output to the same location as we are reading. That isn't optional. What we want to do is strip off the /src/pages/typewriter from the beginning of the read in path and add /build/typewriter/html in front of it. This is easily done with two additional operators, RemovePathPrefix and AddPathPrefix.

To start with, let's remove the prefix from the read methods to normalize the paths.

// In //src/dotnet/Inputs/PagesPipeline.cs
public PagesPipeline(
    ILogger<PagesPipeline> logger,
    ReadFiles readFiles,
    RemovePathPrefix removePathPrefix)
{
    _logger = logger;
    _removePathPrefix = removePathPrefix.WithPathPrefix("/src/pages/typewriter");
    _readFiles = readFiles.WithPattern("/src/pages/**/*.md");
}

And then we chain the operation using an extension method on operations:

// In //src/dotnet/Inputs/PagesPipeline.cs
public override IAsyncEnumerable<Entity> RunAsync(
    IEnumerable<Entity> entities,
    CancellationToken cancellationToken = default)
{
    var list = _readFiles
        .Run(cancellationToken)
        .Run(_removePathPrefix)
        .ToList();

Since each operation also takes an IEnumerable<Entity> and removes the same type, they can flow from one operation to another. When we run the build, the output line looks like this:

[14:23:35 INF] <PagesPipeline> Read in 1 files from /src/pages
[14:23:35 INF] <PagesPipeline> Entity: Path /index.md, Components ["MfGames.Nitride.Contents.IBinaryContent","Zio.UPath"]

Adding the output prefix is done on the output step:

// In //src/dotnet/Html/OutputHtmlPipeline.cs
public OutputHtmlPipeline(
    ILogger<OutputHtmlPipeline> logger,
    AddPathPrefix addPathPrefix,
    StyleHtmlPipeline styleHtmlPipeline)
{
    _logger = logger;
    _addPathPrefix = addPathPrefix.WithPathPrefix("/build/typewriter/html");

    AddDependency(styleHtmlPipeline);
}

public override IAsyncEnumerable<Entity> RunAsync(
    IEnumerable<Entity> entities,
    CancellationToken cancellationToken = default)
{
    var list = entities
        .Run(_addPathPrefix)
        .ToList();

Because we already have a list of entities, we can just use the extension method to pipe it through the operation and produce the following output:

[14:28:35 INF] <PagesPipeline> Read in 1 files from /src/pages
[14:28:35 INF] <PagesPipeline> Entity: Path /index.md, Components ["MfGames.Nitride.Contents.IBinaryContent","Zio.UPath"]
[14:28:35 INF] <SimplifiedMarkdownPipeline> Reading 1 entities
[14:28:35 INF] <StyleHtmlPipeline> Reading 1 entities
[14:28:35 INF] <OutputHtmlPipeline> Writing out 1 files
[14:28:35 INF] <OutputHtmlPipeline> Entity: Path /build/typewriter/html/index.md, Components ["MfGames.Nitride.Contents.IBinaryContent","Zio.UPath"]

The reason PagesPipeline doesn't use entities.Run(...) is because that is the first pipeline and the entities is going to be an empty list. Instead, we need to make entities, which is why we call _readFiles.Run() instead.

Writing Files

And the last bit for today's post is to write out files. This, creatively enough, uses the WriteFiles operation.

// In //src/dotnet/Html/OutputHtmlPipeline.cs
public OutputHtmlPipeline(
    ILogger<OutputHtmlPipeline> logger,
    AddPathPrefix addPathPrefix,
    WriteFiles writeFiles,
    StyleHtmlPipeline styleHtmlPipeline)
{
    _logger = logger;
    _writeFiles = writeFiles;
    _addPathPrefix = addPathPrefix.WithPathPrefix("/build/typewriter/html");

    AddDependency(styleHtmlPipeline);
}

public override IAsyncEnumerable<Entity> RunAsync(
    IEnumerable<Entity> entities,
    CancellationToken cancellationToken = default)
{
    var list = entities
        .Run(_addPathPrefix)
        .Run(_writeFiles)
        .ToList();

In this case, we've already done all the fancy operations to adjust the path so it writes out the files in the proper location:

$ find build -type f
build/typewriter/html/index.md

Convenience Methods

Now, adding and removing path prefixes is pretty common, so both the ReadFiles and WriteFiles have methods to do those in a single call.

// In //src/dotnet/Inputs/PagesPipeline.cs
public PagesPipeline(
    ILogger<PagesPipeline> logger,
    ReadFiles readFiles)
{
    _logger = logger;
    _readFiles = readFiles
        .WithPattern("/src/pages/typewriter/**/*.md")
        .WithRemovePathPrefix("/src/pages/typewriter");
}

// In //src/dotnet/Html/OutputHtmlPipeline.cs
public OutputHtmlPipeline(
    ILogger<OutputHtmlPipeline> logger,
    WriteFiles writeFiles,
    StyleHtmlPipeline styleHtmlPipeline)
{
    _logger = logger;
    _writeFiles = writeFiles.WithAddPathPrefix("/build/typewriter/html");

    AddDependency(styleHtmlPipeline);
}

What's Next

And there you go, an over-engineered system for copying a single file from one directory to another. In the next post of this series, we'll turn that Markdown file into an HTML page.

2025-06-07 Using Nitride - Introduction - I'm going to start a long, erratic journey to update my publisher website. Along the way, I'm going to document how to create a MfGames.Nitride website from the ground up, along with a lot of little asides about reasoning or purpose.
2025-06-08 Using Nitride - Pipelines - The first major concept of Nitride is the idea of a "pipeline" which are the largest units of working in the system. This shows a relatively complex setup of pipelines that will be used to generate the website.
2025-06-09 Using Nitride - Entities - The continued adventures of creating the Typewriter website using MfGames.Nitride, a C# static site generator. The topic of today is the Entity class and how Nitride uses an ECS (Entity-Component-System) to generate pages.
2025-06-10 Using Nitride - Markdown - Examples and explanations of converting Markdown to HTML using MfGames.Nitride and MarkDig.
2025-06-12 Using Nitride - Front Matter - How to use Nitride to take the front matter from pages and add them as a component into the Entity class.

Using Nitride - Pipelines

2025-06-08T05:00:00Z

I found a good way of working on a project is being able to see the end product. That means forging a minimal path from the start to end, and then going back to flesh it out. Interestingly, this is the opposite of how I write novels because, in those, the journey is the important part. Not so much with development projects.

In this case, the goal is to get a website generated and the ability to see it locally.

Series

2025-06-07 Using Nitride - Introduction - I'm going to start a long, erratic journey to update my publisher website. Along the way, I'm going to document how to create a MfGames.Nitride website from the ground up, along with a lot of little asides about reasoning or purpose.
2025-06-08 Using Nitride - Pipelines - The first major concept of Nitride is the idea of a "pipeline" which are the largest units of working in the system. This shows a relatively complex setup of pipelines that will be used to generate the website.
2025-06-09 Using Nitride - Entities - The continued adventures of creating the Typewriter website using MfGames.Nitride, a C# static site generator. The topic of today is the Entity class and how Nitride uses an ECS (Entity-Component-System) to generate pages.
2025-06-10 Using Nitride - Markdown - Examples and explanations of converting Markdown to HTML using MfGames.Nitride and MarkDig.
2025-06-12 Using Nitride - Front Matter - How to use Nitride to take the front matter from pages and add them as a component into the Entity class.

Project Setup

Since this mainly a .NET project, we start by creating a new project and solution. This is a fairly simple generator and I don't plan on needing more than one assembly, so I'm going to throw everything into //src/dotnet/ for the website.

$ dotnet new console --name Generator --output src/dotnet
$ dotnet new solution --name Website
$ dotnet sln add src/dotnet/Generator.csproj

And since we want to have the project run, we also update the Justfile:

# //Justfile

# Builds the typewriter.press website
build-typewriter:
    dotnet run --project src/dotnet/Generator.csproj --

And that gives us an easy command to run that works anywhere underneath the Git repository (thanks to Just).

$ just build-typewriter 
dotnet run --project src/dotnet/Generator.csproj --
Hello, World!

Setting up NuGet

At the moment, the MfGames.* assemblies are not on nuget.org. There are a couple of reasons, mostly self-doubt that anyone would find this useful and the dread of setting up a SSL certificate which appeared to be required. My current intent is to do it once I have a number of sites using Nitride or I get a request to do so. Until then, I use NuGet.config file to hook up my libraries to the website.

<!-- //src/dotnet/NuGet.config >
<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <packageSources>
    <clear />
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
    <add key="mfgames.com" value="https://src.mfgames.com/api/packages/mfgames-cil/nuget/index.json" protocolVersion="3" />
  </packageSources>
  <packageSourceMapping>
    <packageSource key="nuget.org">
      <package pattern="*" />
    </packageSource>
    <packageSource key="mfgames.com">
      <package pattern="MfGames.*" />
    </packageSource>
  </packageSourceMapping>
</configuration>

Setting up the Generator

While MfGames.Nitride could use configuration files to do the bulk of the setup, I went with a configuration-as-code approach to it since I almost always end up writing custom code. That way, everything is done in the same manner. This also uses a model roughly based on the generic host for command line.

🛈 I'm in the process of reworking the initial setup to use the generic host more completely. The current implementation was my best approach to write a dependency-injected console application that is configured via the CLI instead of JSON files.

The first step is to include the required package. Nitride is organized in a lot of small, focused packages. This is it avoid pulling in too much for a given project but also to help keep everything organized. For the initial work, we can add the first two required assemblies.

$ cd src/dotnet
$ dotnet add package MfGames.Nitride
$ dotnet add package MfGames.Nitride.IO
$ dotnet add package Autofac

Nitride using the excellent Zio library for abstracting file system access, plus making it easier to write test to make sure everything work. We also use Autofac for a lot of the module registration and processing.

🛈 I'm working on removing the Autofac dependency and using Microsoft.Extensions.DependencyInjection instead but it will take a while. There are a lot of useful features in Autofac that the extensions don't have so it will take some time.

First, we create the website module with a very generic, naive implementation (unless otherwise mentioned, all of these files are in //src/dotnet/):

// WebsiteModule.cs
using Autofac;

namespace Generator;

public class WebsiteModule : Module
{
    /// <inheritdoc />
    protected override void Load(ContainerBuilder builder)
    {
        builder
            .RegisterAssemblyTypes(GetType().Assembly)
            .AsSelf()
            .AsImplementedInterfaces()
            .SingleInstance();
    }
}

With that, we change the initial program to use Nitride:

// Program.cs
using MfGames.Nitride;
using MfGames.Nitride.IO.Setup;

namespace Generator;

public static class Program
{
    public static async Task<int> Main(string[] args)
    {
        var rootDirectory = new DirectoryInfo(Environment.CurrentDirectory);

        var builder = new NitrideBuilder(args)
            .UseIO(rootDirectory)
            .UseModule<WebsiteModule>();

        return await builder.RunAsync();
    }
}

If we run this, we'll get an error:

$ just build-typewriter 
dotnet run --project src/dotnet/Generator.csproj --
Required command was not provided.

Description:

Usage:
  MfGames.Nitride [command] [options]

Options:
  --log-level <log-level>                    Controls the verbosity of the output, not case-sensitive and prefixes allowed: 
                                             Verbose, Debug, Information, Warning, Error, Fatal [default: Warning]
  --log-context-format <log-context-format>  Controls the format of the source context for log items, not case-sensitive and 
                                             prefixes allowed: None, Class, Full [default: Class]
  --log-time-format <log-time-format>        Controls the format of the time in the log messages, such as 'HH:mm:ss' (default) or 
                                             'yyyy-MM-ddTHH:mm:ss'. Blank means exclude entirely. [default: HH:mm:ss]
  --log-level-override <log-level-override>  Overrides log levels for certain contexts in the format of either 'Context' or 
                                             'Context=Level' (repeat for multiple) [default: Microsoft=Warning]
  --version                                  Show version information
  -?, -h, --help                             Show help and usage information

Commands:
  build  Generate the website
  watch  Generates the site and then watches for changes

error: Recipe `build-typewriter` failed on line 15 with exit code 1

The watch command doesn't work very well right now, but we want build, so we'll update our Justfile:

# //Justfile
build-typewriter:
    dotnet run --project src/dotnet/Generator.csproj -- build

And then try again:

$ just build-typewriter
dotnet run --project src/dotnet/Generator.csproj -- build
[12:27:17 ERR] <PipelineManager> There are no registered pipelines run, use ConfigureContainer to include IPipeline instances
error: Recipe `build-typewriter` failed on line 15 with exit code 1

Pipelines

Now we get to the first big concept of Nitride: pipelines. Some of the ideas are built up from Statiq and also the patterns I've worked out with CobblestoneJS. They basically are larger “units” of processing for the site that are intended to gather a bunch of files and then feed them into another pipeline which then splits out into different ones for output.

It also allows a group of inputs to be handled differently, such as linking the “next” and “previous” to posts which doesn't make for pages. Another case where I've used this heavily is to copy binary assets directly to the styled output, generate CSS through webpack or esbuild, or generate project-specific pages from the input.

        +-------+ +-------+
        | Pages | | Posts |
        +-------+ +-------+
            |         |
            +----+----+
                 | 
                 v
          +-------------+
          |  Simplified |
          |   Markdown  |
          +-------------+
                 |
     +-----------+-----------+
     |           |           |
     v           v           v
+---------+ +---------+ +---------+
|  Style  | |  Atom   | | Style   |
|  HTML   | |  Feeds  | | Gemtext |
+---------+ +---------+ +---------+
     |                       |
     v                       v
+---------+             +---------+
|  Output |             | Output  |
|  HTML   |             | Gemtext |
+---------+             +---------+

Pipelines attempt to run in parallel to reduce the amount of time on the developer's machine. They are also asynchronous at their core since a pipeline could be used to query another system, require long-running processes, or perform some other action that would require using that functionality.

Obviously, you could use a single pipeline. I use to start that way, but the pattern of gathering the information, combining them together to generate tags and categories, and then styling fits better as separate steps for me. Since this is my website, we're going to create a minimum from the pages, generate simplified Markdown, style it, and then write it out.

Since we use dependency injection throughout Nitride, we pass in the dependencies into constructor of a pipeline and then add it to that project. Internally, Nitride runs every pipeline in a separate task when it is building the site.

// ./Pipelines/Inputs/PagesPipeline.cs
using MfGames.Gallium;
using MfGames.Nitride.Pipelines;

namespace Generator.Pipelines.Inputs;

public class PagesPipeline : PipelineBase
{
    public override IAsyncEnumerable<Entity> RunAsync(
        IEnumerable<Entity> entities,
        CancellationToken cancellationToken = default)
    {
        throw new NotImplementedException();
    }
}

// ./Pipelines/Content/SimplifiedMarkdownPipeline.cs
using Generator.Pipelines.Inputs;
using MfGames.Gallium;
using MfGames.Nitride.Pipelines;

namespace Generator.Pipelines.Content;

public class SimplifiedMarkdownPipeline : PipelineBase
{
    public SimplifiedMarkdownPipeline(
        PagesPipeline pagesPipeline)
    {
        AddDependency(pagesPipeline);
    }

    public override IAsyncEnumerable<Entity> RunAsync(
        IEnumerable<Entity> entities,
        CancellationToken cancellationToken = default)
    {
        throw new NotImplementedException();
    }
}

// ./Pipelines/Html/StyleHtmlPipeline.cs
using Generator.Pipelines.Content;
using MfGames.Gallium;
using MfGames.Nitride.Pipelines;

namespace Generator.Pipelines.Html;

public class StyleHtmlPipeline : PipelineBase
{
    public StyleHtmlPipeline(
        SimplifiedMarkdownPipeline simplifiedMarkdownPipeline)
    {
        AddDependency(simplifiedMarkdownPipeline);
    }

    public override IAsyncEnumerable<Entity> RunAsync(
        IEnumerable<Entity> entities,
        CancellationToken cancellationToken = default)
    {
        throw new NotImplementedException();
    }
}

// ./Pipelines/Html/OutputHtmlPipeline.cs
using MfGames.Gallium;
using MfGames.Nitride.Pipelines;

namespace Generator.Pipelines.Html;

public class OutputHtmlPipeline : PipelineBase
{
    public OutputHtmlPipeline(
        StyleHtmlPipeline styleHtmlPipeline)
    {
        AddDependency(styleHtmlPipeline);
    }

    public override IAsyncEnumerable<Entity> RunAsync(
        IEnumerable<Entity> entities,
        CancellationToken cancellationToken = default)
    {
        throw new NotImplementedException();
    }
}

And since we want to avoid exceptions, we'll ignore the entire Entity bit and do a couple of changes that will be explained in the next post.

In PagesPipeline.cs, we want to return an empty list:

public override IAsyncEnumerable<Entity> RunAsync(
    IEnumerable<Entity> entities,
    CancellationToken cancellationToken = default)
{
    return Array.Empty<Entity>().ToAsyncEnumerable();
}

For all the others, we're just going to pass whatever we got in back out:

public override IAsyncEnumerable<Entity> RunAsync(
    IEnumerable<Entity> entities,
    CancellationToken cancellationToken = default)
{
    return entities.ToAsyncEnumerable();
}

I'll admit, I stumbled into the usage of IAsyncEnumerable. It lets me use await calls inside a function without needing to load an entire list into memory just to copy it out again into another list to make the pipelines to work. Given that one of my baseline websites was to process my entire fiction website without killing my machine, I want to reduce the number of copies in memory and reduce the pressure there.

Using IAsyncEnumerable is cumbersome though, so might I might create a convenience method in PipelineBase that allows for a Task<IEnumerable<Entity>> and change it an IAsyncEnumerable but that is a future idea to work out.

When we run this, we get this (after adding ``–log-level debuginto the//Justfile`):

$ just build-typewriter 
dotnet run --project src/dotnet/Generator.csproj -- build --log-level debug
[13:22:48 DBG] <BuildCommand> Processing 0 pipeline options
[13:22:48 INF] <BuildCommand> Running pipelines
[13:22:48 INF] <PipelineManager> Completed in 00:00:00.1051617
[13:22:48 INF] <BuildCommand> Command took 00:00:00.1075694

It doesn't really do anything in the 0.1 seconds that it takes to run, but it basically gets the pipeline chain working so things will be generating with the next in this series.

Observations

I'm also aware that this appears to be overkill for most websites. If I was doing an HTML-only website, I would have started with only a single pipeline but I already know that I'm going to generate Atom feeds out of that and they work better with simple HTML instead of fancy chrome. Being able to style the Atom's output separately gives me some flexibility to do different things there. Also there are going to be multiple inputs eventually so experience tells me that this is the “best” approach for generating output given the circumstances of this specific website.

You'll also notice that I'm not doing a site-specific pipelines. I've tried a couple of approached and my current one is to run the entire generation process once per website using some configuration settings, which I'll get after I finish the main website.

Future Plans

The pipelines also are the units that are triggered on the watch command (which doesn't really work). If a pipeline identifies that it needs to be run, it will run but then trigger a re-run of every pipeline that depends on that.

The advantage of that if I'm working on styling CSS, I can have a pipeline feeding into the StyleHtmlPipeline that generates that. When the hypothetical CssHtmlPipeline triggers a rebuild, it runs itself, StyleHtmlPipeline, and OutputHtmlPipeline only because of those dependencies. But changing a page would trigger everything but the CssHtmlPipeline which should keep watches fairly performant.

What's Next

In the next post, I'll explain what Entity is and start make this actually generate something useful.

2025-06-07 Using Nitride - Introduction - I'm going to start a long, erratic journey to update my publisher website. Along the way, I'm going to document how to create a MfGames.Nitride website from the ground up, along with a lot of little asides about reasoning or purpose.
2025-06-08 Using Nitride - Pipelines - The first major concept of Nitride is the idea of a "pipeline" which are the largest units of working in the system. This shows a relatively complex setup of pipelines that will be used to generate the website.
2025-06-09 Using Nitride - Entities - The continued adventures of creating the Typewriter website using MfGames.Nitride, a C# static site generator. The topic of today is the Entity class and how Nitride uses an ECS (Entity-Component-System) to generate pages.
2025-06-10 Using Nitride - Markdown - Examples and explanations of converting Markdown to HTML using MfGames.Nitride and MarkDig.
2025-06-12 Using Nitride - Front Matter - How to use Nitride to take the front matter from pages and add them as a component into the Entity class.

Using Nitride - Introduction

2025-06-07T05:00:00Z

One drawback of having a lot of irons in the fire is occasionally projects that should not have been forgotten are and they bitrow. A good example is the Typewriter Press and it's associated reflected websites:

Sassy - Romance novels
Dusty - Historical fiction
Broken - Fantasy including contemporary

Eventually, when I start publishing my sci-fi stories, they would go under “Electric”. I call them “reflected” sites because they are just filtered versions of the main Typewriter site with some slightly different branding. That way, if I wrote more romance, I could put them there and folks could go to the romance-only site or jump over to the main one to see that I have no single genre (much like I don't have a single genre for my writing).

However, it's been years since I've updated those sites and they are beginning to creak underneath their age. Not to mention, I haven't included the last two books I published on them and that is humiliating for me.

So, I decided to switch the sites over to MfGames.Nitride and document the process.

Series

This is going to be broken into multiple posts, mostly because I know I'm going to get distracted around the fifteenth of the month when I switch to writing, but also to keep these topic-focused.

2025-06-07 Using Nitride - Introduction - I'm going to start a long, erratic journey to update my publisher website. Along the way, I'm going to document how to create a MfGames.Nitride website from the ground up, along with a lot of little asides about reasoning or purpose.
2025-06-08 Using Nitride - Pipelines - The first major concept of Nitride is the idea of a "pipeline" which are the largest units of working in the system. This shows a relatively complex setup of pipelines that will be used to generate the website.
2025-06-09 Using Nitride - Entities - The continued adventures of creating the Typewriter website using MfGames.Nitride, a C# static site generator. The topic of today is the Entity class and how Nitride uses an ECS (Entity-Component-System) to generate pages.
2025-06-10 Using Nitride - Markdown - Examples and explanations of converting Markdown to HTML using MfGames.Nitride and MarkDig.
2025-06-12 Using Nitride - Front Matter - How to use Nitride to take the front matter from pages and add them as a component into the Entity class.

Conventions

My usual conventions when giving file names is to use // for the Git repository root. So, //flake.nix means the flake.nix in the root level of the project. Also, I'll add a trailing / when I'm talking specifically about a directory, such as //node_modules/.

NixOS Flakes

Since I'm going to be creating a new site instead of converting the existing, I can easily start from the beginning. That usually means starting with a NixOS flake which is my preferred way of writing.

# //flake.nix
{
  inputs = {
    nixpkgs.url = "nixpkgs/nixos-25.05";
    flake-utils.url = "github:numtide/flake-utils";
    mfgames-project-setup.url = "git+https://src.mfgames.com/nixos-contrib/mfgames-project-setup-flake.git";
  };

  outputs = inputs @ { self, nixpkgs, flake-utils, mfgames-project-setup }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = nixpkgs.legacyPackages.${system};
        project-config = mfgames-project-setup.lib.mkConfig {
          inherit system;
          pkgs = nixpkgs.legacyPackages.${system};
          dotnet.enable = true;
          prettier.proseWrap = "never";
        };
      in
      {
        devShell = pkgs.mkShell {
          packages = [
            # Development
            pkgs.dotnet-sdk
            pkgs.nodejs_20

            # Local Serving
            pkgs.miniserve
            pkgs.agate
          ]
          ++ project-config.packages;

          shellHook = project-config.shellHook;
        };
      });
}

There really isn't much to this other than my use of mfgames-project-setup-flake. This is a flake that sets up a lot of the common elements I use: a consistent .editorconfig, treefmt for formatting various parts of the system, lefthook for setting up conventional commits.

And naturally, I have to set up direnv because I want to set up my environment as soon as I change into the directory.

# //.envrc
use flake || use nix

dotenv_if_exists .env

PATH_add $PWD/node_modules/.bin

I'm adding the node_modules path because I know I'm going to be using webpack as part of this site.

Once those two files are in, then I can get my basic setup.

$ git add flake.nix .envrc
$ direnv allow
direnv: nix-direnv: Renewed cache
nixago: updating repository files
nixago: '.conform.yaml' link updated
nixago: '/.conform.yaml' added to .gitignore
nixago: '.editorconfig' copy created
nixago: 'lefthook.yml' link updated
Error: unknown shorthand flag: 'a' in -a
Error: unknown shorthand flag: 'a' in -a
nixago: '/lefthook.yml' added to .gitignore
nixago: '.prettierrc.json' link updated
nixago: '/.prettierrc.json' added to .gitignore
nixago: 'treefmt.toml' link updated
nixago: '/treefmt.toml' added to .gitignore
mfgames-project-setup: '/.direnv/' added to .gitignore
sync hooks: ✔️ (commit-msg, pre-commit)
direnv: export +AR +AS +AWS_ACCESS_KEY_ID +AWS_BUCKET +AWS_ENDPOINT +AWS_REGION +AWS_SECRET_ACCESS_KEY +CC +CONFIG_SHELL +CXX +DOTNET_CLI_TELEMETRY_OPTOUT +DOTNET_NOLOGO +DOTNET_SKIP_FIRST_TIME_EXPERIENCE +DOTNET_SKIP_WORKLOAD_INTEGRITY_CHECK +HOST_PATH +IN_NIX_SHELL +LD +MSBUILDALWAYSOVERWRITEREADONLYFILES +MSBUILDTERMINALLOGGER +NIX_BINTOOLS +NIX_BINTOOLS_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu +NIX_BUILD_CORES +NIX_CC +NIX_CC_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu +NIX_CFLAGS_COMPILE +NIX_ENFORCE_NO_NATIVE +NIX_HARDENING_ENABLE +NIX_LDFLAGS +NIX_STORE +NM +NODE_PATH +OBJCOPY +OBJDUMP +RANLIB +READELF +SIZE +SOURCE_DATE_EPOCH +STRINGS +STRIP +__structuredAttrs +buildInputs +buildPhase +builder +cmakeFlags +configureFlags +depsBuildBuild +depsBuildBuildPropagated +depsBuildTarget +depsBuildTargetPropagated +depsHostHost +depsHostHostPropagated +depsTargetTarget +depsTargetTargetPropagated +doCheck +doInstallCheck +dontAddDisableDepTrack +mesonFlags +name +nativeBuildInputs +out +outputs +patches +phases +preferLocalBuild +propagatedBuildInputs +propagatedNativeBuildInputs +shell +shellHook +stdenv +strictDeps +system ~PATH ~XDG_DATA_DIRS
$ l
total 64K
drwxr-xr-x  5 dmoonfire users 4.0K Jun  7 11:10 .
drwxrwxr-x 18 dmoonfire users 4.0K Dec 26 08:08 ..
lrwxrwxrwx  1 dmoonfire users   56 Jun  7 11:10 .conform.yaml -> /nix/store/pckycri07q2ah90swk6hf21ilsi59a4s-conform.yaml
drwxr-xr-x  4 dmoonfire users 4.0K Jun  7 11:10 .direnv
-rw-r--r--  1 dmoonfire users 5.2K Jun  7 11:10 .editorconfig
-rw-------  1 dmoonfire users  259 Jun  7 10:40 .env
-rw-r--r--  1 dmoonfire users  13K Jun  7 11:10 flake.lock
-rw-r--r--  1 dmoonfire users  983 Jun  7 11:03 flake.nix
drwxrwxr-x  8 dmoonfire users 4.0K Jun  7 11:10 .git
-rw-r--r--  1 dmoonfire users  139 Jun  7 11:10 .gitignore
lrwxrwxrwx  1 dmoonfire users   56 Jun  7 11:10 lefthook.yml -> /nix/store/ihdb9vnc6jp535m43ndas79s4p9viyjn-lefthook.yml
lrwxrwxrwx  1 dmoonfire users   59 Jun  7 11:10 .prettierrc.json -> /nix/store/mjdm21r27r4n43aq92qkvmdcvy0l2qrc-prettierrc.json
-rw-r--r--  1 dmoonfire users 1.2K Jun  7 10:40 README.md
lrwxrwxrwx  1 dmoonfire users   56 Jun  7 11:10 treefmt.toml -> /nix/store/4lk6rmb2khs5l3yn91rah9y4c6zmxybf-treefmt.toml

Probably one of my favorite things is that it also sets up the initial .gitignore file:

$ cat .gitignore
# nixago: ignore-linked-files
/treefmt.toml
/.prettierrc.json
/lefthook.yml
/.conform.yaml
# mfgames-project-setup: ignore-files
/.direnv/

Overall, using that just cuts out that first hour of starting up a new project. And it lets me evolve my changes to style and different tools by simply running nix flake update.

Directory Layout

Related to using a flake to setup a lot of the boilerplate, I have written my thoughts on setting up polyglot projects such as this in a consistent manner. I think it is a relatively short digital garden plot, but there are a couple applicable things.

Source code goes into //src/ and the output is going into //build (which has an entry in .gitignore). One of the artifacts of my WordPress days is that I have static pages, which I put into //src/pages/, and blog posts which go into //src/posts/. There is some magic that goes on with posts, but that's a different post.

$ find src
src
src/pages
src/pages/typewriter
src/pages/typewriter/index.md
src/posts
src/posts/2025-07-01-website-redesign.md

I put an extra layer of directories under //src/pages/ for a site “slug” to identify which site the page will go on. In this case, I'm going to have typewriter for the main site, sassy, dusty, broken, and electric. There will also be a common for pages that are shared across all pages such as the contact page.

For output, those same slugs are going to be used. I'm intending to have this output both HTML and Gemtext pages for HTTPS and Gemini respectively. I know not a lot of people use Gemini these days, but I still think it is useful and there is a certain minimalism to it that appeals to me.

find build
build
build/typewriter
build/typewriter/html
build/typewriter/gemtext

Just

Another part of my standard project layout is using Just to handle the build system. There are a number of reasons, but I'm going to be dealing with Node programs, such as webpack, and also .NET projects for the build. There is also CLI executables for hosting locally (miniserv and agate).

I made a conscious decision not to write wrappers for Node and the other CLIs into Nitride. There is the ability to execute, but I don't want to tightly tie the site generator to a specific tool. Not to mention, most of the time, I want to set them up in different tabs to watch them so I want a more “generic” task runner than the ones built into Node or .NET which have some biases I've struggled with.

Just is something that is self-contained, doesn't make a lot of assumptions, and doesn't drag an entire ecosystem in with it.

# //Justfile
set dotenv-load

_default:
    just --choose

# Cleans up the project without removing the local .env
clean:
    git clean -xfd --exclude .env

# Builds all the websites
build: build-typewriter

# Builds the typewriter.press website
build-typewriter:
    echo I did something

# Serves the Typewriter website
serve-typewriter-html:
    miniserve --index index.html build/typewriter/html

# Serves the Typewriter Gemini pod
serve-typewriter-gemtext:
    mkdir -p .cache/agate
    agate --content build/typewriter/gemtext --certs .cache/agate --hostname localhost

Naturally, .cache/ has to be added to the .gitignore file. I use .cache from the project layout plot.

What's Next

Now that we have the basic boilerplate for a new project, time to start adding .NET into the mix.

2025-06-07 Using Nitride - Introduction - I'm going to start a long, erratic journey to update my publisher website. Along the way, I'm going to document how to create a MfGames.Nitride website from the ground up, along with a lot of little asides about reasoning or purpose.
2025-06-08 Using Nitride - Pipelines - The first major concept of Nitride is the idea of a "pipeline" which are the largest units of working in the system. This shows a relatively complex setup of pipelines that will be used to generate the website.
2025-06-09 Using Nitride - Entities - The continued adventures of creating the Typewriter website using MfGames.Nitride, a C# static site generator. The topic of today is the Entity class and how Nitride uses an ECS (Entity-Component-System) to generate pages.
2025-06-10 Using Nitride - Markdown - Examples and explanations of converting Markdown to HTML using MfGames.Nitride and MarkDig.
2025-06-12 Using Nitride - Front Matter - How to use Nitride to take the front matter from pages and add them as a component into the Entity class.

Not Quite a New Leg

2025-05-12T05:00:00Z

It's been a while since I posted. That isn't too much of a surprise, blogging is sometimes difficult for me when I'm under stress. But, there are times like now when things have relaxed a little and I figured it would be good to make a post. If I'm lucky, I'll do even a few more on what is going on my writing and coding aspects.

Some years ago, I fell and smashed my knee into driveway. It was a nasty injury and took me almost a month before I could kneel again, but there was a lingering pain that continued to get worse. First slowly with a tingling in my toes and a constant pressure but steadily becoming a constant, unrelenting agony.

I managed to hold off for about eighteen months before I was struggling to stand for more than a few seconds and I couldn't really walk in the mornings beyond a hobble from room to room. When I brushed my teeth, I had to half-crouch and lean against the sink just to remain standing. Even a grocery store run knocked me out.

I went to physical therapy where they stretched and prodded to stretch the sciatica. When it ended, with the feeling that it wasn't going to get much better, I was still failing the “slump test” where I couldn't look at my toes if I was sitting down. They tried a lot of things, but they only lasted a few hours or a day at most before everything hurt again.

I walked as much as I could but I could feel the distance contracting. Near the end of last year, I wasn't able to sleep or move or really do anything. My entire right leg, from hip to toes was constantly tingling or feeling like it was ice. It also felt like I was a few seconds away from having a charlie horse in my right leg.

It was finally too much and I went to the doctor again, asking for something else. This time, she sent me to get a MRI which indicated I had slipped a disk in my lumbar region and it was digging into my spinal cord; well, that wasn't optimal but it explained the pain.

The doctor said there was a risk because of my weight and suggested I lost about twenty pounds. But, if I couldn't and would accept the risk, he could do it. Until then, he got me a Cortisone shot.

That was a “fun” experience. For three days, I got to enjoy walking around the house without pain.

Before a week was over, it was back to the way it was. Which is ironic since they kept saying “it could last for months or even years” when I honestly think there was no way it would have been any lasting pain. Instead, it was a stalling gesture or something to placate insurance.

So I tried to lose the weight.

I really did.

I tried when three steps on the elliptical caused my leg to shake and sharp pains to radiate up my leg. I tried when even walking the length of the basement was agony. The walls in the house had scuff marks from my shoulder because I couldn't walk straight.

For three months I tried and I failed.

I called the doctor back. I was willing to take the risk, but I couldn't lose any more weight.

The nurse called back. The doctor refused to do anything unless I lost the weight.

I can't really describe how much that gutted me. I was too fat to help and I was in a situation where I couldn't swim anymore, I couldn't walk, I couldn't do anything unless I was going to cripple myself. I didn't have the strength or willpower to grind through the pain and that… person just dismissed me as no longer worth the effort.

(As a side note, I got the same behavior from that office when I broke my leg in 2014. Different doctor, same “compassionate” office. Needless to say, I'm not planning on going back.)

Thankfully, things worked out. As I bitched about the doctor (to whoever looked like they wanted to know), random people in my life started talking about the same doctor: Dr Chad Abernathy. I heard about husbands, family members, or even themselves that he had helped. It wasn't 100% and sometimes they had to go back, but he did something.

I needed something.

I called. I got an appointment.

It was only a twenty minute conversation where he ensured that I wasn't too fat, he was very good at the surgery, and he could take me the next week. I even heard a medical version of “bless his heart” in the conversation.

A week later, I was walking without pain. No limping in the morning, no hobbling. I'm still limited in what I can do: no bending more than thirty degrees or lifting more than thirty pounds, but I'm not in pain anymore. My leg isn't tingling and I'm not waking up thinking I'm drowning in ice. I can get out of bed and I can stand.

I can go to the grocery store and walk around the block. It has been years since I could do that without pain. Hopefully, I'll be able to take the dog on a walk and be able to lift things again.

Throughout this, Child.0 has done an amazing job of stepping up and helping me. Even when I drop something on the floor and announce “well, that was lost forever”. Last weekend, they dragged two couches onto the sun porch to decorate it without asking for help. I'm really lucky that I have a good kid to help me.

I'm back in physical therapy, but this time, things are getting better. I don't feel like I was hitting a plateau of recovery yet, only slow and steady progress where each day is getting better. I still have another month or so before I'm reevaluated.

I also have two months before I'm going for my followup for my diabetes. Part of this inability to move has done some noticeable damage because I couldn't work out. I'm scared about this next one because of that, but I have a small reprieve so I'm going to do what I tried: get my strength back and hopefully lose some weight.

Writing Mannerisms

2025-03-08T06:00:00Z

This post is just a fun little writing game to point out some of the quirks of my writing, some intentional, some of them I thought about, and others that just keep showing up no matter what I do.

This was inspired by Joel's post of the same name which was inspired by shellshark's post. Mostly I see it as a little fun, not unlike the hash games that show up on the fediverse. Also an opportunity to say why I do some things and some that haunt me.

Tokenization

Let's start with the most annoying: I frequently write the opposite word of what I mean. I also skip words when I'm writing. Because I also type it correctly, that means I don't notice it when I'm checking for spelling or even scanning the page. This has been one of my most frustrating little bugs I've never really handled.

I also miss the plurals of words or skip a couple key letters (but still spell some random word right).

I also start paragraphs frequently with “So,...” or "Because…".

I also say “does that make sense” way too often in general.

I do not like complex sentences. I loved learning German, but hated having to wait until the end of the sentence to know what was going on. And, while I have a relatively large vocabulary, I don't use when I write. I like simple sentences; according to a lot grammar programs, I write at a “third grade level.” I'm not sure what that means since I was reading David Eddings and Andre Norton in third grade.

I also love interrobangs but I use them for one specific purpose: when someone is screaming out a question. Honestly, if I had the nerve, I would typeset the books with “‽” instead of “!?” (and never "?!"). You never know, I might some day.

Likewise, I have rules for using ellipses (only for trailing sentences or thoughts, eliding, and leading examples) and em-dashes (parathenitcal asides and interrupted sentences and thoughts) along with specific formatting. Outside of those, I almost never use them.

Outside of ellipses, I don't use repeated punctuation. You will never see a “!!” or a “??” out of me.

Likewise, you will almost never see a casual use of italics or bolds in my writing and rarely in my conversations. Italics are for names of books and plays, ships, and whatever else the Chicago Manual of Style says they are for. One of my early lessons was “the words you write should give the intensity or purpose” right after they told me I should never use an interrobang again. Still going to use those.

Details

I'm terrible at details. I mean, I usually scatter in a few here and there, but I don't need to know someone's height or how big their bust is. Most of the time, it's a light brush of hair color, eye color, and specific build.

But, that depends on the character. Since I write single point-of-view stories almost exclusively, the personality and interests of the main character also dictates the details. The seamstress notices stitches and the fit of clothing. A warrior will have obvious weapons right up there before they notice hair color. The teenage boy is going to be staring at someone's ass or breasts. My OCD mage has a stunning amount of detail that I have to dial back because it can be overwhelming (I don't write Mudd much anymore because of that).

Now, the part of me that shows up in every character is how I show emotions. Remember the scene in The Accountant when they are in prison and Ben Affleck's character says “you are angry.”

That's kind of me.

I don't always know when I'm feeling an emotional unless I look at myself objectively. ("I appear to be speaking loudly and typing faster, I think I'm upset.") I'm not so great at reading other people's emotions at an unconscious level, so I'm usually looking for someone's body language, how their language structure changes, how they fidget or what goes on in their faces. That shows up in my writing because I don't really have a concept of “they are angry” in my head. I have to show it, because that is how I perceive emotions.

Sadly, this happens even for my characters who can understand emotions and don't have my struggles.

Structure

My paragraph structure is formulaic. Some of this comes from my favorite writing book, Techniques of a Selling Writer by Dwight V. Swain but others come from patterns. I'm fairly strict about one paragraph per character. I used to have one person saying something, then have the next character in the same paragraph but then I found it confusing. So, the actions and dialog are kept in the same paragraph to make it easier to know who is speaking.

I also use the back and forth with these paragraphs, the motivation-reaction unit (MRU) that Swain talks about. Those two together tie together into the beat of moving the story faster but also help me picture the scene since one thing leads to another in a logical sequence.

It also helps prevent something Mickey Zucker Reichert told me: there is no such thing as simultaneous action on the page.

Swain and Jim Butcher's also create another reoccurring pattern in my writing. I like to take a chapter to smell the roses. The so called “sequel” scenes where the characters have a chance to process what happened in the previous one, come up with a new plan that will eventually fail, and then go into it. That helps break up the constant build-up of scenes, which I usually reverse for the end of books, and instead keeps things moving steadily.

And the last is: I don't do scene breaks. I mean, I used to but then I realized I don't like the “hours later” starts to paragraphs, so I break apart the chapters. New location? Chapter. Hour later? Chapter. Character takes a twenty minute nap? Chapter. Sex scene? Depends on the book, but fade to black is always a chapter.

Exposition

I hate exposition. I'm not talking building a scene and setting it. I'm mostly talking about the “as everyone in the entire room already knows, the current president of France is….” If everyone knows, then I want the scene to be natural.

That is why I have epigraphs for when I really need to tell everyone a rod is sixteen and a half feet, or a chain is sixty-six feet. When I need to let readers know who the royalty is in the country or how a certain aspect of magic works.

This does mean I don't resolve every plot. If a character doesn't have the opportunity to know why something happened, they will never know. Someone else might, but not them. I try to resolve the “important” plots, but I'll leave some dangling ones here and there to be picked up later.

Style Guide

Now, given all those, you probably aren't surprise to know that I've formalized some of the intentional aspects of my writing and posted it as a style guide for my editors. That way, they won't call out things I've done on purpose or I do because of my tools.

Conversations

And the last section, how I talk informally. The biggest one is when I have a problem, I give a lot of details to explain it. Way too much in some cases, but I don't like wasting people's time, have a relatively high typing speed, and can read at a rather impressive rate. So, to avoid someone going through the basic diagnostic problems (“is your computer turned on”, “have you rebooted”), I will try to detail what I have tried so we can skip the easy steps.

This leads into first “wall of text” responses I give. The other is when I'm upset. When I am upset, I type faster. Usually, I'm trying to explain why I did something and my reasoning behind it because someone implied I was stupid or didn't understand the problem. This leads to the other wall of text.

Probably the most impressive one was when I was going through college. I was trying to get out of a networking class because I had been doing it for years. The person who reviewed it came back with a “nothing in this person's resume indicates they have any idea how networking goes.”

My response started with asking what level of the OCI networking did they feel I didn't have enough experience. And then started detailing my experience in all seven layers. I waxed poetically about running cat5 in the drop ceilings of my work or the hours I sat in front of a nine-inch monitor writing a real-time stock trading application. It took me an hour to write thirty pages of a response, which I sent back before I realized I should have sat on it or maybe edited it.

About two hours later, I got the “yeah… I'm not going to read this, good enough” response and I got out of the class.

The last thing is probably related to whatever value of the autism spectrum I'm on: I tag a lot of my sentences with an emotional state at the end. Usually in terms of “old school” emoji (:), :(). I know that text is a lossy form of communication, and it is highly likely the people will misunderstand you, so I try to give hints of whatever emotional state I think I'm trying to convey.

I don't like graphical emoji though, mainly because I can never figure out the keys and I don't like the packaged emojis on most systems. But, if I find ones I like, such the fruits on Microsoft Teams, I'll use them fairly heavily.

Now, emojis at the beginning of a sentence is a thread indicator. That means I'm having multiple conversations going on at once and need to group them. Kind of like playing D&D and having an in-character chat, and out-of-character chat, and a “anyone see the cheetos” chat.

Final Thoughts

So, those are some of my quirks of writing and the origin of a few of them. Hopefully, this will interest someone but it was also a chance to self-reflect on myself.

So Close, Yet So Far

2025-01-28T06:00:00Z

As much as I hoped to be done with the major repairs from the derecho, we hit a few snags along the way that got in the way. Overall, they aren't major ones and my anxiety has dropped significantly with what they had done.

The contractors finished with one room but it was too cold to finish the repairs on the porch (the derecho blew out windows, tore up wood, and soaked everything in rain). Repairing that is going to need a soffit and we hit some nasty cold weather, so they are going to wait until March to finish up.

The repairs they did were… okay? I mean, after the amazing work they did on the basement, the upstairs felt more like a 80% job in terms of polish and quality. Some of it is that I'm really sensitive to the floor and I can feel it flexing, but in the end, I decided it was good enough for now.

I mean, the entire reason we're getting contractors for this bit was because I didn't have the proper tools for everything and I'm struggling with kneeling and getting up. The bulk of the repairs involved tearing out carpet and putting down planks, so not being able to knee was kind of a big deal. In that regard, I'm as happy as I'm going to be so I'm just telling myself it is okay.

That also means the big stuff is finally done. At least the things I have to get anxious about. All that is left are little things: bringing back tables, hanging pictures, and the like. In March, they'll repair the porch but that should involve me getting some couches off the porch and letting them do their thing.

Of course, “little things” will probably take me another year to finish. It's easier to put “replace an electrical outlet” to the side when it involves shutting down my home lab, or buying new cold air return plates. But I'm okay with that.

That isn't to say that there aren't other big things. Besides the porch, I'm also hoping to get my mother's library that I inherited into the house. That is going to be a trial in itself. If I'm lucky, I'll build some bookcases along the way and get a nice, dense arrangement that doesn't try to take over too much of the house.

I also need to work on my health and see if I can get my dislocated disks fixed.

Despite last post being rather negative, I feel a lot better.

The End of 2024

2024-12-31T06:00:00Z

In a few hours from now, 2024 ends. I don't have a lot of emotional attachment to the new year, mainly because it is just an abstract point of time for a calendar we've created over the centuries. But, it isn't a bad time to look back and see what I have and haven't done.

Overall, 2024 is probably my worst year ever. There was a lot going on, but I'm not going to go into the details of those because it doesn't really help anyone. Needless to say, it was rough for me, mentally and physically, professionally and personally. And a whole bunch of other “-ly” I can't really think about.

Warning: This isn't a happy or positive post.

Mental Health

2024 was the first time I've ever had a panic attack.

That was scary and I had to look at why it was happening. It was just “more of the same” as it were but I'm still trying to figure out what got me there. I think a lot of it is this feeling that I can't solve things anymore (Long Covid maybe) and everything is ten times harder than it used to be.

More importantly, I haven't been writing. I want to write, I want to write so badly. Writing makes me feel good but I've had this writer's block for a few years now and its beginning to dig into my thoughts. I have ideas in my head, but not the focus that isn't interrupted by obligations and deadlines.

This is also the year I've been pushed into the rubicon of being a individual contributor at work and a manager. That means 4-6 hours of Zoom meetings almost every single day. It starts at 08:30 with a 1:1 with my manager, goes into an ninety minute meeting with the developers, an hour of stand up, and then the “when we're done, I need Dylan” that follows. In most cases, I have less than a minute between meetings or I'm five minutes late for the next, so I don't get any break for four hours every day.

Like commuting, that meeting train is grinding me down. I'm struggling to be excited about programming because I'm just stagger from that train to try getting a half hour nap before going into the next set.

Something is going to break in 2025, I only hope it isn't me. More likely, I'm going to transition completely off programming and will no longer have the joy of refactoring code or getting it solved. It isn't going to help that politics are going to be digging into me, watching friends being afraid and hurt, and struggling through… everything.

Physical Health

2024 was when I found out that I had a dislocated disk which was digging into my spinal cord, which is why I've been in pain since March of 2023. The surgeon said it could be fixed, but I need to lose weight to get there. A lot of the tail end was just trying to dig through my backlog of obligations and needs so I could get the home exercise equipment working again. I'm still not there. I'm still trying to do that, the Cortisone shot only gave me a few days of relief, but then a slow slide back to where I want.

The main part is, I can barely pick up anything and have had to leverage Child.0 into picking things up. The good thing, I know what is causing the pain so I can stop. The bad part is that there is no exercise, no stretch, nothing that can prevent it other than losing weight, getting surgery, and hoping it gets better.

And not thinking about the catastrophic failure of my “routine” hernia surgery. Or the severe problems I had from my vasectomy.

Derecho

The 2020 derecho is still in my life, but we're so close to finally closing that chapter of our life. In January, we should be getting the “last” of the repairs done on the house.

Related, I got a letter this week from my insurance company about replacing my roof and needing deposits to register it. How can I provide that went the guy who did it stole twelve thousand dollars from me and walked away? It isn't like I can say “yeah, you know how you owe me a tremendous amount of money? Could I get a receipt? By the way, fuck you.”

I'm hoping between that, getting the final room done, and organizing my workshop (to integrate my late father's shop) will finally let me set everything aside. Thought, at this point, it feels like it has been burrowing in my skull for so long that I don't know how to live without it.

I'm hoping to find out.

Writing

The urge to write is coming back. I think it's finally time to focus on getting the culture library finished, figure out Second-Hand Dresses, and maybe design some covers. Or even finish Nor Curse Be Found, which has been hanging around for a while.

If anything, I want 2025 to be a year of being creative again. I want to write, I want to work on libraries, I want to make things and have that feeling of being satisfied with what comes out.

Plus handle my monthly obligations that never go away.

Coding

I want to polish up some of my libraries, including MfGames.Nitride and maybe put it out into the open. I have other libraries that have been hanging out (MfGames.Culture being one of them) that I need to buckle down and finish up.

Play

I'm hoping to play around more in 2025. Not just games with the children, but drawing, writing, and maybe building a kitchen cabinet.

Goals

I can't really say I have goals for the year though. Instead, I just have hopes that 2025 is going to be better than 2024. Beyond that, I'm not really planning or expecting anything, just a vague hope that it works out in the end and I find my smiles again.

There is only one way to find out.

Just keep swimming.

Dory, Finding Nemo

Why SeaweedFS?

2024-11-08T06:00:00Z

I got an email last week asking if I could explain why I set up a Seaweed server. It's been a while since I talked about it, mostly in hints in this post from 2022 and this post from 2024. The request gives me an opportunity to expand on it, and to remind myself why I do it.

Stories

In some regards, there is a bit of trauma in me when it comes to losing out on things. This includes hearing stories that my father told me and realizing that when he died, they were gone forever. These same thoughts came up in my novel Sand and Bone:

The little girl's confessions echoed in Rutejìmo's head in a quiet symphony as he wrote in his book. The hand-bound collections of pages creaked under his hand, the leather thong strained to hold the almost fifty pages of tightly-spaced writing. Over the years, he had added a dozen pages to the collection. It wouldn't be too long before the binding couldn't handle the additional pages, but he thought he had a few more years left before that happened.

Even with his additional pages, he didn't have room to write down all of the stories he had heard over the years. He wanted to detail the joys of the little girl's death, such as the choked story about how she had stolen her brother's toy when he wasn't looking. He had also wanted to write the horrors, like one man's confession for killing his sister. Each one was precious and important. Time would erase their stories and a part of Rutejìmo died every time he forgot one.

— Sand and Bone 8: Alone

This fear is more than just one person, losing stories of things that happened decades ago as told by an old man. This includes my own stories, which will also be lost when I die. There is little chance I will be “done” with Fedran. There is little chance I will get all of the stories out of my head and onto paper. There isn't enough time, I don't have the self-esteem to do it, and I don't have the drive. When I'm gone, so are those stories. But, even going beyond that, there are so many stories of horror and joy that I feel we are losing with the passage of time. Happy stories of people falling in love, horror stories of Auschwitz, and everything in between. It is that breadth of stories, lessons, and happiness that we lose.

But the big question is, will anyone care? Does anyone want to know the first time I got caught shoplifting. Or the first time I finally said “I love you” to Partner?

Probably not.

But, you never know.

I'm never going to record them, but when I was given my father's artwork and decades of effort, I don't want to lose them yet. I don't want to just chuck it aside. So, I need to keep them.

Hoarding

I come from a long line of hoarders. I realized that as I look at my DVD collection or the boxes of books I have. My LEGO collection isn't huge, but it does fill five or six boxes. I once got rid of it, and I felt guilty for years for doing that; which is why I built it up again. Sometimes, I just play.

My mother did it. She had a massive display case of dragons and cats. She has dozens of sets of china in the basement. Entire room with the bones of a bankrupt lumber mill. A massive library (that I inherited).

My father did it. I spent days going through pieces of paper where he kept every calendar, shred of note, and personal letter he sent to his children. I saw research when I asked for help to get through college, mainly to prove that I was going the wrong path and I really should just drive two hours one way to go the college he thought was a better choice for me. I ended up saying “screw you” and went into debt for a couple decades instead.

My father has terabytes of images that he drew. Decades of him struggling to be a “good” artist, self-doubt and pain. But I'm so happy to watch him to do it. Also, he did the artwork for the nuclear reactor project he was on, and the particle accelerator. And birthday cards for all of his children and grandchildren.

For me, there is a good size of digital hoarding going on. I mean, a couple thousand ebooks is one things, but I have PDFs, for game systems that go back years. The original PDFs of HERO System 5 and 6. Every scan of the Dragon magazine. GURPS. Legal documents, funny stories, and the like. And then there are my DVDs. Back when I didn't have children, I was buying 3-5 a week. When Suncoast Video went out of business, I had a huge refund from the IRS. Walked into the store and said “I'm gunna buy everything I can.” Now, there are DVDs I can't buy anymore, I can't find. But they are slowly rotting in my file cabinet (did you know they only have a 20-30 year shelf life?). I don't want to lose them, even if I don't use them every day.

And then there is Partner's photography business. They do large photo shoots but they also need to keep them around for years “just in case” someone loses their wedding photos. She doesn't “have to”, but I don't want to ever have to say “sorry, I don't have them” for a senior photos or someone's puppies. Or family members are lost.

So I want to keep them.

Storage

All this takes space. The nice thing about digital space is that I can make backups. I can take them with me with a few kilograms of hard drives or upload them into the cloud. I can make copies to make it harder to lose (like when I was hit with a ransomware that took out my media server). The goal is 3-2-1 Backup Strategy: three copies, two locations, and at least one offline copy.

Now, dealing with storage is something I've done for quite a long time. I remember being pulled out of sixth grade school for a day to help my mother recover her RAID 5 box. At the time, drives were in the low 100s of megabytes, but I had to learn a lot about how RAID worked to figure it out. Then watching the slow recovery that took an entire day… then having another drive fail about a day after we recovered the first time and doing it all again. I used and watched hardware and software RAID controllers fail.

At the time, the data wasn't static. We were processing millions of records to do analysis. The drives would get corrupted by poor power, heavy usage, and the eventual failure of mechanical drives. We had customers lose data, watched our data centers blow a disk. Like Partner's photos, we had to keep our analysis for years after because of re-evaluation or the occasional lawsuit.

Over time, it became apparent that there was a maximum size where RAID was useful. After a while, it wasn't a matter “if” a drive goes back but “when” a drive goes bad. Backblaze (where I keep my backups, I recommend them) had a quarterly blog post about drive stats. They track with my own experiences. Yeah, 1.7% failure rate doesn't seem like much but when I'm dealing with stuff that needs to be kept for years, it hits a lot faster than you'd think it would. And offline backups fail too.

When I left my mother's company, I went back a little and just threw some drives into a big machine. Didn't bother with RAID because I knew it would fail since I was looking at a decade of hard drive use and started with a manual mirroring between a couple Windows partitions. But as the DVDs got ripped, music got purchased from Amazon, and photos kept being taken, I started to run out. Soon, it wasn't just Q:\, R:\, and S:\ being copies but each one having a portion of everything.

When I got hit with the ransomware attack, I lost my DVD collection (years of ripping) but not most of Partner's photos and the other things. I started to recover them but one of the drives didn't make it.

Then I lost another drive a few months later.

Then we didn't have money to handle the failure indicators, so I was just watching the SMART notices popping up knowing that there was nothing to do but watch the slow-moving train accident.

Then the media server decided to go down for a week.

Not having access to anything was a stark reminder that I had to do “something” if I didn't want to lose everything. And it wasn't just tossing in another drive every once in a while. I had hit the threshold where I needed to spread it out across more than just drives, I needed to spread it across servers.

Ceph

Enter Ceph. I've read about Ceph for many years before that point and it always appealed to me. It seemed to solve a lot of the problems I've experienced over the years. And it didn't have to require same-sized disks to pull of a RAID. And I could add machines if I needed to add more storage.

The biggest thing with Ceph is that it balanced across multiple servers to reduce failure, but also let you mark files as needing one, two, or more copies. Partner's photo library? Two copies. Videos? One is probably good enough. The servers didn't have to be that powerful either, so I could use my older machines to keep the disk when I had to upgrade to a newer one. In theory, I could use a Raspberry Pi to act as a cluster.

The incident when this happened was when the media server died again. I had saved some money from my commissions (earmarked to get a book edited) and used it to buy fresh hard drives. If anything, replacing the 10+ year old drives with something new and bigger would give me some. I also took the opportunity to switch to NixOS instead of Windows, which I dislike anyways. I mean, NixOS had options for Ceph, how hard could it be?

Apparently, it really wasn't ready for prime time. Eventually the wiki popped up to say that.

Took me a week to figure it out. There was a lot missing (and still is) from the core, but I managed to write up some notes for myself on how to make it work. But, after days of trial and errors, of “almost got it” joys only to watch it crash, I finally got something working. And it was glorious. At least until I realized was swapping like mad and Ceph really needs 1 GB per TB of storage. A few frantic purchases later, and I had a couple more cheap Dell machines (one of the ones that actually died this week) and I ended up with a fairly balanced Ceph cluster.

SeaweedFS

I liked Ceph, but I'm not on it anymore. Partially it was selfish reasons, I want to fix the struggles I had bringing on a new drive into the cluster but the community decided to go a different way. But there was also no one who was able to to do that different way while I was down. To test a fix myself, I had to build for twelve hours just to see if something worked (I also work with 10+ year old computers a lot). I was on unstable, so I went weeks being unable to build because I couldn't downgrade to stable. I also learned that nixpkgs had some patterns that were really hard to get into, like redefining lua at the package level to mean a specific version of lua and no one on the Matrix or Discourse forum was able to tell me that until I happened to find one person who explained it and said I should have just “known” about that mapping.

And then, the same day I realized my efforts to get Ceph working were linked on the NixOS wiki for Ceph, I also saw a little line:

Another distributed filesystem alternative you may evaluate is SeaweedFS.

I vaguely remembered looking at SeaweedFS before, but I was focused on getting Ceph working. And in that moment, when Ceph was not building and I didn't have the resources to fix the problem myself, I decided to try it out.

SeaweedFS does 80% of Ceph. It didn't have all the fancy features, but it had the features I wanted:

Distributed across multiple machines
Ability to add or remove storage on the fly
Variable replication copies
Currently built and could be run
Can be mounted on Linux
It uses 30 GB volumes on standard ext4 partitions instead of a custom one I cannot debug
Single Go executable (don't code Go but it only takes twenty minutes build, not twelve hours)
Could create S3/cloud tier backup (Ceph couldn't do that)

Yeah, it didn't have NixOS options but Google and GitHub gave me a starting point for me to puzzle it out on my own. I learned a new library and started with a little partition to see if it worked. Like Ceph, I had a lots of fits and starts, trials and puzzling through it, but eventually got it working.

And it was more scattered in terms of information but it worked.

It didn't do Ceph's “deep cleaning” to detect bit rot on failing drives.

It blew up when you tried to create too many encoding shards without the space.

It blew up when you tried to replicate without having enough nodes.

Around that time, Ceph started building on NixOS again but I was already enamored by SeaweedFS. It was “good enough” for me. It took me about a week to migrate hunks of the Ceph data to the Seaweed, decommission a Ceph drive and make it a SeaweedFS drive. Then repeat until everything was moved over.

A few weeks ago, I tried to consolidate my dad's hard drives into the cluster and ran out of space. I ended up buying a new minicomputer and throwing 11 TB worth of drives into it to give me room (two copies of everything, even the media files). This week, I lost one of those old Dells so I had to shuffle around the volumes with a few commands and it “just worked”. I'm going to replace the dead computer and I'm confident that it will “just work” then too. And, more importantly, I don't have to spend a week to figure out the commands to make it happen since I can just copy/paste a bunch of Nix code and redeploy my servers.

Thoughts

I could hoard less. It takes time and energy to keep the computers running but it makes sure Partner has their Golden Girls, the kids have their videos, and I have my dad's artwork. Also, there is something peaceful about looking at a 29.9 TB partition and having it working smoothly.

+---------------------------------------------------------------------------------------------------------------+
| 1 fuse device                                                                                                 |
+-------------+-------+-------+-------+-------------------------------+----------------+------------------------+
| MOUNTED ON  |  SIZE |  USED | AVAIL |              USE%             | TYPE           | FILESYSTEM             |
+-------------+-------+-------+-------+-------------------------------+----------------+------------------------+
| /mnt/home   | 29.9T | 20.9T |  9.0T | [#############.......]  69.8% | fuse.seaweedfs | fs.home:8888:/         |
+-------------+-------+-------+-------+-------------------------------+----------------+------------------------+

Eventually this is all going to go away. When I die, my family isn't going to be able to keep it going. Like Rutejìmo's stories and my dad's artwork, all this will fade. But I'm going to keep it going as long as I can. And try to find more pages for my book.