If I wrote the reasons and methods I got to this point, I haven't either tagged properly or never got around to writing it. Regardless, here is a very brief history of these tools and how they evolved up to this point. Hopefully it will explain why I did certain things and which problems I was trying to solve.

Series

I appear to be writing a short series of post about the tools I use for publication and writing.

1. Semantic Versions and Releases: Why semantic versioning helps with the writing process.
2. Evolution of MfGames Writing: A brief history and reasoning behind the tools.
3. First Steps Using MfGames Writing: Starting a new project with MfGames Writing.
4. Adding Content to MfGames Writing: Adding front and back matter to novels.
5. Working with MfGames Writing, CI, and Docker: Adding automatic building with commits.
6. Additional Formats for MfGames Writing: How to create PDF, MOBI, DOCX, and HTML versions.
7. Theming for MfGames Writing: A light introduction on how to customize the output.
8. Integrating Semantic Versioning into MfGames Writing: Tying semantic releases into the process.

Markdown and YAML

Let's start with the start of this: I write in Markdown. I've written a few posts about the process and reasons, so I won't go into those. If you want a short version: text files don't corrupt and source control means I can work across multiple machines without accidently deleting an entire chapter.

The problem with Markdown is that most people don't want to read a text file. It has no formatting. It doesn't look pretty. It also doesn't go into books very well. Normally, to get around this, many authors use tools like Microsoft Word or Scrivner (which I recommend to non-technical writers) which allow for writing a story/novel and then formatting it into something usable for readers. I don't use Scrivner myself because it uses binary files (doesn't play well with Git) and doesn't run nicely on Linux (my primary platform).

Most people aren't burdened by these self-inflicted limitations.

I also use YAML to keep track of useful information about the page.

Model-View-Controller as Applied to Writing

In development, there is the concept of the Model-View-Controller. This consists of three parts:

• The model is the logic and data.
• The view renders/formats the model into something useful.
• The controller changes the model.

The nice thing is that when writing in Markdown, I don't worry about formatting. I have four basic things: bold, italic, blockquotes, and section breaks. I don't really need anything else because of my writing style. I also think this contributes to my productivity, if I can't play with it, I can't distract myself. This is basically the model of my novel; the chunky bits in a text file that constitute the contents of the novel.

The view is what takes that Markdown + YAML file and generates something useful or pretty out of it. That can include:

• Online Reading: HTML pages for reading online
• PDF: PDF for print-ready output.
• EPUB: Ebook formats for those who like readers.

The controller is what I write with the novel. I basically use Atom or Notepad++. This is also where Author Intrusion will fit into the process.

The nice part about having the contents (the model) and the output (the view) is that they are completely separate in this step. If I decided to change the font of my books, I just update the view and reformat the books. All the words and everything will come over and I will just have a new appearance. The same if I want to change how epigraphs are formatted or want to support a new ebook format.

Automation

Having a separate view from the model also means I can automate the process of formatting and typesetting the book. This is useful because it is a tedious job formatting the book. Yes, I don't do it that often but I also like consistency. So as my “style” evolves, I also find that I go back and tweak the previous versions for consistency. For example, I moved from a 2 cm lead on the chapter pages to a 4 cm because I thought it looked better. I also made the epigraph a little larger because I can read 8 pt font easily but not everyone else.

Automation also takes out the little details that throw me. I know some books that start with “Chapter 1” and “Chapter 2”, but somewhere in the middle they switch to “Chapter Twenty-Two”. Or sometimes using typographical quotes and other time using ticks. These are things that will throw out a detail-oriented person (like me) but I also feel make an unprofessional looking book.

These are things that Microsoft Word and Scrivner will do, some better than others. I want the same thing because my goal is to create as technically correct and beautiful book as I can possibly create.

MfGames Writing sits in this place, to automate the formatting and handle all of the little details while giving me the ability to focus on one thing—the words—and let computers handle the rest.

Early Versions

When I started this epic quest of trying to automate formatting and typesetting, I started with a simple program: Make. It basically used a few tools in Linux to transform the Markdown into something usable. Shell scripts ended up not being enough, so I supplemented those with a combination of Perl, Python, and C# tools. Each language had its strengths and weaknesses (the biggest is how they handle Unicode since I use accents heavily).

Where most people write a little game or card program with new languages, I write a publication system.

At the time, those languages didn't have good support for installation. Both Perl and Python have a tendency to install packages for the entire machine or a user. C# didn't have NuGet at the time, so it was copying files around (plus it was always the weakest of the three implementations). This meant I could only have one version at a time, so I basically directed everything to my /home/mfgames-writing/ folder and used that.

Which… was good enough until I started changing my build process. I've been using this for about a hundred books and short stories now. The novels were the hardest because I would add features for fancy headers (like the PlayStation controller on top of Fangs for Nothing) or epigraphs in my page. As I went along, the process got better.

It also broke older versions. A template would change, or the flow. I started having to have v1, v2, and v6 hanging around because of the drastic changes as the tools got better at formatting and typesetting. Every time I would rewrite it, I would have slightly different input because of the language changes, and that would break older ones.

That's when I realized I needed to be able to “pin” a version of the tools for a specific book.

The latest attempt at C# was to take advantage of NuGet but then I realized the default NuGet implementation also created a machine-wide version.

I'm aware there are virtual environments for Python. I didn't know about them at the time. After I've played with them, I still find them a bit cumbersome compared to using the approach I went with.

TypeScript and NPM

I finally figured it out when I started playing with TypeScript. I like learning new languages and I was looking for a replacement for my scripting language (I'm very good at Perl and pretty good at Python). One of the tools with JavaScript based languages is NPM which is the package management (much like NuGet).

The beauty of NPM is that it is self-contained inside the development projects. All of the required files go into node_modules/ which means that each project is a folder that contains everything it needs to run and build. This also means that different projects can have different versions of the same library.

This ended up being the solution I went with. A self-contained, project-level packaging system that would let me evolve the tools for one version but still keep another novel at an earlier vesion so it doesn't break.

Fast-forward about a year of coding and tooling and we get to where we are, MfGames Writing.