Enforcing Standards with NixOS

2023-12-02T06:00:00Z

Some time ago, I stumbled into std, a batteries-included development stuff. It looks like something I would really like to get into, mainly because it gave off notes of Buck2, which is something that interests me when dealing with microservice ecosystems and polyglot frameworks. And I know I love a polyglot solution to problems.

There were a few things that I fought again. I didn't care for the menu system that always shows up (noise), it's instability (still alpha), and the difficulty getting it to work with my way of thinking. I could have worked on some of those and figure out how to accept what I couldn't change and alter what I needed to be productive.

I don't really have that energy at the moment. I'm painfully aware that my time and attempt budget has been eroded by my family, drama, and the other things going on in my life. I find that I don't have the energy to do much and getting std to play with me was one of those things I decided to bump.

Automation Tools

However, there were some things I really liked about std that I wasn't aware of, namely Nixago which is a way of having the shell hook of a Nix setup automatically write out the various configuration files for things like Conform for Git messages (I like my conventional commits), EditorConfig for formatting, and Lefthook to make sure everything is honored. Standard also taught me about Treefmt which is a single command to reformat a command base.

There was also a way of doing arbitrary configurations, such as maybe setting up my project configuration files or handling other things, but I couldn't figure out how with a cursory look.

In short, all of those things I like to handle automatically instead of remembering all the little details.

With me getting rid of std, I wanted to keep this. Ideally in a manner that I could eventually create a flake of my common configurations and then apply them to every story or programming project.

Necessity Calls

Getting rid of std meant I hand to figure it out. Last night, I sat down and went through the code with my growing skill at Nix (I still do not enjoy the language but I'm getting more fluent with it). I'm also messing with FlakeHub so you'll see some elements from that library.

I already use direnv for setting up my flakes as I enter directories. That is part of my normal tool set and I plan on using that for a great deal of time.

Layout

I like small, individual files. Naturally this means I would like every automated system to have its own file but grouped together in a folder to make it obvious how they are used. (Needless to say, I don't advocate coding without folders but that is also how I work/)

$ find src -type d
src
src/configs

Inputs

The first part is pulling in the inputs for Nixago and its extensions.

# flake.nix
{
  inputs = {
    nixpkgs.url = "https://flakehub.com/f/NixOS/nixpkgs/*.tar.gz";

    nixago.url = "github:jmgilman/nixago";
    nixago.inputs.nixpkgs.follows = "nixpkgs";

    nixago-exts.url = "github:nix-community/nixago-extensions";
    nixago-exts.inputs.nixpkgs.follows = "nixpkgs";
  };

  # Flake outputs that other flakes can use
  outputs = inputs @ { self, nixpkgs, nixago, nixago-exts }:
    let
      # This bit comes from Flakehub's init and seems to be a reasonable pattern.
      supportedSystems = [ "x86_64-linux" ];
      forEachSupportedSystem = f: nixpkgs.lib.genAttrs supportedSystems (system: f {
        inherit system;
        pkgs = import nixpkgs { inherit system; };
      });
    in
    {
      devShells = forEachSupportedSystem ({ system, pkgs }:
        let
          # This pulls in the configurations from the configuration directory.
          configs = import ./src/configs/default.nix { inherit system pkgs nixago nixago-exts; };
        in
        {
          default = pkgs.mkShell {
            # Pinned packages available in the environment
            packages = with pkgs; [
              git         # Needed for life until I find something more awesome
              nixpkgs-fmt # Needed for Lefthook
              treefmt     # Needed for Lefthook
              lefthook    # Needed for Lefthook
            ];

            # Configuration setup
            shellHook = ''
              ${configs.shellHook}
              lefthook install
            '';
          };
        });
    };
}

Configurations

The basic default is just so I have a single line to configure all the libraries. This just acts as an index file.

# src/configs/default.nix
inputs:
inputs.nixago.lib.${inputs.system}.makeAll [
  (import ./conform.nix (inputs))
  (import ./editorconfig.nix (inputs))
  (import ./lefthook.nix (inputs))
  (import ./prettier.nix (inputs))
  (import ./treefmt.nix (inputs))
]

Prettier

Prettier was the first one I used, since I have very little customization in it. The nixago-exts is an extension library that figures out a lot of the formats so I don't have to.

# src/configs/prettier.nix
inputs @ { system, nixago, nixago-exts, ... }:
nixago-exts.prettier.${system} {
  data = {
    printWidth = 80;
    proseWrap = "always";
  };
}

Lefthook

Lefthook's configuration is the same, but you'll notice there is no data = element like most of the others. This threw me because it is different than the others. I also found that I had to add && git add {staged_files} from most of the examples I saw others when I commit, it would reformat the code but then leave them modified for the next check in. Adding the files fixes that and keeps things relatively speedy.

You also can see how I refer to specific paths for the executables while cleaning up the code.

# src/configs/lefthook.nix
inputs @ { system, pkgs, nixago, nixago-exts, ... }:
nixago-exts.lefthook.${system} {
  commit-msg = {
    commands = {
      # Runs conform on commit-msg hook to ensure commit messages are
      # compliant.
      conform = {
        run = "${pkgs.conform}/bin/conform enforce --commit-msg-file {1}";
      };
    };
  };
  pre-commit = {
    commands = {
      # Runs treefmt on pre-commit hook to ensure checked-in source code is
      # properly formatted.
      treefmt = {
        run = "${pkgs.treefmt}/bin/treefmt {staged_files} && git add {staged_files}";
      };
    };
  };
}

As a side note, I have not found a single fast C# reformatter that only handles a few files. It has been intensely frustrating because I really like ReSharper's “Silently Clean” feature and I don't have a way of doing it from the command line in a reasonable period of time.

Conform

Conform is nice because it enforces Git commit messages for conventional commits.

# src/configs/conform.nix
inputs @ { system, nixago, nixago-exts, ... }:
nixago-exts.conform.${system} {
  commit = {
    header = { length = 89; };
    conventional = {
      # Only allow these types of conventional commits (inspired by Angular)
      types = [
        "build"
        "chore"
        "ci"
        "docs"
        "feat"
        "fix"
        "perf"
        "refactor"
        "style"
        "test"
      ];

      # If you want scopes, then add:
      #scopes = ["allows" "scopes" "here"];
    };
  };
}

Treefmt

Cleaning up code is something that is tedious but really needs to be done to lower the bar of allowing others into the code. It is also something that can be “mostly” automated, which I'm also in favor of. Sadly, there are gaps in the tools that I want, like a C# or Rust formatter that will organize members (such as grouping public properties together and making them alphabetical), but I can live without those.

# src/configs/treefmt.nix
inputs @ { pkgs, ... }:
let
  data = {
    formatter = {
      prettier = {
        command = "${pkgs.nodePackages.prettier}/bin/prettier";
        options = [ "--write" ];
        includes = [
          "*.css"
          "*.html"
          "*.js"
          "*.json"
          "*.jsx"
          "*.md"
          "*.mdx"
          "*.scss"
          "*.ts"
          "*.yaml"
          #"*.toml"
        ];
        excludes = [ "**.min.js" ];
      };

      nix = {
        command = "nixpkgs-fmt";
        includes = [ "*.nix" ];
      };
    };
  };
in
{
  # I don't understand the reason why many Nix examples define
  # the data in the let section and then just inherit it here.
  inherit data;
  output = "treefmt.toml";
}

EditorConfig

I think one of the best things that came out of the last decade or so of coding was a slow migration to having a semi-universal file for configuring line endings, trimming white space, and the others. Also, both Microsoft and Jetbrains have embraced EditorConfig so I can check in a file that reduces the trivial (but needed) rejects for pull requests. I want more tools to use this and extend them because I don't want to bother with line indents, tabs verses spaces (tabs lost, but I've accepted spaces now), and formatting rules (braces on new lines).

# src/configs/editorconfig.nix
inputs: # I don't use the inputs, I just wanted all the calls in `default.nix` to be consistent.
let
  data = {
    root = true;

    "*" = {
      end_of_line = "lf";
      insert_final_newline = true;
      trim_trailing_whitespace = true;
      charset = "utf-8";
      indent_style = "space";
      indent_size = 4;
      indent_brace_style = "K&R";
      max_line_length = 80;
      tab_width = 4;
      curly_bracket_next_line = true;
    };

    "*.md" = {
      max_line_length = "off";
    };

    "package.json" = {
      indent_style = "space";
      indent_size = 2;
      tab_width = 2;
    };

    "{LICENSES/**,LICENSE}" = {
      end_of_line = "unset";
      insert_final_newline = "unset";
      trim_trailing_whitespace = "unset";
      charset = "unset";
      indent_style = "unset";
      indent_size = "unset";
    };
  };
in
{
  inherit data;
  hook.mode = "copy";
  output = ".editorconfig";
  format = "toml";
}

Obviously, the C# version is huge with lots of settings to fit my standards.

Benefits

The nice part about this is all I have to do is change into the directory and direnv will automatically make sure all the files are correct and up to date. Since I'm mostly on the command line, this works out beautifully for me and how I work.

$ cd bakfu
direnv: loading ~/src/bakfu/.envrc
direnv: using flake
evaluating derivation 'git+file:///home/dmoonfire/src/bakfu#devShells.x86_64-linux.default'
nixago: updating repository files
nixago: '.conform.yaml' link updated
nixago: '.editorconfig' copy is up to date
nixago: 'lefthook.yml' link updated
nixago: '.prettierrc.json' link updated
nixago: 'treefmt.toml' link updated
direnv: export +AR +AS +CC +CONFIG_SHELL +CXX +HOST_PATH +IN_NIX_SHELL +LD +NIX_BINTOOLS +NIX_BINTOOLS_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu +NIX_BUILD_CORES +NIX_BUILD_TOP +NIX_CC +NIX_CC_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu +NIX_CFLAGS_COMPILE +NIX_ENFORCE_NO_NATIVE +NIX_HARDENING_ENABLE +NIX_LDFLAGS +NIX_STORE +NM +OBJCOPY +OBJDUMP +RANLIB +READELF +SIZE +SOURCE_DATE_EPOCH +STRINGS +STRIP +TEMP +TEMPDIR +TMP +TMPDIR +__structuredAttrs +buildInputs +buildPhase +builder +cmakeFlags +configureFlags +depsBuildBuild +depsBuildBuildPropagated +depsBuildTarget +depsBuildTargetPropagated +depsHostHost +depsHostHostPropagated +depsTargetTarget +depsTargetTargetPropagated +doCheck +doInstallCheck +dontAddDisableDepTrack +mesonFlags +name +nativeBuildInputs +out +outputs +patches +phases +preferLocalBuild +propagatedBuildInputs +propagatedNativeBuildInputs +shell +shellHook +stdenv +strictDeps +system ~PATH ~XDG_DATA_DIRS

Moving Parts

There is a problem with this, in that it is a lot of moving parts to basically write out a file that could easily be checked into code once and be done with. I fully admit that we are going through a lot to hoops that could easily be done with simple files with only one exception.

The biggest exception is Lefthook. Someone needs to run lefthook init after the repository is cloned to ensure the hooks are all configured so it enforces the commit messages and makes sure the code is formatted before committing. Since Git won't ever provide that, having the shell hook from the flake enforce it cuts out a tedious step that is easily overlooked.

The other reason for going with this approach is my ability to update it. Each of my stories are in their own repository for a variety of reasons, but the structure and layout of them are typically manipulated en masse as I update a standard. I also do theme and style changes across the boards, such as finding a new font or fixing the ebook generation.

With the flake setup, I could easily migrate these configurations to a dedicated flake that is shared across all of them, and then just update the lock for that file to enforce the latest iteration of an evolving standard.

And standards are evolving. While I have common patterns (braces on newlines), how I format the code, organize files, or hook up patterns changes. Sometimes it is a little incremental change, sometimes it is a sweeping change as I switch build systems or introduce a standard format. Last year, I set it up so every project would generate a EPUB and PDF file and work with my Gitea and Woodpecker CI setup.

If I can make those changes cut across all the projects, then it is less effort for me to get conformity but also let me work in an environment I'm comfortable with. Being able to make sure every tool I want is available (such as Author Intrusion or markdowny) means I don't have to think about the plumbing and just do the part that is fun: write.