With the recent drama of GitLab, both with the CI/CD changes and then more recent possible threat of deleting old repositories, I continue my migration to a local Gitea instance, https://src.mfgames.com/ for the bulk of my code and writing.

For the most part, migrating is just a matter of shuffling data. I have a lot of repositories, both active and inactive, and it will take me months to move them over. Plus I haven't decided if I'm going to purge them from my GitLab account so there is a single source of truth or just mirror back to them.

Currently, the most difficult task was figuring out how to handle the build processing. I've mentioned previously that I use Conventional Commits and Semantic Release fairly heavily. I've branched out a little from there using Lefthook and my project layout.

Currently, the CI does the following:

Build the project
Test various conditions including valid commit messages
If the commits indicate a new build:
1. Tag it
2. Build the release version
3. Create a release on Gitea

This changes over time, but it is the basic pattern.

Tags and Git Depth

Woodpecker does not automatically download the needed tags for semantic-release (and GitVersion). This means that the .woodpecker.yml file needs to include tags.

clone:
    git:
        image: woodpeckerci/plugin-git
        settings:
            tags: true
pipeline:
    # The pipeline elements

Unike GitLab, which only limits to the last ten commits, it appears that Woodpecker downloads the full repository by default which is also needed by GitVersion because it calculates every version. Not entirely sure about semantic-release logging indicates it doesn't need the full repository, just enough back to find a version.

Building and Testing

To support task branches, I have a basic build and test code that runs on pushes and pull requests. This lets me identify bugs earlier and catch typos with my commits.

build:
    image: registry.gitlab.com/dmoonfire/nix-flake-docker:latest
    commands:
        - nix develop --command scripts/build.sh
    when:
        # We need both "tag" for the next section.
        event: [push, pull_request, tag]
        tag: v*

test:
    image: registry.gitlab.com/dmoonfire/nix-flake-docker:latest
    commands:
        - nix develop --command scripts/test.sh
    when:
        event: [push, pull_request]

From the build tasks, you can see that I'm using my current project layout which uses scripts in the scripts/ folder instead of npm run or dotnet run. This is to make it easier to work with polyglot plus works around the issue that I need to use nix develop to get into my reproducible environment since the Docker image doesn't automatically do that. This is because both Gitlab and Woodpecker use the image which bypasses initialization files and I couldn't have it run direnv allow automatically to set up environment variables.

One thing that is missing is that Woodpecker doesn't have a clean mechanism for temporary build artifacts. I can't upload the build files and then download them so I can see the final results. Instead, I have to script it out or use a S3 plugin.

Building on Versions

With most cases, I build the release version of the project when the conventional commits indicate that there is a new version (feat and fix). This is an additional pipeline that comes after the test: line.

release-main:
    image: registry.gitlab.com/dmoonfire/nix-flake-docker:latest
    commands:
        - export DRONE="true" # Required to convince `env-ci`
        # semantic-release needs this locally
        - git branch $DRONE_BRANCH origin/$DRONE_BRANCH
        - nix develop --command scripts/release.sh
    secrets:
        - gitea_token
        - git_credentials
    when:
        event: push
        branch: main

There are a number of things in this block that took me a while. The first is the event and branch. We only do releases on the main (I'm still moving away from master as racist language).

The second is the export DRONE line. At the time I set this up, env-ci wasn't aware of Woodpecker, but it was recently added thanks to 6543 on the Woodpecker Matrix channel, #woodpecker-ci:matrix.org. I don't know when the latest semantic-release will have it, but it shouldn't be needed soon, if not already.

The third is the git branch line in the above script. Woodpecker creates a detached head, as does Gitlab. But when it doesn't do is also create a local branch for the one being created. This causes a problem because the release process appears to “jump” to the branch to figure out the changes between the detached head (the commit being built) and the actual branch.

Finally, we have the secrets. semantic-release automatically picks up $GITEA_TOKEN for the release process but also needs $GIT_CREDENTIALS to verify Git access.

The token is easy, that is what given by Gitea for the user.

GIT_CREDENTIALS is slightly harder, it is a colon-separate tuple of the user name and the Gitea access token. From observations, the Gitea is basically just a bunch of URL-safe characters, so the URL-escaping isn't needed in my case (you need to URL-escape the left and right of the colon but not the colon itself).

$ export GITEA_TOKEN=9fc6d72c72e4b149f07491a0b2d3ec9215d57caf
$ export GIT_CREDENTIALS="dmoonfire:$GITEA_TOKEN"

These need to be set on a per-project basis since Woodpecker, unlike GitLab and sourcehut, there doesn't appear to be a good way of having a shared set of secrets for projects (GitLab has organization/group level secrets, sourcehut has the secret storage). This means I have to set the same GITEA_TOKEN and GIT_CREDENTIALS for all 80+ of my Fedran repositories†.

† Woodpecker has a CLI, woodpecker-cli which will let me automate that. I will use that.

Create Release on Tag

One thing I'm moving toward is creating a release entry on the forge. Since this happens after the build process and Woodpecker uses a different Docker image, I need it to be a post-release event so I hang it off the tagging process instead of the push to main.

6543 came to my rescue again with this one, so that is why there are those two “tag” elements in the script above. I also have a new stanza for the release process:

release-gitea:
    image: plugins/gitea-release
    settings:
        base_url: https://src.mfgames.com
        files:
            - "*.pdf"
            - "*.epub"
        api_key:
            from_secret: gitea_token
    when:
        event: tag
        tag: v*

If I didn't have the event: and tag: in the build: stanza, it wasn't working for the tags. This caused me some difficulties because I usually treat hashes as being unordered, but Woodpecker uses file order for processing pipelines. So, I needed to have the build: target build the file (with the correct version because it was tagged) and then release-gitea: to use that output for the release process. The test: and release-main: are skipped because they don't have those events listed.

In addition, secrets are handled differently when done as a parameter for a plugin. That is why I have the from_secret: element in the above script. This inconsistency threw me for a few days.

Putting it Together

If you want to see the final version, check out this example which has my current version as a single file.

Conclusion

I'm happy to move over to Woodpecker (you know, except for the cost of hosting) both because of the control and the challenge. I also don't have a need for speed, so if it takes a while to get through the queue, I'm okay.

Metadata

Categories:

Development

Tags: