Often, the most difficult part of bringing an application into Fedora isn’t getting the application itself to build, it’s the large tree of dependencies the application has adopted, which haven’t been imported into Fedora yet.

Package registries are a core part of the workflow for developers in many modern languages. Rust developers have crates.io, Python developers have pypi.org, Node.js developers have npmjs.com (or pnpm.io… or jsr.io… it’s complicated). Package registries allow developers to easily get reusable libraries, generally pre-built and ready to use.

Fedora’s package repositories offer similar functionality. They provide a collection of reusable libraries that are ready to use. There are several differences from the language-specific package registries, including the requirement that packages in Fedora’s collection have to be built from source in Fedora’s build systems.

In order to support that requirement, Fedora packagers need to wrap each project’s build system in a common build system that their build infrastructure understands. Package maintainers are effectively providing an alternate registry.

One hurdle in this endeavor is that in a typical package registry, a developer can publish multiple releases in parallel. When an application needs a component from a registry, it will typically request information by the name of the package, it will receive information describing the available versions, and it will select a version according to constraints provided by the developer. However, Fedora does not function like a typical registry in this respect. In Fedora, there is only one package by any name, so in order to provide multiple versions, there must be multiple packages that include parts of the version in the name of the component.

Fortunately, we don’t have that constraint while preparing packages. We can build local source repositories that function more like a registry, and sort out the package name specifics once everything is more or less ready to review.

That brings us to the first tools that can make packaging a little easier.

Registry packager

Fedora includes tools designed to make it easier to bundle components from crates.io and from PyPI. Since we aren’t always sure what version or versions we will need when we begin building a complex application, it might be helpful to assemble a description of how to build all of them, similar to the data in the original registry.

I used Claude to construct simple wrappers for Fedora’s registry import tools. These wrappers create a local git repository in which branches represent minor release series of the component. Once the git repo is assembled, we can check out any branch and build the latest release in that minor release series. (If it’s necessary, we could also check out a previous patch and build that…)

For example:

$ ./crate_packager.py fs4
$ cd crate-repos/rust-fs4
$ git branch
* main
  release-0.11
  release-0.12
  release-0.13
  release-0.5
  release-0.6
  release-0.7
  release-0.8
  release-0.9

Build chains

Many of the packages imported in this manner will build, but some of them will reveal more dependencies that need to be added. As the set of dependencies grows, it can be difficult to track the set that’s needed for a specific application and the order in which they need to be built.

It would be helpful to have a tool to not only track this information, but to manage the build of a list of rpm packages in sequence.

Once again, I used Claude to construct a simple program that wraps mock-scm.

Mock is a tool that manages build environments in which package maintainers can build rpm packages, and its “scm” extension supports building a package directly from a source code repository, so that the package maintainer doesn’t need to manually create a source RPM to start the process.

The wrapper is “rpm-build-assist”. This program takes a YAML file that describes what release to use as the base environment for builds, what type of source code repos are used for the packages (which might be dist-git or source-git), where the resulting RPM packages will be saved, and other details of the build process.

As the packager works through the dependency set, they can simply add new dependencies to the beginning of the list. The build-assist yaml file will serve as a record of all of the packages that need to be reviewed together, what version of each package is currently needed, and the order in which they need to be built, while the script automates the process of building them in sequence during development.

base: fedora-44-x86_64
localrepo: /home/gordon/git/nodejs-electron-results

build:
  - type: dist-git
    url: /home/gordon/git
	packages:
	  - rust-walrus:release-0.24
	  - rust-wasmparser:release-0.240
	  - rust-wasmprinter:release-0.243
...
	  - nodejs-playwright:main
	  - nodejs-husky:main
	  - nodejs-electron:main

Automation

Automation through CI can improve this workflow further by moving the actual builds to dedicated compute infrastructure, and it also creates opportunities for groups of maintainers to work together on a collection of packages, coordinated in a shared source code repository.

Claude helped here, too. Claude wrote a basic container action for use in GitHub runners. It has its own workflow to prepare a container image that provides mock and rpm-build-assist, as well as the action.yml that implements the reusable action.

Now, a repo that contains the build-assist.yaml file can also contain a workflow that runs the build chain in GitHub CI.

name: source-git build and test

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: source-git build
        uses: gordonmessmer/rpm-build-assist-action@main

All of these tools are “proofs of concept”, so there are lots of opportunities to improve them. But even at an early stage, they might be useful to Fedora package maintainers who are preparing complex applications.