I have a lot of opinions about CI. I think a CI definition should be very brief and readable. It should use an agent that developers can run locally so that they can build before they push to a remote system. It would be great if the CI process and the release process are the same, so that developers don’t have to maintain redundant build definitions.

Use RPM spec as the body of the CI definition

Since an RPM spec defines a) how to prepare an environment for a build, b) how to build the software and c) how to run the test suite, it satisfies most of the requirements that I have for a CI pipeline.

I was surprised when I could not find a GitHub action that could use an RPM spec for CI, so I put together a proof of concept on a Friday night.

I have a demo project named “libFoo” which is intended to demonstrate a few core concepts about interface evolution, backward compatibility, and symbol versioning. It exists to provide a reference for conversations I have about package managers and dependency generation, and to support tutorials, so it serves nicely as a demo for the new CI action.

The workflow using a “source-git” GitHub action is short and readable. The workflow runs if a developer pushes changes into the “main” branch, or if a developer proposes changes to the “main” branch through a pull request. The workflow runs on a GitHub-provided GNU/Linux system. First, it clones the source code repo, and then it runs a build and test sequence described by the RPM spec. The build is run and tested in “fedora-44-x86_64”, so all of the dependencies will be whatever version is in that release of Fedora. If I want to test the package against different versions of its dependencies, I can also build against other releases of Fedora.

name: source-git build and test

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: source-git build
        uses: gordonmessmer/source-git-action@main
        with:
          mock-root: 'fedora-44-x86_64'

The RPM spec provides a description of the software in the metadata fields at the beginning of the file. The “BuildRequires” fields define the packages that need to be installed to support the build. The “%build” and “%install” sections build the code, and the “%check” section runs unit tests.

Name:           libFoo
Version:        1.1
Release:        %autorelease
Summary:        A simple library
License:        MIT
Source0:        %{name}-%{version}.tar.gz

BuildRequires: gcc
BuildRequires: make

%description
A simple example spec file for Fedora.

%prep
%setup -q

%build
%make_build

%install
%make_install

%check
make test

%files
%{_bindir}/Bar
%{_includedir}/foo.h
%{_libdir}/libFoo.so
%{_libdir}/libFoo.so.1
%{_libdir}/libFoo.so.1.1.0

%changelog
%autochangelog

As CI definitions go, this is all pretty readable. Developers can use RPM (or Mock) to build and test the software locally. The same process provides installable software, which is usable both for further interactive testing or for release.

Chain builds

That process is good for components that are well isolated, but complex projects might be developed in multiple repositories that need to be built in sequence.

The “libFoo” repo provides both an application and a shared library. If those components were developed separately, the “Bar” application might need to build and test “libFoo” as a part of its CI process.

Let’s look at another GitHub action that uses a chain build. This workflow is similar to the previous one, except that it uses a different action for the second step.

name: source-git build and test

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: source-git build
        uses: gordonmessmer/rpm-build-assist-action@main

The new action uses a short script called “rpm-build-assist” that simplifies the process of building a sequence of source code repositories. It is basically a fancy loop over source code repos that provide RPM specs.

The new action requires a new configuration file to define which repositories to loop over. This configuration file, .rpm-build-assist/build-assist.yaml describes the release in which the packages will be built (again, “fedora-44-x86_64”), the base location of the repositories, and a list of repositories at that location, along with the name of the branch to build.

base: fedora-44-x86_64

build:
  - type: git
    url: https://github.com/gordonmessmer
    packages:
      - libFoo:v1.1
      - libFoo-bar:main

The “libFoo” RPM spec appears above, and the spec for the “bar” application is here. It lists “libFoo” as a build requirement, so that the package built in the previous step will be installed during its build.

Name:           libFoo-bar
Version:        1.1
Release:        %autorelease
Summary:        A simple app
License:        MIT
Source0:        %{name}-%{version}.tar.gz

BuildRequires: gcc
BuildRequires: automake
BuildRequires: libFoo

%description
A simple example spec file for Fedora.

%prep
%setup -q

%build
autoreconf -fiv
%configure
%make_build

%install
%make_install

%check
make test

%files
%{_bindir}/Bar

%changelog
%autochangelog

Again, this action is a proof of concept. A future version will run multiple builds in parallel when multiple build-assist.yaml configs are given. This will allow the developer to provide configurations that describe different versions of supported dependencies, such as building the “bar” application with “libFoo:v1.0” and “libFoo:v1.1”.

That’s important because software compatibility is expected to have both a lower and an upper boundary, so it’s important that CI pipelines build and test compatibility with various dependency versions in order to ensure that software remains compatible with the versions that developers want to support.

And that brings us to two points of view that might be unconventional:

RPM is a CI agent

Most users think about RPM as a local package manager, and it is that. But RPM is not merely the program that installs and removes packages from a host. It’s also the software that handles macros, build environment setup, and build processing. The UNIX design philosophy is often described as “do one thing and do it well”, Composing a CI agent from a program that integrates with an SCM service and polls for events, and a program that runs a build (one that can be reused by otherwise incompatible SCM services) is the UNIX design in action.

Fedora is a package registry

Many developers think about package registries like PyPI and crates.io as a different category of service than distributions, but they have a lot more characteristics in common than they have differentiation. Each offers a collection of software that may include applications for users and shared libraries for developers. Each offers a service that publishes packages and metadata about the packages. Each includes a package manager intended to install, update, and remove packages on client systems using the online service.

The biggest difference between the a distribution and a package registry is that most package registries are language-specific and platform-agnostic. That is, Python developers use PyPI whether they are on Windows, macOS, or Linux systems. Distributions, on the other hand, are language-agnostic but platform-specific. Each distribution is its own platform, and offers package that run on that platform.