DLRN builds packages and generates repositories in which these packages will be hosted.

It is the tool that is developed and used by the RDO community to provide the repositories on trunk.rdoproject.org. It continuously builds packages for every commit for projects packaged in RDO.

RDO is completely open source and community driven: anyone can submit patches to improve RDO packaging.

When someone submits a patch for review on review.rdoproject.org, we have a gate job called “DLRN-rpmbuild” that will run DLRN to see if the package builds successfully.

The problem

Until now, that was essentially the extent of our testing coverage on code reviews for distgit packaging projects – and we weren’t satisfied with that.

The reality is that while the packages may build successfully, they could actually be broken. We could potentially carry bad systemd unit files, bad config, missing runtime dependencies, etc.

RDO continuously tests it’s trunk repositories with, amongst other things, WeIRDO and TripleO-Quickstart. These testing pipelines help us stay up to date throughout the development cycles. If we happened to inadvertently merge a bad packaging review, these jobs would eventually pick up the faulty packages and fail, letting us know that the package is broken and we’d go back to square one.

It would be great to test the new packages even before they land in trunk repositories, though.

Testing even before the trunk repositories

I mentioned we already had the DLRN-rpmbuild job that built packages, generated repositories and even uploaded them to Swift as part of log artifact collection.

We needed a way to have integration tests use those uploaded repositories and, after a lot of tinkering and different implementations, came to a satisfying solution.

First, we needed to override the default log path provided by Zuul but we found that it didn’t work properly and we required a patch that had not yet landed upstream. Overriding the default log path was necessary – otherwise, the log path is suffixed by $ZUUL_UUID which is unique to every job.

In our context, we want a “child” job to be able to consume artifacts from a “parent” job. Since the $ZUUL_UUID of the parent job is not exposed to the child job, we are not able to “guess” where the artifacts have been uploaded without some dirty hacking. We needed the artifact upload path of the parent job to be predictable and computable by the child job.

We’re now exposing this process through a jenkins builder on review.rdoproject.org. This builder computes the location where there should be a DLRN repository and configures it if there is one.

This builder enables us to re-use existing WeIRDO integration gate jobs transparently and seemlessly, just by adding the builder to the jobs and making it a children of DLRN-rpmbuild. There will effectively be two DLRN repositories configured:

• the one that was just built in the gate (configured before the job runs)
• the one from trunk.rdoproject.org (eventually configured by WeIRDO)

In our testing, we use yum-plugin-priorities. Since the date (epoch) is embedded in the package NVR (name-version-release) and because both DLRN repositories are configured with priority “1”, the package that was just built will have priority because it is more recent.

There’s still work to do

We’re happy with the implementation and tested it successfully. Now, we need to work on configuring the right jobs for the right projects.

For example, for Cinder packaging reviews, we don’t want to be running integration jobs that don’t involve Cinder. It would be inefficient and a bad usage of our resources.

Both Packstack and Puppet-OpenStack provide great testing scenarios for different projects which are leveraged by WeIRDO.

Still, it will prove rather challenging to choose which scenario to run on which project – even if just for the sheer amount of projects we have to deal with. It’s worth it, though, we will be able to boast better testing coverage as a result and end up with less broken packages landing in trunk repositories.