I used Jenkins Configuration as Code, JCasC, and GitLab to demonstrate how Jenkins and its configuration (jobs, build process and global configuration) can be version controlled and fully automated. I also integrated with GitLab to show how webhooks configuration can be automated to trigger the jobs in Jenkins.

TL;DR: Here is the code: https://github.com/tomasbjerre/jenkins-configuration-as-code-sandbox

The Problem

When I started with this I was at a client who had several large projects. Each project with several developers working full time. They all developed different systems. They all shared one Jenkins installation with a vast amount of jobs. Some of the problems with this setup are:

  • You don't know how an update of a plugin affects all the jobs.
  • Hard to keep track of what plugins are actually needed.
  • Hard to keep track of what jobs are still needed. Git repositories changes name, gets removed, gets abandoned...
  • Hard to support developers because the way the different projects are using Jenkins diverge.
  • Build scripts are deployed in production for everyone at the same time.
  • Blocks innovation because you will not have confidence enough to refactor build scripts. You will instead just make smallest possible changes to global scripts and pray that it works for everyone.
  • Cannot, easily, revert changes made to the Jenkins installation.
  • Cannot, easily, work locally with the Jenkins installation. Cannot work in feature branches, just like you would in any other application repository.

The Solution

My solution is to express the entire Jenkins setup with scripts. Scripts that are version controlled in a Git repository. Now that JCasC is released we have all parts need.

  • I have a master branch in Git that represents the bleeding edge setup with the latest features.
  • Each project have their own branch in Git, pointing to their setup.

Each project have their own installation of Jenkins. They can decide for themselves when they want to follow the road map and get the latest features. The road map is the path, a bunch of commits in Git, between the projects current branch and the master branch.

A master branch with bleeding edge features

At some point in time a snapshot of the current state, of the Jenkins configuration Git repository, may look like this.

Different projects can use different parents of the bleeding edge master

Here we see that:

  • The Master configuration points to Git commit G. Which means G is the bleeding edge with the latest greatest features.
  • Project E is an early adopter and is only one commit behind the Master.
  • Project D is far behind.

In this snapshot, perhaps, the people in Project D may not have much interest in CI/CD. They just want it to do its thing while they focus on their application code. The people in Project E seems to be more enthusiastic and wants the latest features.

This solution has a bunch of advantages:

  • It encourages innovation. Anyone, in any project, can create their own feature branch in the Git repository to try any idea they might have. They can, easily, setup the entire Jenkins installation locally to fiddle with it. Develop scripts and try ideas. Even push the feature branch to a remote and suggest features to be included in the Master.
  • It reduces time spent on support. When the way the different projects use Jenkins does not diverge, their can be documentation that all projects can use. It also means that if Project C has a problem, they can talk to any member of any of the other projects. They don't necessarily need to talk to a member of the Master project.
  • Gives self confidence to refactor scripts. Ideas can be tested locally without having to be deployed in a shared Jenkins instance.
  • Allows revert of failed upgrades. If a project upgrades and something goes wrong, you simply revert the change in Git and you will be back to where you were before the upgrade.
  • Nice and tidy structuring of jobs. You can organize the Jenkins jobs in folders to create a structure that maps to the structure you have in your Git service. Like in GitLab you have namespaces and I have a Jenkins folder for each namespace.

Implementation

The code, the Git repository containing the entire Jenkins setup, is available here: