How to spin it up from scratch or migrate an existing instance

Jenkins Configuration as Code is finally here. In this blog I’ll cover how to convert an existing Jenkins instance to JCasC and also how to start from scratch. Let’s get it up and running!

What is Jenkins Configuration as Code (JCasC)?

JCasC is a Jenkins plugin that allows you to store and maintain your Jenkins configuration in a YAML file. I provided an outline of the main benefits in a previous blog post: Jenkins Configuration as Code - Automating an automation server.

Installation

Scenario 1: Starting from scratch with Docker

To demonstrate how to start from scratch with Docker I’ll be using our Praqma/praqma-jenkins-casc GitHub repository. It contains a Dockerfile and docker-compose.yml ready for use. You can use your own images or you can fork ours and use them.

Let’s take a look at the repository contents: First up is the master/Dockerfile. This is based on the praqma/jenkins4casc:1.0 Docker image, which in turn is based on the official Jenkins Docker image. It comes with JCasC pre-installed and its Setup Wizard disabled, so when you spin up your container it’s ready to go.

FROM praqma/jenkins4casc:1.0

LABEL maintainer="man@praqma.net"
COPY plugins_extra.txt /usr/share/jenkins/ref/plugins_extra.txt

ARG JAVA_OPTS

RUN xargs /usr/local/bin/install-plugins.sh < /usr/share/jenkins/ref/plugins_extra.txt

Up next is the master/plugins_extra.txt file which contains a list of plugins that will be pre-installed on the Jenkins instance. If you want more plugins to come with your instance, add them here.

Next, there’s the docker-compose.yml file which is used to start the container. This one’s worth a closer look:

version: '3.3'

services:
  jenkins:
    build: #TODO: Once plugin installation is handled through c-as-c this can be removed and replaced with 'image'.
      context: ./master
    ports:
      - 80:8080
      - 50000:50000
    volumes:
      - jenkins_home:/var/jenkins_home
      - ./jenkins.yaml:/var/jenkins_home/jenkins.yaml
    secrets:
      - github  #Github username with password as an example.  Purpose here is to have a user with write-access for special cases.  Like the git-publisher.
      - adminpw  #Initial administrator user password.  Added for debugging purposes.  Should be removed prior to go-live.
      - agent_private_key #ssh private key for connecting ssh slaves
    environment:
      - CASC_JENKINS_CONFIG=/var/jenkins_home/jenkins.yaml #Use a github repo.  Be aware that the raw content is cached, so it can take a couple of minutes to take effect
secrets: #Be careful with newlines in your secrets, make sure to remove them before running docker-compose up.
  github:
    file: /var/deploy/secrets/github
  adminpw:
    file: /var/deploy/secrets/adminpw
  agent_private_key: #This should point to the private key you want your jenkins master to use when connecting to slaves.  So in AWS for example this would be your .pem file
    file: ~/.ssh/id_rsa
volumes:
  jenkins_home:

The Jenkins configuration file, jenkins.yaml, is mapped to /var/jenkins_home/jenkins.yaml (line 12), and sets the CASC_JENKINS_CONFIG environment variable to point to our newly mapped config file (line 18). Great! Now the JCasC plugin has access to our Jenkins configuration file and knows where to find it.

Docker secrets are also declared (line 19+) allowing us to pass in sensitive data such as credentials and SSH keys. You’ll need to tweak these to point to the secrets you want to pass in.

If you want to replicate my setup you’ll need to put the SSH key you want to use in ~/.ssh/id_rsa and two text files containing a password at:
/var/deploy/secrets/github
/var/deploy/secrets/adminpw

If you decide to change the secret definitions make sure to update the passing in of secrets as well (line 13+).

Now you’re ready to move on to the next section.

Note: Text files containing a password should contain only a single line. Watch out for editors (like Atom) that automatically add a trailing newline when you save a file.

Scenario 2: Getting started on existing Jenkins instances

This section focuses on introducing JCasC to a running Jenkins instance, freshly set up or otherwise. This should go without saying, but avoid tinkering with your production instance without properly testing your configuration changes.

Head over to the Plugin Manager under Manage Jenkins to install the Configuration as Code and configuration-as-code-support plugins. You’ll find them under the Available tab. Use the filter for convenience, select the plugins, and hit Install without restart.

Once the plugins are installed you can continue to the next section.

Installation

Configuration

Scenario 1 & 2: Initial configuration

How you run Jenkins doesn’t influence what your configuration file looks like, so you’re able to use the same file for both Dockerized and classic Jenkins instances. The only requirement is that the plugins you configure must be installed.

For Dockerized Jenkins instances refer to the “Scenario 1: Starting from scratch” section above on pre-installing plugins. For classic setups I’ll rely on manual installation for the moment. Configuration as Code can handle installing plugins, but I’ll cover that in a separate blog post.

For now, here’s what your very first JCasC configuration file can look like:

_jenkins.yaml_

jenkins:
  systemMessage: "Hello, world!"

Scenario 1: Using your configuration file with Docker

In the “Scenario 1: Starting from scratch” section above I showed how the docker-compose.yml file sets up jenkins.yaml as our configuration file. I encourage you to edit that file directly. Either start with the basic example above, do your own thing, or use parts of my configuration. Once you’re happy with the configuration file spin up your container by calling:

docker-compose up –build

Once your container is up and running you can access your Jenkins instance at localhost:8080.

Scenario 2: Using your configuration file in a classic setup

Once you’ve installed the plugin you’ll find the Configuration as Code menu item under Manage Jenkins. This is where you’ll be configuring JCasC.

Configuration as Code

Under Configuration as Code change the Path or URL setting to point to your Jenkins configuration file and hit Apply new configuration. Voila! And no restart required!

Add Source

You might run into the following error: “Configuration cannot be applied. File or URL cannot be parsed or does not exist”. If so, check that the path is correct and the Jenkins user has read access.

If you run into an exception or the following error: “Invalid configuration elements for […]”, Jenkins failed to parse your configuration file - it likely contains a syntax error and isn’t valid YAML.

Scenario 1 & 2: Resources for extending your configuration

Now you have your “Hello, world!” configuration up and running. In the unlikely event that’s all the configuration you want, you can stop reading here. Otherwise, let’s press on!

Figuring out what to add to your YAML file and how to configure various settings can be tricky in the beginning, but there’s a few resources to help you out.

Plugin contributors have already added numerous examples in the JCasC GitHub repository. This is a great place to find out how to configure specific plugins.

Under Manage Jenkins > Configuration as Code, near the bottom of the page, you’ll find handy reference links to both documentation and schemas you can use to validate your YAML file.

In our Praqma/praqma-jenkins-casc GitHub repository you’ll find another example configuration file, although this one requires the installation of various plugins listed in the plugins_extra.txt file.

Extend your configuration file as you see fit - just make sure you maintain a valid entry hierarchy and, of course, a valid YAML syntax.

Example run-through

In this section I’ll be explaining the contents of my Jenkins configuration file from the Praqma/praqma-jenkins-casc GitHub repository.

jenkins:
  systemMessage: "Welcome to the demo setup for Jenkins Configuration as Code plugin.  For more information look in the official repo with our demo setup: https://github.com/Praqma/praqma-jenkins-casc"
  agentProtocols:
    - "JNLP4-connect"
  securityRealm:
    local:
      allowsSignup: false
      users:
       - id: demoAdmin
         password: ${adminpw:-passw0rd}
  authorizationStrategy:
    globalMatrix:
      grantedPermissions:
        - "Overall/Read:anonymous"
        - "Job/Read:anonymous"
        - "View/Read:anonymous"
        - "Overall/Administer:authenticated"
  crumbIssuer: "standard"
credentials:
  system:
    domainCredentials:
      - credentials:
          - usernamePassword:
              scope:    SYSTEM
              id:       github-user
              username: ReleasePraqma
              password: ${github}
          - basicSSHUserPrivateKey:
              scope: SYSTEM
              id: agent-private-key
              username: agentuser
              passphrase:  ""
              description: "ssh private key used to connect ssh slaves"
              privateKeySource:
                directEntry: # The key 'agent_private_key' resolved from a docker secret defined in docker-compose.yml
                  privateKey: ${agent_private_key}

Most importantly, I set up a user that I can always connect with. I added, among other things, a local Security Realm with disabled signups and the “demoAdmin” user. To configure user’s password I expand an environment variable passed in as a Docker secret, or simply present in the environment. If the environment variable isn’t available it falls back to passw0rd.

Always make sure you have a user configured or you could lock yourself out.

securityRealm:
    local:
      allowsSignup: false
      users:
       - id: demoAdmin
         password: ${adminpw:-passw0rd}

I also configured an authorizationStrategy to restrict anonymous user permissions.

 authorizationStrategy:
    globalMatrix:
      grantedPermissions:
        - "Overall/Read:anonymous"
        - "Job/Read:anonymous"
        - "View/Read:anonymous"
        - "Overall/Administer:authenticated"

Up next is adding some credentials. These are used for authentication against other applications, for example when you’re cloning down Git repositories in your pipelines. I again rely on Docker secrets and environment variables for these.

domainCredentials:
      - credentials:
          - usernamePassword:
              scope:    SYSTEM
              id:       github-user
              username: ReleasePraqma
              password: ${github}
          - basicSSHUserPrivateKey:
              scope: SYSTEM
              id: agent-private-key
              username: agentuser
              passphrase:  ""
              description: "ssh private key used to connect ssh slaves"
              privateKeySource:
                directEntry:
                  privateKey: ${agent_private_key}

The tools and security root elements also contain some configuration. This is a great time to look at the documentation to check configurable sub-elements.

tool:
  git:
    installations:
      - name: Default
        home: "git"

security:
  remotingCLI:
    enabled: false

Looking up the security element’s documentation helps us find how to configure the remoting CLI setting. Below you’ll find respective documentation pages for security and remoting CLIs.

Docs

Remoting

You may notice that a number of configurators land under the unclassified root element. Of the various configurable elements here I set Jenkins’ URL and admin address.

unclassified:
  location:
    url: https://jenkins.company.com
    adminAddress: support@company.net

There’s many other elements to configure, but let’s assume you’re happy with the configuration. Let’s apply it!

Reloading changed configuration

To apply any changes you’ve made to your configuration file, return to the Configuration as Code page under Manage Jenkins and click Reload existing configuration. Once the page finishes refreshing your configuration has been reloaded. Repeat the reload step every time you edit your configuration file. This can be automated, but I’ll be covering that in a future post.

Reload

In closing

I hope you’ve had fun getting started with JCasC. There’s many more configurable elements available than I covered here, and the Configuration as Code plugin has a bright future ahead of it. Stay tuned for more posts regarding Jenkins Configuration as Code.

Now get out there and start migrating your configuration to code using JCasC!

Watch the hands-on session ‘How to use Jenkins Configuration as Code’ at Day of Jenkins

Published: Oct 1, 2018

Updated: Mar 26, 2024

CI/CD