Manage your secrets with HashiCorp Vault

Security is important like never before. Keeping your secrets safe should be a top priority. This blog shows you how to get started in production.

In a software delivery pipeline, there are several environments involved and thus many types of secrets. We create our pipelines as code, but of course we cannot store our secrets in version control. Here at Praqma, we had a customer who raised the question of securing credentials for a container cluster used for building and testing.

Enter HashiCorp Vault, an encrypted key store with mechanisms to control access to the stored values. Values can be everything from passwords, certificates, URLs to other sensitive data. Vault supports a variety of backends to authenticate users, like GitHub, AWS, LDAP and Radius.

It is really easy to try out Vault, using what they call dev-mode. Things get a bit more tricky when you want to put it into production. We will show you how to setup Vault in production mode and control access to the content.

Let’s get started

We love containers at Praqma, so it would be nothing but strange to not run Vault in a container. Our setup is based on the official Docker image for Vault maintained by HashiCorp, and a free web UI called djenriquez/vault-ui. Both can be found on the Docker Hub.

A simple Dockerfile for Vault would look like this:

FROM vault:0.7.0 MAINTAINER Praqma

And for the UI:

FROM djenriquez/vault-ui:2.1.0 MAINTAINER Praqma

Place these in separate folders called vault and vault-ui, and we are ready to build and push to the Docker Hub:

docker build -t praqma/vault:0.7.0 vault/. docker build -t praqma/vault-ui:2.0.2 vault-ui/. docker push praqma/vault:0.7.0 docker push praqma/vault-ui:2.0.2

When done, we have two images in the Docker Hub, ready to use in our organization.

Prepare the network and configuration

The Vault UI needs to connect to the container running Vault, so let’s create a network:

docker network create vault-network

You can verify the network is created by running:

docker network ls NETWORK ID      NAME             DRIVER    SCOPE 7b332ca38df5    bridge           bridge    local 60f40b5e80f7    vault-network    bridge    local

Next we create a configuration file. We need to specify a location for the backend storage, in this case a file, which address range Vault should listen to and whether or not TLS should be disabled.

Create a file called vault.conf with the following content:

backend "file" {   path = "/vault/file/vaultsecrets" } listener "tcp" {    address = "0.0.0.0:8200"    tls_disable = 1  }

We choose to put the storage file at /vault/file inside the container, as this is one of the volumes defined by the base container image. This way we can mount it on the host if we want to.

We tell Vault to listen on all network interfaces on port 8200, and to turn off TLS as we don’t have any certificates. When you do, add two lines like this to the listener section:

tls_cert_file = "/root/server.crt" tls_key_file = "/root/server.key"

Remember to copy your certificates into the container at the location you specify.

Start the Vault container

Now we are ready to start our Vault container:

docker run -d    --hostname vault    --network vault-network    --privileged    --name vault    --cap-add=IPC_LOCK    -v vault.conf:/vault/config/vault.conf    -v /opt/persistant_storage/vault/:/vault/file/ \  praqma/vault:0.7.0 server -config /vault/config/vault.conf

Vault should be running, and you can confirm this by running:

docker logs vault

You should get output like this:

==> Vault server configuration:                       Cgo: disabled                Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", tls: "disabled")                 Log Level: info                     Mlock: supported: true, enabled: true                  Storage: file                   Version: Vault v0.7.0               Version Sha:614deacfca3f3b7162bbf30a36d6fc7362cd47f0 ==> Vault server started! Log data will stream in below:

Hooray, Vault is running. Now let’s initialize it, so we can start using it.

Initialization

Start a terminal session inside the Vault container:

docker exec -it vault /bin/sh

Set an environment variable to tell the Vault client where to connect, and then initialize Vault:

export VAULT_ADDR=http://127.0.0.1:8200 vault init

From this you will get five keys and a root token. When Vault starts up, it is sealed, preventing access to any data. The keys generated are used to unseal Vault and you need at least three keys to do so.

Unseal Key 1: z+LgaNbDbejE+hxqq9Q87vjBBfQH6VII1Ey/vmiH+Q4B Unseal Key 2: Zmy5jsn/+LQCMKzaM97imgLOQ2XIBHrmms6Oyxdz5eoC Unseal Key 3: xFEtXOUM6w4JMG+4tf4VaXu/bjoMbQx68/YKzMxO5G4D Unseal Key 4: MKsgyCt935cPcKzvNRPE+0MKOveD0GYvCh7ktsTJvp0E Unseal Key 5: kpa0GgeOzC0EcG+NszMzCDp7F6hHuRCzYyZgsR/0vxkFInitial Root Token: 9cfb29e7-0fde-cc24-df72-76ee59f5c712 Vault initialized with 5 keys and a key threshold of 3. Please securely distribute the above keys. When the Vault is re-sealed, restarted, or stopped, you must provide at least 3 of these keys to unseal it again. Vault does not store the master key. Without at least 3 keys, your Vault will remain permanently sealed.

You unseal Vault by using three of the keys like this:

/ # vault unseal z+LgaNbDbejE+hxqq9Q87vjBBfQH6VII1Ey/vmiH+Q4BSealed: true Key Shares: 5 Key Threshold: 3 Unseal Progress: 1 Unseal Nonce: 88d38704-415f-b945-e00e-747a51aa8f14 / # vault unseal Zmy5jsn/+LQCMKzaM97imgLOQ2XIBHrmms6Oyxdz5eoCSealed: true Key Shares: 5 Key Threshold: 3 Unseal Progress: 2 Unseal Nonce: 88d38704-415f-b945-e00e-747a51aa8f14 / # vault unseal xFEtXOUM6w4JMG+4tf4VaXu/bjoMbQx68/YKzMxO5G4D Sealed: false Key Shares: 5 Key Threshold: 3 Unseal Progress: 0 Unseal Nonce:

Now that Vault is unsealed, we can authenticate by using the root token:

/ # vault auth 9cfb29e7-0fde-cc24-df72-76ee59f5c712 Successfully authenticated! You are now logged in. token: 9cfb29e7-0fde-cc24-df72-76ee59f5c712 token_duration: 0 token_policies: [root]

And we are in.

We need to create a policy for our team to use. This will give them the permissions they need when fetching data from Vault.

In the container, create a policy file in HashiCorp’s HCL format:

vi /vault/praqma-policy.hcl

With the following content:

path "*"{     capabilities = [ "create", "read", "update", "list" ] }

This will give people access to everything related to these operations, but for example not deleting secrets. This is for demonstration purposes and you should adjust according to your security standards.

Then create the policy:

/ # vault policy-write praqma /vault/praqma-policy.hcl Policy 'praqma' written.

Now we will enable a GitHub backend so our employees to use their GitHub account to login to the Vault UI.

/ # vault auth-enable github Successfully enabled 'github' at 'github'!

Set the organization that a GitHub user must be member of:

/ # vault write auth/github/config organization=Praqma Success! Data written to: auth/github/config

/ # vault write auth/github/map/teams/default value=praqma Success! Data written to: auth/github/map/teams/default

In this example we used the team default. You can have multiple teams with different policies, like ops and dev, to get more fine grained access level control.

Start the Vault UI container

Exit the Vault container, so we can start the UI container.

We need to specify two environment variables: VAULT_URL_DEFAULT points to the Vault server and VAULT_AUTH_DEFAULT specifies the default authentication method, in this case GitHub:

docker run -d \  -p 8000:8000 \  --network vault-network \  --hostname vault-ui \  -e VAULT_URL_DEFAULT="http://vault:8200" \  -e VAULT_AUTH_DEFAULT="GITHUB" \  --name vault-ui \  praqma/vault-ui:2.0.2

Running docker ps should show you two running containers: vault and vault-ui:

CONTAINER ID        IMAGE             COMMAND                  CREATED STATUS              PORTS         NAMES 5941ad1aaeb8   praqma/vault-ui:2.0.2  "/entrypoint.sh np..."  5 seconds ago Up 4 seconds  0.0.0.0:8000->8000/tcp  vault-ui d0d126ac150e   praqma/vault:0.7.0   "docker-entrypoint..."   19 minutes ago    Up 19 minutes   8200/tcp vault

We see that Vault listens on port 8200 and Vault-UI on port 8000. Go to http://localhost:8000 and you should get the Vault-UI login screen.

Now go to github.com, click settings under your profile picture and then Personal access tokens in the left menu. Click on the button “Generate new token”, give the token a name and choose the permissions you want to give this token (You need at least read permissions).

Insert the token in the Vault-UI to login. You should now see the UI.

Click on the “secret” menu item in the left menu and click the “new secret” button. Give the secret a name and a secret value (by clicking on the square next to “(empty object)”) and then click Save. The new item should appear. Try and click the trash can in the right. You will be told “Error, Access denied”, as defined in our policy from earlier.

Summary

We got started with Vault, beyond dev-mode. We saw how to run and configure the Vault server and a UI where we can get an overview.