A Detailed Guide to Docker Secrets

By Community / Developer
Jul 19, 2023

This post was written by Talha Khalid, a full-stack developer and data scientist who loves to make the cold and hard topics exciting and easy to understand.

No one has any doubt that microservices architecture has already proven to be efficient. However, implementing security, particularly in an immutable infrastructure context, has been quite the challenge.

With issues ranging from how to separate a password from the template in an image to how to change an access password to a service without interrupting it, there’s no shortage of workarounds. But there’s a better way: Docker secrets. In this article, we explain what secrets are in Docker, how to create and update docker secrets, and other good things to know about them.

Why do we even need Docker secrets?

When working on a project, you may need to pass sensitive information to the environment, such as passwords, private keys, tokens, API keys, and the like. However, storing sensitive information directly in version control systems (such as Git) can be risky. When code repositories are shared or made public, these secrets can be exposed to unintended individuals, leading to potential security breaches.

We often use environment variables to store this information, but this is not a best practice. Environment variables break the Principle of Least Surprise. This is a design principle that suggests systems and software should behave to minimize confusion and surprises for users. The goal is to make the system’s behavior intuitive and predictable, reducing the likelihood of unexpected or unintended consequences.

Environment variables break the principle because they can be accessed and observed by various entities within the system. This includes linked containers, child processes, system tools like Docker inspect, and logging mechanisms. The reason is that if application exceptions occur, many frameworks dump the context, including the value of environment variables into the log file. This is where Docker secrets help.

What are Docker secrets?

Docker secrets function as a vault that allows you to store sensitive information securely. Only individuals with the vault key, which Docker assigns only to the service nodes that require it, can access the information. A Raft algorithm stores the secret in an encrypted format across all manager nodes and ensures distribution of the secret to containers associated with the relevant service.

Secrets are Docker’s solution for working with all the secrets in a swarm mode cluster, which is a multinode or multicontainer cluster. Secrets only work in swarm mode, which encrypts all communication between nodes by default. The secret can contain anything, but it has a maximum size of 500 KB. For now, this little beauty isn’t available outside of the context of a Docker swarm — in fact, it’s not clear if this will ever change. For now, you need to use a service to deploy individual containers.

Docker swarm mode

As an important player in the DevOps environment, Docker extends beyond its basic function of managing containers. In the case of an application divided into multiple microservices, a higher level of orchestration is required compared to simple scripts. To address this need, Docker introduced a service abstraction that facilitates the orchestration of containers across multiple hosts through its swarm mode. Docker offers two versions of swarm. The older version functions as a standalone solution that necessitates a slightly more complex setup involving its key-value store. In contrast, the newer variant, swarm mode, has been integrated into the Docker Engine since version 1.12, eliminating the need for a specialized setup.

The traditional usage of Docker images involved packaging and exchanging artifacts or applications. Initially, Docker images based on complete Ubuntu distributions were commonly employed. However, this approach has been surpassed by utilizing minimal binaries in customized operating systems such as Alpine Linux. The concept of a container has evolved from being a substitute for virtual machines to becoming a process capsule.

However, rather than rendering containers obsolete, Docker services introduced additional configuration options, such as specifying the desired number of replicas, deployment constraints, and update policies.

Tasks are the smallest unit that runs within a service. Docker swarm refers to the container as a task when scheduling a container to run as part of a service. The task represents an instance of a container managed by the Docker swarm. The container itself may not be aware of the Docker swarm and its service abstraction. Since containers function as process capsules, the task serves as a link between the swarm and the container it represents.

Enabling Docker swarm

By default, the Docker Engine starts with swarm mode disabled. To enable it, type docker swarm init in the console.

docker swarm init

Docker initializes a new swarm on the current node and designates it as a manager node by acknowledging this command. After that, docker generates a swarm join token, which is a unique identifier that allows other nodes to join the swarm as either managers or workers. A corresponding message indicates if you previously switched the Docker Engine to swarm mode.

Creating a Docker secret

You can create a secret in two ways:

using STDIN:

$ echo "mysecretpassword" | docker secret create my_password -

Or by reading a file:

$ docker secret create new-secret $HOME/my_password.txt

Listing, inspecting, and removing a secret

To list the existing secrets in a container:

$ docker secrets ls

To inspect a secret:

$ docker secret inspect my_password

Inspect, contrary to its name, does not show the contents of your secret. Instead, it shows a lot of information about the secret, including its creation and modification dates. Although there’s no way to actually modify a secret using the Docker CLI, there’s an endpoint in the Docker Swarm API to update a secret:

update secret -- "/secrets/{id}/update

To remove a secret, type:

docker secret rm my_password

Using a Docker secret

Services consume secrets, as mentioned above, and this happens through explicit association, using the –secret flag when creating a service. You can create a service using a secret with a simple command:

$ docker service create --name db --secret my_password mongodb:6.0

This example creates and uses a secret as a password for a MongoDB database.

Or you can add a secret to some existing service:

$ docker service update --secret-add my_password app

When you attach a secret to a service, you can access any containers running on that service, in the /run/secrets path. Inside the container on that path, you’ll find a plain text file with the secret’s content, with the same name defined in the secret name. In our case, it would be in the path /run/secrets/my_password.

Accessing and updating a secret

With the service created, the secret will be available to all containers of that service, in files inside the /run/secrets directory, mounted in tmpfs (temporary file system stored in memory). For example, if you named your secret my_password, as in the example, you can find its contents in /run/secrets/my_password.

You can include some parameters when adding a secret to a service, such as a target, which changes the name of the file in the destination; you can even include security items, such as uid, gid, and mode:

docker service create --detach=false --name app --secret source=my_password,target=password,uid=2000,gid=3000,mode=0400 mongodb:6.0

Updating a service’s secret

To change a secret for a service, you need to create a new secret and add it to the service. Modifying existing secrets directly is not supported since secrets are designed to be immutable. However, to update the contents of a secret by replacing it using the Docker Swarm API, you can:

• Create a new secret using Docker CLI

• Obtain the ID or name of the secret you wanted to update. You can retrieve the list of secrets using the Docker Swarm API endpoint: /secrets.

• Use the Docker Swarm API endpoint for updating secrets, which is /secrets/{secret-id}/update or /secrets/{secret-name}/update.

• Make an HTTP request to the appropriate API endpoint using a tool like CURL or any other HTTP client library. Specify the updated secret file as the payload or provide the updated secret in the request body.

• Finally, send the request to the Docker Swarm API endpoint, and it will successfully replace the secret with the newly created, updated one.

Final words

Docker secrets ensure the immutability of sensitive data and prevent their storage on disk or transmission in plain text over the network. When considering the use of Docker Swarm in a production environment, it is advisable to employ Docker secrets, even for local development.

By integrating Docker secrets into your workflow, you can enhance the overall security of your containerized applications. This practice ensures proper handling of sensitive information, effectively protecting against potential vulnerabilities and unauthorized access.