How to set up a Docker Registry with a Swarm S3 storage target
Containers are a great way to build small services and run multiple sandboxed services in a single host. One of the most popular tools to run containers is a service called Docker.
Docker distributes containers using a Registry, a server-side storage application for Docker images. It works much like yum or apt package managers, except that it is for full images.
When you issue a docker pull image
command, it will contact the primary public registry and try to locate and download that image. However, if you need tight control over your images, your organization can run its own registry to pull from.
The registry comes from Docker and is a small container. For its storage back end, it defaults to the file system, but you can configure any of several storage back ends, such as S3.
The S3 storage back end supports S3-compatible services. This allows the registry service to be stateless, with no volumes to be backed up or preserved.
To set up a registry
Prerequisites
- A running Docker environment with an internet connection.
- A running Swarm cluster with S3 exposed.
Note that you can choose between two methods for setting up the container:
- Docker run with an add-in config file
- Docker run with environment variables
Pull the registry image
First, you need to pull the docker image for the current version of the registry (which, at the time of writing, is version 2.7):
docker pull registry:2.7 2.7: Pulling from library/registry Digest: sha256:8004747f1e8cd820a148fb7499d71a76d45ff66bac6a29129bfdbfdc0154d146 Status: Image is up to date for registry:2.7
This pulls down the docker container from the public registry to your local docker host.
The host will then be able run the container via a docker run
command:
docker run -d -p 5000:5000 --restart always --name registry:2.7
This command would run a docker registry with local storage bound to port 5000 on the host.
Option 1: Configure with YAML file
You can set up S3 using a YAML config file. The Docker configuration documentation starts with this example:
https://raw.githubusercontent.com/docker/distribution/master/cmd/registry/config-example.yml
version: 0.1 log: fields: service: registry storage: cache: blobdescriptor: inmemory filesystem: rootdirectory: /var/lib/registry http: addr: :5000 headers: X-Content-Type-Options: [nosniff] auth: htpasswd: realm: basic-realm path: /etc/registry health: storagedriver: enabled: true interval: 10s threshold: 3
The elements to change are in the storage
section.
Replace storage: | with S3: |
---|---|
storage: cache: blobdescriptor: inmemory filesystem: rootdirectory: /var/lib/registry |
s3: accesskey: awsaccesskey secretkey: awssecretkey region: us-west-1 regionendpoint: http://myobjects.local bucket: bucketname encrypt: true keyid: mykeyid secure: true v4auth: true chunksize: 5242880 multipartcopychunksize: 33554432 multipartcopymaxconcurrency: 100 multipartcopythresholdsize: 33554432 rootdirectory: /s3/object/name/prefix |
Example of a full config.yml:
version: 0.1 log: fields: service: registry storage: cache: blobdescriptor: inmemory s3: accesskey: 480ee4b88c3380b70d957a3fb2d69054 secretkey: test region: us-west-1 regionendpoint: http://dockertest.caringo.com bucket: registry1 secure: false v4auth: true chunksize: 5242880 multipartcopychunksize: 33554432 multipartcopymaxconcurrency: 100 multipartcopythresholdsize: 33554432 http: addr: :5000 headers: X-Content-Type-Options: [nosniff]
Save the config file and do a run
to apply it:
docker run -d -p 5000:5000 --restart=always --name registryS3t2 -v `pwd`/config.yml:/etc/docker/registry/config.yml registry:2.7
This causes docker to start a container named registryS3t2, pull the config file from /config.yml
, and overwrite the one in the container.
Option 2: Configure with environment variables
You could do the same configuration using environment variables. Run docker and supply all of these values:
docker run -d -p 5000:5000 --name registry --restart=always \ -e REGISTRY_STORAGE=s3 \ -e REGISTRY_STORAGE_S3_REGION=us-east-1 \ -e REGISTRY_STORAGE_S3_BUCKET=your.bucket.example.com \ -e REGISTRY_STORAGE_S3_ROOTDIRECTORY=docker-registry-exp1 \ -e REGISTRY_STORAGE_S3_V4AUTH=false \ -e REGISTRY_STORAGE_S3_ACCESSKEY=480ee4b88c3380b70d957a3fb2d69054 \ -e REGISTRY_STORAGE_S3_SECRETKEY=secret registry:2.7
Push containers to registry
Once your registry is up and running, you can download containers and push them to your local registry.
docker pull ubuntu Using default tag: latest latest: Pulling from library/ubuntu Digest: sha256:b88f8848e9a1a4e4558ba7cfc4acc5879e1d0e7ac06401409062ad2627e6fb58 Status: Downloaded newer image for ubuntu:latest
Next, tag
the pulled image with info for the local registry:
docker image tag ubuntu localhost:5000/myfirstimage
Then push
the image to the S3-enabled Swarm cluster:
docker push localhost:5000/myfirstimage e80c789bc6ac: Mounted from my-ubuntu 6c3332381368: Mounted from my-ubuntu ef1a1ec5bba9: Mounted from my-ubuntu a1aa3da2a80a: Mounted from my-ubuntu latest: digest: sha256:1bbdea4846231d91cce6c7ff3907d26fca444fd6b7e3c282b90c7fe4251f9f86 size: 1152
The push refers to the repository [localhost:5000/myfirstimage].
Use docker ps
to list your containers:
docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d3b5f0c552e4 registry:2.7 "/entrypoint.sh /etc…" 7 days ago Up 7 days 0.0.0.0:5000->5000/tcp registryS3t2
With that done, your local developers will be able to push containers from their local machines to this registry using the forward-facing IP and port.
Important
The config file example shows how to interact with the storage and is sufficient only for a small lab environment. To put the registry into production, more configuration is required.
Follow the Docker registry guidance:
Option 3: Configure with YAML file
Here is a docker-compose.yml for our internal registry, of course with a Swarm S3 backend.
# /var/permanent/s3-registry/docker-compose.yml # # $ cd /var/permanent/s3-registry # $ docker compose up -d # $ docker compose ps # $ docker compose logs -f # # Assumes directory /var/permanent/certs exists on the docker # server containing a valid cert for the docker server hostname. version: '3.7' services: s3registry: restart: always image: registry:2 deploy: resources: limits: memory: 2g ports: - "3333:5000" secrets: - docker-repo.tx.caringo.com.crt - docker-repo.tx.caringo.com.key - s3_accesskey - s3_secretkey entrypoint: ["sh", "-xc", "REGISTRY_STORAGE_S3_ACCESSKEY=`cat /run/secrets/s3_accesskey` REGISTRY_STORAGE_S3_SECRETKEY=`cat /run/secrets/s3_secretkey` registry serve /etc/docker/registry/config.yml"] environment: - REGISTRY_LOG_LEVEL=debug - REGISTRY_LOG_FIELDS_SERVICE=registry - REGISTRY_LOG_FIELDS_ENVIRONMENT=development - REGISTRY_STORAGE_DELETE_ENABLED=true - REGISTRY_HTTP_TLS_CERTIFICATE=/run/secrets/docker-repo.tx.caringo.com.crt - REGISTRY_HTTP_TLS_KEY=/run/secrets/docker-repo.tx.caringo.com.key - REGISTRY_STORAGE=s3 - REGISTRY_STORAGE_S3_SECURE=true - REGISTRY_STORAGE_S3_REGION=generic - REGISTRY_STORAGE_S3_REGIONENDPOINT=https://registry-blobs.cloud.caringo.com - REGISTRY_STORAGE_S3_ENCRYPT=false - REGISTRY_STORAGE_S3_SKIPVERIFY=false - REGISTRY_STORAGE_S3_BUCKET=docker-repo - REGISTRY_STORAGE_S3_ROOTDIRECTORY= - REGISTRY_STORAGE_S3_ACCESSKEY - REGISTRY_STORAGE_S3_SECRETKEY - REGISTRY_STORAGE_S3_CHUNKSIZE=104857600 - REGISTRY_HTTP_HOST=https://docker-repo.tx.caringo.com:3333 # Normally docker is redirected to the S3 endpoint for blob downloads. # Uncomment this if that endpoint is not exposed to docker. # - REGISTRY_STORAGE_REDIRECT_DISABLE=true secrets: s3_accesskey: file: /home/build/s3-access-key.txt s3_secretkey: file: /home/build/s3-secret-key.txt docker-repo.tx.caringo.com.crt: file: /var/permanent/certs/docker-repo.tx.caringo.com.crt docker-repo.tx.caringo.com.key: file: /var/permanent/certs/docker-repo.tx.caringo.com.key
Related articles
© DataCore Software Corporation. · https://www.datacore.com · All rights reserved.