Containers are a great way to build small services and run multiple sandboxed services in a single host. One of the most popular tools to run containers is a service called Docker.
Docker distributes containers using a Registry, a server-side storage application for Docker images. It works much like yum or apt package managers, except that it is for full images.
When you issue a docker pull image command, it will contact the primary public registry and try to locate and download that image. However, if you need tight control over your images, your organization can run its own registry to pull from.
The registry comes from Docker and is a small container. For its storage back end, it defaults to the file system, but you can configure any of several storage back ends, such as S3.
The S3 storage back end supports S3-compatible services. This allows the registry service to be stateless, with no volumes to be backed up or preserved.
To set up a registry
Prerequisites
- A running Docker environment with an internet connection.
- A running Swarm cluster with S3 exposed.
Note that you can choose between two methods for setting up the container:
- Docker run with an add-in config file
- Docker run with environment variables
Pull the registry image
First, you need to pull the docker image for the current version of the registry (which, at the time of writing, is version 2.7):
docker pull registry:2.7 2.7: Pulling from library/registry Digest: sha256:8004747f1e8cd820a148fb7499d71a76d45ff66bac6a29129bfdbfdc0154d146 Status: Image is up to date for registry:2.7
This pulls down the docker container from the public registry to your local docker host.
The host will then be able run the container via a docker run command:
docker run -d -p 5000:5000 --restart always --name registry:2.7
This command would run a docker registry with local storage bound to port 5000 on the host.
Option 1: Configure with YAML file
You can set up S3 using a YAML config file. The Docker configuration documentation starts with this example:
https://raw.githubusercontent.com/docker/distribution/master/cmd/registry/config-example.yml
version: 0.1
log:
fields:
service: registry
storage:
cache:
blobdescriptor: inmemory
filesystem:
rootdirectory: /var/lib/registry
http:
addr: :5000
headers:
X-Content-Type-Options: [nosniff]
auth:
htpasswd:
realm: basic-realm
path: /etc/registry
health:
storagedriver:
enabled: true
interval: 10s
threshold: 3
The elements to change are in the storage section.
| Replace storage: | with S3: |
|---|---|
storage:
cache:
blobdescriptor: inmemory
filesystem:
rootdirectory: /var/lib/registry
|
s3:
accesskey: awsaccesskey
secretkey: awssecretkey
region: us-west-1
regionendpoint: http://myobjects.local
bucket: bucketname
encrypt: true
keyid: mykeyid
secure: true
v4auth: true
chunksize: 5242880
multipartcopychunksize: 33554432
multipartcopymaxconcurrency: 100
multipartcopythresholdsize: 33554432
rootdirectory: /s3/object/name/prefix
|
Example of a full config.yml:
version: 0.1
log:
fields:
service: registry
storage:
cache:
blobdescriptor: inmemory
s3:
accesskey: 480ee4b88c3380b70d957a3fb2d69054
secretkey: test
region: us-west-1
regionendpoint: http://dockertest.caringo.com
bucket: registry1
secure: false
v4auth: true
chunksize: 5242880
multipartcopychunksize: 33554432
multipartcopymaxconcurrency: 100
multipartcopythresholdsize: 33554432
http:
addr: :5000
headers:
X-Content-Type-Options: [nosniff]
Save the config file and do a run to apply it:
docker run -d -p 5000:5000 --restart=always --name registryS3t2 -v `pwd`/config.yml:/etc/docker/registry/config.yml registry:2.7
This causes docker to start a container named registryS3t2, pull the config file from /config.yml, and overwrite the one in the container.
Option 2: Configure with environment variables
You could do the same configuration using environment variables. Run docker and supply all of these values:
docker run -d -p 5000:5000 --name registry --restart=always \ -e REGISTRY_STORAGE=s3 \ -e REGISTRY_STORAGE_S3_REGION=us-east-1 \ -e REGISTRY_STORAGE_S3_BUCKET=your.bucket.example.com \ -e REGISTRY_STORAGE_S3_ROOTDIRECTORY=docker-registry-exp1 \ -e REGISTRY_STORAGE_S3_V4AUTH=false \ -e REGISTRY_STORAGE_S3_ACCESSKEY=480ee4b88c3380b70d957a3fb2d69054 \ -e REGISTRY_STORAGE_S3_SECRETKEY=secret registry:2.7
Push containers to registry
Once your registry is up and running, you can download containers and push them to your local registry.
docker pull ubuntu Using default tag: latest latest: Pulling from library/ubuntu Digest: sha256:b88f8848e9a1a4e4558ba7cfc4acc5879e1d0e7ac06401409062ad2627e6fb58 Status: Downloaded newer image for ubuntu:latest
Next, tag the pulled image with info for the local registry:
docker image tag ubuntu localhost:5000/myfirstimage
Then push the image to the S3-enabled Swarm cluster:
docker push localhost:5000/myfirstimage e80c789bc6ac: Mounted from my-ubuntu 6c3332381368: Mounted from my-ubuntu ef1a1ec5bba9: Mounted from my-ubuntu a1aa3da2a80a: Mounted from my-ubuntu latest: digest: sha256:1bbdea4846231d91cce6c7ff3907d26fca444fd6b7e3c282b90c7fe4251f9f86 size: 1152
The push refers to the repository [localhost:5000/myfirstimage].
Use docker ps to list your containers:
docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d3b5f0c552e4 registry:2.7 "/entrypoint.sh /etc…" 7 days ago Up 7 days 0.0.0.0:5000->5000/tcp registryS3t2
With that done, your local developers will be able to push containers from their local machines to this registry using the forward-facing IP and port.
Important
The config file example shows how to interact with the storage and is sufficient only for a small lab environment. To put the registry into production, more configuration is required.
Follow the Docker registry guidance:
Option 3: Configure with YAML file
Here is a docker-compose.yml for our internal registry, of course with a Swarm S3 backend.
# /var/permanent/s3-registry/docker-compose.yml
#
# $ cd /var/permanent/s3-registry
# $ docker-compose --compatibility up -d
# $ docker-compose --compatibility ps
# $ docker-compose --compatibility logs -f
#
# Assumes directory /var/permanent/certs exists on the docker
# server containing a valid cert for the docker server hostname.
version: '3.7'
services:
s3registry:
restart: always
image: registry:2
deploy:
resources:
limits:
memory: 2g
ports:
- "3333:5000"
secrets:
- docker-repo.tx.caringo.com.crt
- docker-repo.tx.caringo.com.key
- s3_accesskey
- s3_secretkey
entrypoint: ["sh", "-xc", "REGISTRY_STORAGE_S3_ACCESSKEY=`cat /run/secrets/s3_accesskey` REGISTRY_STORAGE_S3_SECRETKEY=`cat /run/secrets/s3_secretkey` registry serve /etc/docker/registry/config.yml"]
environment:
- REGISTRY_LOG_LEVEL=debug
- REGISTRY_LOG_FIELDS_SERVICE=registry
- REGISTRY_LOG_FIELDS_ENVIRONMENT=development
- REGISTRY_STORAGE_DELETE_ENABLED=true
- REGISTRY_HTTP_TLS_CERTIFICATE=/run/secrets/docker-repo.tx.caringo.com.crt
- REGISTRY_HTTP_TLS_KEY=/run/secrets/docker-repo.tx.caringo.com.key
- REGISTRY_STORAGE=s3
- REGISTRY_STORAGE_S3_SECURE=true
- REGISTRY_STORAGE_S3_REGION=generic
- REGISTRY_STORAGE_S3_REGIONENDPOINT=https://registry-blobs.cloud.caringo.com
- REGISTRY_STORAGE_S3_ENCRYPT=false
- REGISTRY_STORAGE_S3_SKIPVERIFY=false
- REGISTRY_STORAGE_S3_BUCKET=docker-repo
- REGISTRY_STORAGE_S3_ROOTDIRECTORY=
- REGISTRY_STORAGE_S3_ACCESSKEY
- REGISTRY_STORAGE_S3_SECRETKEY
- REGISTRY_STORAGE_S3_CHUNKSIZE=104857600
- REGISTRY_HTTP_HOST=https://docker-repo.tx.caringo.com:3333
# Normally docker is redirected to the S3 endpoint for blob downloads.
# Uncomment this if that endpoint is not exposed to docker.
# - REGISTRY_STORAGE_REDIRECT_DISABLE=true
secrets:
s3_accesskey:
file: /home/build/s3-access-key.txt
s3_secretkey:
file: /home/build/s3-secret-key.txt
docker-repo.tx.caringo.com.crt:
file: /var/permanent/certs/docker-repo.tx.caringo.com.crt
docker-repo.tx.caringo.com.key:
file: /var/permanent/certs/docker-repo.tx.caringo.com.key
Related articles