Secure private Docker Registry

Docker Registry

Last week I had the use case to deploy a private Docker Registry in a Cloud environment. One prerequisite was that the Images should be available for everyone (pull) but pushing Images should be secured by a password authentication. It was decided to host the Registry on Azure using the Azure Kubernetes Service (AKS) beside some other services. In this post, I will provide an overview how to deploy a Docker Registry in AKS. What is Azure Kubernetes Service (AKS)?

AKS is a managed Kubernetes environment aka Kubernetes-as-a-Service. Which means you don’t need to care about the OS, your Docker version or your Kubernetes deployment. You just select a sizing and your Kubernetes version and after some minutes you are ready to deploy your services. In this case, we decided to go with the Pay-as-you-go subscription. You can lower your monthly costs by ordering a reservation.

Deploying Docker Registry

Because the Docker Registry should be available from all over the world, first of all, we need a static IP as well as an Ingress Controller.

Requesting a static IP using the Azure CLI:

az network public-ip create --resource-group <aks_resource_group> --name <my_public_ip> --allocation-method static

Deploy Ingress Chart using the Helm CLI:

helm install stable/nginx-ingress --namespace kube-system --set controller.service.loadBalancerIP=<public_ip>

Afterward, you are will be able to access your Ingress controller using https://<public_ip>.

With this, you are ready to deploy the Docker Registry. Once again we will use a Helm chart to deploy it:

helm install -f values-registry-aks.yaml --name registry stable/docker-registry

The value-registry-aks.yaml is used to define our setting. An example:

replicaCount: 1
updateStrategy:
image:
  repository: registry
  tag: latest
  pullPolicy: IfNotPresent
service:
  name: registry
  type: ClusterIP
  port: 5000
  annotations: {}
ingress:
  enabled: true
  hosts:
    -
  annotations:
  tls:
resources: {}
cpu: 200m
memory: 256Mi
persistence:
  accessMode: 'ReadWriteOnce'
  enabled: true
  size: 20Gi
  storageClass: 'default'
storage: filesystem
secrets:
  haSharedSecret: ""
  htpasswd: ""
configData:
  version: 0.1
  log:
    fields:
      service: registry
  storage:
    cache:
      blobdescriptor: inmemory
  http:
    addr: :5000
    headers:
      X-Content-Type-Options: [nosniff]
  health:
    storagedriver:
      enabled: true
      interval: 10s
      threshold: 3

Important is the size of the persistent volume (size parameter) as well as the Storage class (storageClass parameter). We decided to use Azure managed disks which provides two different Storage classes.

After deploying the Helm chart you can access your private Docker Registry using https://<public_ip>/v2/_catalog.

Patching the Deployment

As mentioned above the use case was to allow read access (pull) for everyone but pushing Images should be protected by username and password. Therefore, we needed to customize our installed Helm chart. Of course, the best solution would be to build our own Helm chart which is planned to be done in the near future. Until then we decided to patch our deployment manually.

First of all, you need to delete the old Ingress configuration using kubectl:

kubectl delete ing registry-docker-registry

Afterward, you need to create a secret which will store the certificate. In this case, we used a wildcard certificate:

kubectl create secret tls registry-tls --key <key_file> --cert <certificate_chain_file>

The Docker Registry will be served on two different URLs. One will be used for pull request and will therefore only allow GET request. The other one will be used to push Images and will be protected by a password. To expose those URLs you need to create a new Ingress definition:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: registry-nginx
  namespace: default
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: "0"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
spec:
  rules:
  - host:registry-push.<your_domain>
    http:
      paths:
      - backend:
          serviceName: registry-nginx
          servicePort: 80
        path: /
  - host:registry.<your_domain>
    http:
      paths:
      - backend:
          serviceName: registry-nginx
          servicePort: 80
        path: /
  tls:
  - hosts:
    - registry.<your_domain>
    - registry-push.<your_domain>
    secretName:registry-tls

Our Ingress definition is configured to forward the request to a Service called registry-nginx. Which will be created using following service definition:

apiVersion: v1
kind: Service
metadata:
  labels:
    app: docker-registry
  name: registry-nginx
  namespace: default
spec:
  ports:
  - name: registry-nginx
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: docker-registry
  type: ClusterIP

This service will then talk to a pod which is running a Nginx instance. The Nginx is configured to serve two different URLs. One which allows GET requests only, the other with password authentication.

The nginx.conf, as well as the password file (you can create it using the htpasswd command), are stored in a Configmap. You create them using following definitions:

apiVersion: v1
kind: ConfigMap
metadata:
  name: registry-nginx
data:
  nginx.conf: |
    user  nginx;
    worker_processes  1;
    error_log  /var/log/nginx/error.log warn;
    pid /var/run/nginx.pid;
    events {
      worker_connections  1024;
    }
    http {
      default_type application/octet-stream;
      log_format main '$remote_addr - $remote_user [$time_local] "$request" '
        '$status $body_bytes_sent "$http_referer" '
        '"$http_user_agent" "$http_x_forwarded_for"';
      access_log /var/log/nginx/access.log  main;
      sendfile off;
      keepalive_timeout 65;
      server_tokens off;
      tcp_nopush on;
      map $upstream_http_docker_distribution_api_version $docker_distribution_api_version {
        '' 'registry/2.0';
      }
      upstream docker-registry {
        server registry-docker-registry.default.svc.cluster.local:5000;
      }
      server {
       listen 80;
       server_name registry.;
        client_max_body_size 0;
       location / {
          limit_except GET {
            deny all;
         }
         add_header 'Docker-Distribution-Api-Version' $docker_distribution_api_version always;
          proxy_pass http://docker-registry;
          proxy_read_timeout 900;
        }
      }
      server {
        listen 80;
        client_max_body_size 0;
        server_name registry-push.;
        location / {
          auth_basic "Registry realm";
          auth_basic_user_file /etc/nginx/conf.d/nginx.htpasswd;
          add_header 'Docker-Distribution-Api-Version' $docker_distribution_api_version always;
          proxy_pass http://docker-registry;
          proxy_read_timeout 900;
        }
      }
    }
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: registry-nginx-auth
data:
  nginx.htpasswd: |
    push:<your_password>

The Nginx is configured to forward your request to http://registry-docker-registry.default.svc.cluster.local:5000 which will be served by your Docker Registry.

Finally, you need to create your Nginx pod using following Deployment definition:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: registry-nginx
spec:
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  selector:
    matchLabels:
      app: docker-registry
  replicas: 1
  template:
    metadata:
      labels:
        app: docker-registry
    spec:
      containers:
      - name: registry-nginx
        image: nginx:latest
        imagePullPolicy: IfNotPresent
        volumeMounts:
            - mountPath: /etc/nginx
              name: nginx-conf
            - mountPath: /etc/nginx/conf.d
              name: nginx-conf-auth
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /
            port: 80
            scheme: HTTP
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        ports:
        - containerPort: 80
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /
            port: 80
            scheme: HTTP
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
      volumes:
      - name: nginx-conf
        configMap:
          name: registry-nginx
      - name: nginx-conf-auth
        secret:
          secretName: registry-push

You can now access your Docker Registry using following URLs:
https://registry-push.<your_domain>: You will use this URL to push Images using docker push after authenticating with docker login.
https://registry.<your_domain>: This URL only allows GET requests. Everyone can pull your Images with docker pull.

Leave a Reply

Your email address will not be published. Required fields are marked *