How to Configure Private Registry for Kubernetes cluster running with containerd

When using containerd as a runtime container, we can still use the docker registry for pod images in kubernetes because basically docker images use OCI format so they are compatible with containerd.

For creating kubernetes cluster with containerd you can learn on previous post “How To Create Kubernetes Cluster With Containerd ”.
I will use the same kubernetes cluster from previous and add 1 more server for Registry Server

Step 1. Prepare Registry Server

Do this configuration on private registry server

apt update
apt -y install docker.io docker-registry apache2-utils

Enable basic authentication on docker registry

sudo vi /etc/docker/registry/config.yml
#change htpasswd path
htpasswd:
    realm: basic-realm
    path: /etc/docker/registry/.htpasswd

Enable insecure registry on docker daemon

sudo vi /etc/docker/daemon.json
{
  "insecure-registries":
    ["172.16.4.93:5000"]
}

Restart docker and registry service

sudo systemctl restart docker
sudo systemctl restart docker-registry

Enable private registry port on firewall

sudo ufw allow 5000/tcp

Create User for Docker Registry Auth

#you can change regadmin with your user admin
sudo htpasswd -Bc /etc/docker/registry/.htpasswd regadmin

Try to pull images

sudo docker pull nginx

Tag image before push images to private registry

sudo docker image ls
sudo docker tag nginx 172.16.4.93:5000/nginx:1.0

Login to private registry and push tagged image

sudo docker login 172.16.4.93:5000
sudo docker push 172.16.4.93:5000/nginx:1.0

View the login auth with base 64 encode, this is necessary to added in containerd config.toml

sudo cat /root/.docker/config.json

Note the auth code

Step 2. Enable Private Registry on Kubernetes Nodes

Do this configuration on master nodes and worker nodes

Configure containerd config.toml

sudo vi /etc/containerd/config.toml
#find  [plugins."io.containerd.grpc.v1.cri".registry] and add the following configuration
[plugins."io.containerd.grpc.v1.cri".registry]
  config_path = ""
  [plugins."io.containerd.grpc.v1.cri".registry.auths]
  [plugins."io.containerd.grpc.v1.cri".registry.configs]
    [plugins."io.containerd.grpc.v1.cri".registry.configs."172.16.4.93:5000".tls]
      insecure_skip_verify = true
    [plugins."io.containerd.grpc.v1.cri".registry.configs."172.16.4.93:5000".auth]
      auth = "cmVnYWRtaW46QzBtcG4zdCE="
  [plugins."io.containerd.grpc.v1.cri".registry.headers]
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
    [plugins."io.containerd.grpc.v1.cri".registry.mirrors."172.16.4.93:5000"]
      endpoint = ["http://172.16.4.93:5000"]

Save and exit
next restart containerd service

sudo systemctl restart containerd
sudo containerd config dump

Step 3. Test private registry

Do this on master nodes

Test Pull Images with crictl to make sure our containerd configuration is working

sudo crictl -r /run/containerd/containerd.sock pull 172.16.4.93:5000/nginx:1.0

Create Pod with private registry images

kubectl run private-pod --image=172.16.4.93:5000/nginx:1.0 --restart=Always