How to Make ‘ctr image pull’ Use Custom ‘–hosts-dir’ by Default

configurationcontainerdcontainersdocker-registrykubernetes

I've got a custom file in /etc/containerd/certs.d/registry.at.my.company/hosts.toml that has some configuration set up for a local registry container running on my servers. This is working fine when I configure it for a CRI for Kubernetes, but me and my colleagues find it frustrating that we have to add --hosts-dir /etc/containerd/certs.d to all our ctr image pull commands when running ctr manually (outside of K8s stuff.) When we were doing this with docker, it was pretty easy: we just set the right option on the docker daemon's JSON config file ("insecure-registry": "127.0.0.1"), and it worked fine. But I'm having trouble finding an equivalent setting for ctr to behave similarly. Of course, the CRI plugin is using the hosts.toml file I set up just fine, but I would like to be able to manually pull images without having to add an extra flag to the command. Is there a way I can configure containerd so that ctr image pull doesn't need --hosts-dir passed every time?

Here's what that hosts.toml file looks like:

server = "http://registry.at.my.company"

[host."http://registry.at.my.company"]
  skip_verify = true
  plain-http = true

And in the /etc/containerd/config.toml file, I've got CRI pointing to it like so:

[plugins."io.containerd.grpc.v1.cri".registry]
config_path = "/etc/containerd/certs.d"

But, as I've surmised and said above, this seems to only affect how the kubelet interacts with containerd, not how ctr does.


Some background context:

We have this local registry running with an /etc/hosts entry like so: 127.0.1.1 registry.at.my.company. Along with some firewall rules, this makes it so that everyone using our Kubernetes clusters has to push their images to registry.at.my.company and configure their pods to use images from there, because our kubelets won't be able to pull from anywhere but that domain; and to make it so that our registry isn't a SPoF, we run these local registry containers as read-only mirrors (and the registry container's server itself only accepts local connections, keeping the lack of SSL certs from being a problem.) I am not sure if this is relevant to my problem, but it should help explain why I was talking about the old insecure-registry option in docker and why my hosts.toml uses http instead of https, so that it doesn't cause anyone concern.

Also, this is containerd version 1.6.19 that I'm working with.

Best Answer

Apparently, ctr cannot be configured to behave the way I want. According to this discussion on the containerd repo, you are expected to either create a custom wrapper script or shell alias that will automatically inject --hosts-dir, or you should use an alternate containerd client which supports configuration in the same way that the docker CLI behaved: one such example is nerdctl.

Related Topic