Kubernetes Failing with Self Signed Docker Registry Certificate
-
We have a private Docker registry and we're attempting to get a our Kubernetes cluster to pull images from it, but it's failing with the following error:
Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 104s default-scheduler Successfully assigned namespace/service-8599c54df8-592wm to fat2 Normal Pulling 67s kubelet Pulling image "192.168.100.178:5000/service:ds-1.2" Warning Failed 67s kubelet Error: ErrImagePull Warning Failed 67s kubelet Failed to pull image "192.168.100.178:5000/service:ds-1.2": rpc error: code = Unknown desc = failed to pull and unpack image "192.168.100.178:5000/service:ds-1.2": failed to resolve reference "192.168.100.178:5000/service:ds-1.2": failed to do request: Head "https://192.168.100.178:5000/v2/service/manifests/ds-1.2": x509: certificate signed by unknown authority (possibly because of "x509: invalid signature: parent certificate cannot sign this kind of certificate" while trying to verify candidate authority certificate "192.168.100.174: hostname") Warning Failed 65s (x2 over 66s) kubelet Error: ImagePullBackOff Normal BackOff 65s (x2 over 66s) kubelet Back-off pulling image "192.168.100.178:5000/service:ds-1.2"
I have copied the certificates from the Docker registry to
/etc/docker/certs.d/192.168.100.178:5000/
and I have copied the secret verbatim from another cluster that has absolutely no problem pulling images from our Docker registry. Regardless, here it is (as a Helm template):apiVersion: v1 kind: Secret metadata: name: {{ .Values.global.image_pull_secret }} namespace: {{ .Values.global.namespace }} labels: app.kubernetes.io/managed-by: Helm app.kubernetes.io/release-name: {{ .Release.Name }} app.kubernetes.io/release-namespace: {{ .Release.Namespace }} data: .dockerconfigjson: {snipped} type: kubernetes.io/dockerconfigjson
I have verified I can successfully connect with a
docker login
:$ docker login https://192.168.100.178:5000 Username: myuser Password: WARNING! Your password will be stored unencrypted in /home/myuser/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
I have completely run out of ideas here, but it seems the new cluster doesn't trust the CA that signed the certificate from the Docker registry. I'm at my wits end, so any help here is greatly appreciated.
-
Finally got it working. I had to install the root certificate by copying it to
/etc/ssl/tls/certs
. This is the root certificate that was used when creating the certificate for the Docker registry. In the error message, this is the certificate for the CA, which is referenced at the end:"192.168.100.174: hostname"
.