I'm looking for a way to run a Docker-in-Docker (dind) container alongside my Jenkins agent nodes using the Jenkins Kubernetes Plugin configured by Jenkins Configuration-as-Code (JCasC).
I'm currently running a plain agent node, that has all my tools installed. Including the Docker client. But no server/daemon running. My idea was now to add another container, that is running the Docker daemon. Both containers share the same working directory, so they operate on the same data.
Why having it running as two containers? The Docker daemon needs privileged access. And I want to restrict it as much as possible.
First problem that I encountered was, that Jenkins is not killing the pod when its finished building, but just ends the agent process. Now with another container running, the pod stays alive. I fixed this by writing a file to another shared volume when the agent pod stops, which in turn is read by the dind pod and exits when the file is present.
Here's an excerpt from my current Pod config:
apiVersion: "v1"
kind: "Pod"
spec:
serviceAccountName: jenkins-agent
containers:
- image: "jenkins-jnlp-image"
command: [ "/bin/sh", "-c" ]
args:
- |
trap 'touch /usr/share/pod/done' EXIT
/entrypoint.sh
name: "jnlp"
env:
- name: DOCKER_HOST
value: tcp://localhost:2375
securityContext:
privileged: false
tty: true
volumeMounts:
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
- mountPath: /usr/share/pod
name: tmp-pod
workingDir: "/home/jenkins/agent"
- image: "dind-image"
name: "dind"
command: [ "/bin/sh", "-c" ]
args:
- |
dockerd-entrypoint.sh &
while ! test -f /usr/share/pod/done; do
echo 'Waiting for the jnlp to finish...'
sleep 5
done
echo "Agent pod finished, exiting"
exit 0
securityContext:
privileged: true
tty: false
volumeMounts:
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
- mountPath: /usr/share/pod
name: tmp-pod
workingDir: "/home/jenkins/agent"
volumes:
- emptyDir:
medium: ""
name: "workspace-volume"
- emptyDir: {}
name: tmp-pod
Is this the deigned way of doing this? Is there any better?
Bonus question: I install from the https://github.com/jenkinsci/helm-charts/tree/main/charts/jenkins . I could not find a good way to add another container there yet, without overwriting the whole pod template. Does anyone know of a way? If not I would probably create a PR to add that as a feature.
Thanks in advance!