I followed this(https://matthewpalmer.net/kubernetes-app-developer/articles/kubernetes-volumes-example-nfs-persistent-volume.html) link and installed an NFS server in my cluster with blow yaml file:
kind: Service
apiVersion: v1
metadata:
name: nfs-service
spec:
selector:
role: nfs
ports:
# Open the ports required by the NFS server
# Port 2049 for TCP
- name: tcp-2049
port: 2049
protocol: TCP
# Port 111 for UDP
- name: udp-111
port: 111
protocol: UDP
---
kind: Pod
apiVersion: v1
metadata:
name: nfs-server-pod
labels:
role: nfs
spec:
containers:
- name: nfs-server-container
image: cpuguy83/nfs-server
securityContext:
privileged: true
args:
# Pass the paths to share to the Docker image
- /exports
I installed the server via below commnad:
kubectl create namespace nfs
kubectl apply -f nfs-server.yaml -n nfs
The server installed smoothly and is up and running.
I installed nfs-common
on all nodes then I installed the nfs-client provisioner helm chart via below command:
helm install stable/nfs-client-provisioner --set nfs.server=nfs-service --set nfs.path=/exported/path --name=nfs-client --namespace=nfs
But the nfs-client stuck at ContainerCreating and this is the output of describe:
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
nfs-client-root:
Type: NFS (an NFS mount that lasts the lifetime of a pod)
Server: nfs-service
Path: /exported/path
ReadOnly: false
nfs-client-nfs-client-provisioner-token-k9n8n:
Type: Secret (a volume populated by a Secret)
SecretName: nfs-client-nfs-client-provisioner-token-k9n8n
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedMount 59m kubelet, serflex-argus-1 MountVolume.SetUp failed for volume "nfs-client-root" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root --scope -- mount -t nfs nfs-service:/exported/path /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root
Output: Running scope as unit: run-rdc5765db89a64abf87610ba4d853f48d.scope
mount.nfs: Connection timed out
Warning FailedMount 57m kubelet, serflex-argus-1 MountVolume.SetUp failed for volume "nfs-client-root" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root --scope -- mount -t nfs nfs-service:/exported/path /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root
Output: Running scope as unit: run-r0b472b9b183240bd8a4a3020bc13a129.scope
mount.nfs: Connection timed out
Warning FailedMount 55m kubelet, serflex-argus-1 MountVolume.SetUp failed for volume "nfs-client-root" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root --scope -- mount -t nfs nfs-service:/exported/path /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root
Output: Running scope as unit: run-r8f149b9cfb4d414cb15834537f4fd88a.scope
mount.nfs: Connection timed out
Warning FailedMount 52m kubelet, serflex-argus-1 MountVolume.SetUp failed for volume "nfs-client-root" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root --scope -- mount -t nfs nfs-service:/exported/path /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root
Output: Running scope as unit: run-r3f2e2388086f42f987f0e1475ef95385.scope
mount.nfs: Connection timed out
Warning FailedMount 50m kubelet, serflex-argus-1 MountVolume.SetUp failed for volume "nfs-client-root" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root --scope -- mount -t nfs nfs-service:/exported/path /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root
Output: Running scope as unit: run-r8db7f40d8a7948598deae1fb07162372.scope
mount.nfs: Connection timed out
Warning FailedMount 48m kubelet, serflex-argus-1 MountVolume.SetUp failed for volume "nfs-client-root" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root --scope -- mount -t nfs nfs-service:/exported/path /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root
Output: Running scope as unit: run-rff3d9f71b0e44148ba70c3f9f706d8e4.scope
mount.nfs: Connection timed out
Warning FailedMount 46m kubelet, serflex-argus-1 MountVolume.SetUp failed for volume "nfs-client-root" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root --scope -- mount -t nfs nfs-service:/exported/path /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root
Output: Running scope as unit: run-r05af2a5659ad47bea7a0e7010d280444.scope
mount.nfs: Connection timed out
Warning FailedMount 20m (x5 over 56m) kubelet, serflex-argus-1 Unable to attach or mount volumes: unmounted volumes=[nfs-client-root], unattached volumes=[nfs-client-nfs-client-provisioner-token-k9n8n nfs-client-root]: timed out waiting for the condition
Warning FailedMount 9m7s (x15 over 59m) kubelet, serflex-argus-1 Unable to attach or mount volumes: unmounted volumes=[nfs-client-root], unattached volumes=[nfs-client-root nfs-client-nfs-client-provisioner-token-k9n8n]: timed out waiting for the condition
Warning FailedMount 3m20s (x15 over 43m) kubelet, serflex-argus-1 (combined from similar events): MountVolume.SetUp failed for volume "nfs-client-root" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root --scope -- mount -t nfs nfs-service:/exported/path /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root
Output: Running scope as unit: run-r27616bcceccc4ff194ac009997604d31.scope
mount.nfs: Connection timed out
I have to mention that I am running k8s v1.16.
How I can solve this problem?
Update
I don't know what was the problem but when I removed the nfs-server from k8s and used a server located on a bare-metal machine the client started to work properly.
Best Answer
Make sure you have installed
nfs-utils
(rpm-base distros) ornfs-common
(deb-based distros) packages on all Kubernetes nodes.