NFS部署
参考资料
https://jimmysong.io/kubernetes-handbook/practice/using-nfs-for-persistent-storage.html
https://github.com/kubernetes-retired/external-storage/tree/master/nfs-client/deploy
安装NFS-Server
- 安装nfs
yum install -y nfs-common nfs-utils
- 创建数据目录
# mkdir -p /data/nfs
- 访问授权
# vim /etc/exports /data/nfs 192.168.0.0/24(no_root_squash,rw,sync,no_subtree_check)
如果允许所有访问,用/data/nfs *(no_root_squash,rw,sync,no_subtree_check)
- 加载配置
# exportfs -rv
- 启动NFS和rpcbind
# systemctl enable nfs && systemctl enable rpcbind && systemctl start rpcbind nfs
图1 启动NFS和rpcbind1图2 启动NFS和rpcbind2 - 查看 RPC 服务的注册状况
# rpcinfo -p 192.168.0.69
图3 查看 RPC 服务的注册状况 - showmount测试
# showmount -e 192.168.0.69
图4 showmount测试
K8s配置NFS-Client
上传“k8s镜像插件”客户端到K8s的所有Node节点服务器的/soft/nfs,并安装。
上传“yaml文件”到k8s的控制节点服务器。
![](https://support.huaweicloud.com/deepexidip-dma/zh-cn_image_0000001908187733.png)
- 安装nfs-client
yum install -y nfs-common nfs-utils rpcbind
- Node导入镜像
# docker load -i busybox-latest.tar.gz # docker load -i nfs-client-provisioner-v3.2.0.tar.gz
图6 图示2 - Master创建命名空间
# kubectl create namespace nfs
- Master创建NFS服务K8s插件
# kubectl apply -f rbac.yaml apiVersion: v1 kind: ServiceAccount metadata: name: nfs-client-provisioner # replace with namespace where provisioner is deployed namespace: nfs --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: name: nfs-client-provisioner-runner rules: - apiGroups: [""] resources: ["persistentvolumes"] verbs: ["get", "list", "watch", "create", "delete"] - apiGroups: [""] resources: ["persistentvolumeclaims"] verbs: ["get", "list", "watch", "update"] - apiGroups: ["storage.k8s.io"] resources: ["storageclasses"] verbs: ["get", "list", "watch"] - apiGroups: [""] resources: ["events"] verbs: ["create", "update", "patch"] --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: run-nfs-client-provisioner subjects: - kind: ServiceAccount name: nfs-client-provisioner # replace with namespace where provisioner is deployed namespace: nfs roleRef: kind: ClusterRole name: nfs-client-provisioner-runner apiGroup: rbac.authorization.k8s.io --- kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: name: leader-locking-nfs-client-provisioner # replace with namespace where provisioner is deployed namespace: nfs rules: - apiGroups: [""] resources: ["endpoints"] verbs: ["get", "list", "watch", "create", "update", "patch"] --- kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: leader-locking-nfs-client-provisioner # replace with namespace where provisioner is deployed namespace: nfs subjects: - kind: ServiceAccount name: nfs-client-provisioner # replace with namespace where provisioner is deployed namespace: nfs roleRef: kind: Role name: leader-locking-nfs-client-provisioner apiGroup: rbac.authorization.k8s.io
# kubectl apply -f class.yaml apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: managed-nfs-storage provisioner: fuseim.pri/ifs # or choose another name, must match deployment's env PROVISIONER_NAME' parameters: archiveOnDelete: "false"
# kubectl apply -f deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: nfs-client-provisioner labels: app: nfs-client-provisioner # replace with namespace where provisioner is deployed namespace: nfs spec: replicas: 1 strategy: type: Recreate selector: matchLabels: app: nfs-client-provisioner template: metadata: labels: app: nfs-client-provisioner spec: serviceAccountName: nfs-client-provisioner containers: - name: nfs-client-provisioner image: kopkop/nfs-client-provisioner-arm64:v3.1.0-k8s1.11 volumeMounts: - name: nfs-client-root mountPath: /persistentvolumes env: - name: PROVISIONER_NAME value: fuseim.pri/ifs - name: NFS_SERVER value: 192.168.0.69 #NFS服务器IP地址 - name: NFS_PATH value: /data/nfs #NFS服务器上文件存储路径 volumes: - name: nfs-client-root nfs: server: 192.168.0.69 #NFS服务器IP地址 path: /data/nfs #NFS服务器上文件存储路径
看到以下信息,表明deployment创建成功
图7 图示3 - 测试
# kubectl apply -f test-claim.yaml kind: PersistentVolumeClaim apiVersion: v1 metadata: name: test-claim annotations: volume.beta.kubernetes.io/storage-class: "managed-nfs-storage" spec: accessModes: - ReadWriteMany resources: requests: storage: 1000Mi #资源大小限制
# kubectl apply -f test-pod.yaml kind: Pod apiVersion: v1 metadata: name: test-pod spec: containers: - name: test-pod image: busybox:latest command: - "/bin/sh" args: - "-c" - "touch /data/nfs/SUCCESS && exit 0 || exit 1" volumeMounts: - name: nfs-pvc mountPath: "/data/nfs" #挂载到镜像中的路径 restartPolicy: "Never" volumes: - name: nfs-pvc persistentVolumeClaim: claimName: test-claim #存储目录虚拟映射,跟test-claim.yaml的名称一致
图8 图示4到NSF-Server服务器(192.168.0.69)所在机器的/data/nfs,可以看到创建了一个随机默认目录和SUCCESS文件
图9 图示5 - 容器挂载
创建k8s客户端服务,上传daas-claim.yaml到服务器,并执行
图10 图示6# kubectl create ns daas # kubectl apply -f daas-claim.yaml kind: PersistentVolumeClaim apiVersion: v1 metadata: name: daas-claim namespace: daas annotations: volume.beta.kubernetes.io/storage-class: "managed-nfs-storage" spec: accessModes: - ReadWriteMany resources: requests: storage: 1000Mi #资源大小限制
挂载容器指定目录(/etc/daas/kerberos)到NFS服务器
图11 图示7volumeMounts: - name: nfs-pvc mountPath: "/etc/daas/kerberos" #容器目录 volumes: - name: nfs-pvc persistentVolumeClaim: claimName: daas-claim #NFS客户端服务名称
进入NFS服务器,看到test.txt,表明容器到NFS的挂载已成功
# cd /data/nfs/daas-daas-claim-pvc-xxx
图12 图示8