k8s中创建nfs外部供应商（provisioner）

Posted 2022-11-17

1、环境说明

操作系统：cenots7.9
k8s版本：1.25
容器运行时：containerd
nfs：1.3.0

2、搭建nfs服务

1、服务端

1、安装nfs服务
yum install -y nfs-utils rpcbind net-tools

2、修改配置文件
vim /etc/exports
/data/nfs  *(rw,no_root_squash,sync)

3、创建共享目录
mkdir /data/nfs -p

4、授权
chmod 755 -R /data/nfs

5、启动nfs服务
systemctl enable rpcbind --now
systemctl enable nfs --now

6、常用命令
exportfs -avr		#重新加载配置文件
showmount -e 		#查看本地nfs共享的目录

2、客户端

1、安装软件
yum install nfs-utils
ps: 不需要启动服务

2、查看nfs服务器上可用的共享目录
showmount -e 192.168.100.40		
ps：ip为nfs服务器的ip，nfs的客户端安装在k8s的所有worker节点

3、在K8s的==master==节点上创建sa并授权

1、创建sa
vim /yaml/nfs-sa.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-provisioner

2、应用服务
[root@master yaml]# kubectl apply -f /yaml/nfs-sa.yaml
[root@master yaml]# kubectl get sa nfs-provisioner 
NAME              SECRETS   AGE
nfs-provisioner   0         59m

3、给sa授权
[root@master yaml]# kubectl create clusterrolebinding nfs-provisioner-clusterrolebinding --clusterrole=cluster-admin --serviceaccount=default:nfs-provisioner
[root@master yaml]# kubectl get clusterrolebindings.rbac.authorization.k8s.io nfs-provisioner-clusterrolebinding 
NAME                                 ROLE                        AGE
nfs-provisioner-clusterrolebinding   ClusterRole/cluster-admin   60m

4、创建nfs外部提供商

1、使用deployment资源进行创建
[root@master yaml]# cat nfs-deployment-provisioner.yaml 
kind: Deployment
apiVersion: apps/v1
metadata:
  name: nfs-provisioner
spec:
  selector:
    matchLabels:
       app: nfs-provisioner
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: nfs-provisioner
    spec:
      serviceAccount: nfs-provisioner		#使用创建的sa
      containers:
      - name: nfs-provisioner
        image: registry.cn-beijing.aliyuncs.com/mydlq/nfs-subdir-external-provisioner:v4.0.0
        imagePullPolicy: IfNotPresent
        volumeMounts:
        - name: nfs-client-root
          mountPath: /persistentvolumes
        env:				#设置容器环境变量
        - name: PROVISIONER_NAME
          value: example.com/nfs	
        - name: NFS_SERVER
          value: 192.168.100.40		#配置nfs服务器的IP地址或者域名
        - name: NFS_PATH
          value: /data/k8s-pro		#配置nfs服务器的共享目录
      volumes:				#在volumes中执行nfs服务器的IP和共享目录
        - name: nfs-client-root
          nfs:
            server: 192.168.100.40	
            path: /data/k8s-pro

ps: registry.cn-beijing.aliyuncs.com/mydlq/nfs-subdir-external-provisioner:v4.0.0
如果配置了阿里云的镜像加速器，这个镜像会在创建deployment后自动拉取

2、创建deployment
[root@master yaml]# kubectl apply -f /yaml/nfs-deployment-privisioner.yaml

3、查看
[root@master yaml]# kubectl get deployments.apps nfs-provisioner 
NAME              READY   UP-TO-DATE   AVAILABLE   AGE
nfs-provisioner   1/1     1            1           38m

6、测试

1、创建StorageClass

[root@master yaml]# vim nfs-storageclass.yaml 
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs
provisioner: example.com/nfs		#指定创建的供应商

[root@master yaml]# kubectl apply -f nfs-storageclass.yaml 
storageclass.storage.k8s.io/nfs created
[root@master yaml]# kubectl get sc
NAME   PROVISIONER       RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
nfs    example.com/nfs   Delete          Immediate           false                  30m

2、创建pvc

[root@master yaml]# vim nfs-pvc.yaml 
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nfs-pvc
  namespace: default
  labels:
    app: nfs-pvc
spec:
  accessModes:			#指定访问类型
  - ReadWriteOnce
  volumeMode: Filesystem	#指定卷类型
  resources:
    requests:
      storage: 2Gi
  storageClassName: nfs		#指定创建的存储类的名字

[root@master yaml]# kubectl apply -f nfs-pvc.yaml 

查看自动创建的pv和pvc
[root@master yaml]# kubectl get pvc
NAME      STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
nfs-pvc   Bound    pvc-310123f1-4727-4591-b18f-59337fab5ec7   2Gi        RWO            nfs            6s
[root@master yaml]# kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM             STORAGECLASS   REASON   AGE
pvc-310123f1-4727-4591-b18f-59337fab5ec7   2Gi        RWO            Delete           Bound    default/nfs-pvc   nfs                     9s

查看你nfs共享目录
[root@nfs ~]# ll /data/k8s-pro/
total 0
drwxrwxrwx 2 root root 6 Nov 17 00:36 default-nfs-pvc-pvc-310123f1-4727-4591-b18f-59337fab5ec7

总结：创建pvc使使用storageclass，那么将会自动创建pv并进行绑定

参考：https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner