k8s使用StorageClass动态创建PV

Posted 2023-02-19 gsls200808

上一篇文章是静态创建PV，这一篇介绍使用StorageClass动态创建PV

与上一篇文章相比，去掉了mysql-pv.yaml的配置文件，新增三个配置文件rbac.yaml class.yaml nfs-client-provisioner-deployment.yaml

从功能上讲，rbac用于绑定角色权限，class用于定义StorageClass名，nfs-client-provisioner-deployment用于指定nfs路径、rbac的sa名和StorageClass名。

与静态pv不同的是，在nfs生成路径时会创建形如default-mysql-pvc-pvc-0d446403-f342-4918-878d-b8e0f8c6a954的目录，数据会放到这个目录下。

部署参考脚本

1. https://github.com/kubernetes-retired/external-storage/tree/master/nfs-client/deploy

2. https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/tree/master/deploy

部署前配置，这里涉及到nfs挂载，所有节点机器需要安装nfs-utils

安装命令

#安装nfs-utils
yum install -y nfs-utils

这里仍然以部署mysql数据库为例，以下是完整配置

rabc.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

class.yaml

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner: fuseim.pri/ifs # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
  archiveOnDelete: "false"

nfs-client-provisioner-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  labels:
    app: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          #image: quay.io/external_storage/nfs-client-provisioner:latest 不要用这个镜像，会出现pvc一直pending的问题
          #image: k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2
          image: uhub.service.ucloud.cn/iatc/nfs-subdir-external-provisioner:v4.0.2
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: fuseim.pri/ifs
            - name: NFS_SERVER
              value: 192.168.3.86
            - name: NFS_PATH
              value: /root/nfs_root
      volumes:
        - name: nfs-client-root
          nfs:
            server: 192.168.3.86
            path: /root/nfs_root

mysql-pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pvc
  namespace: default
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: managed-nfs-storage   #此处为你命名的StorageClass name
  resources:
    requests:
      storage: 4Gi

mysql-config.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: mysql-config
  namespace: default
data:
  my.cnf: |
        [client]
        default-character-set=utf8mb4
        [mysql]
        default-character-set=utf8mb4
        [mysqld] 
        max_connections = 2000
        secure_file_priv=/var/lib/mysql
        sql_mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION

mysql-deploy.yaml

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql-deploy
  namespace: default
spec:
  selector:
    matchLabels:
      app: mysql-deploy
  template:
    metadata:
      labels:
        app: mysql-deploy
    spec:
      containers:
      - name: mysql-deploy
        image: mysql:8.0.28
        args:
        - --default_authentication_plugin=mysql_native_password
        - --character-set-server=utf8mb4
        - --collation-server=utf8mb4_unicode_ci
        env:                        
        - name: MYSQL_ROOT_PASSWORD
          value: "12345678"
        ports:
        - containerPort: 3306
        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
        - name: mysql-config
          mountPath: /etc/mysql/conf.d/my.cnf         
      volumes:
      - name: mysql-persistent-storage
        persistentVolumeClaim:
          claimName: mysql-pvc
      - name: mysql-config
        configMap:
          name: mysql-config      
---
apiVersion: v1
kind: Service
metadata:
  name: mysql-deploy
  namespace: default
spec:
  type: NodePort
  ports:
  - port: 3306
    targetPort: 3306
    nodePort: 31111
  selector:
    app: mysql-deploy

执行以下命令部署

kubectl apply -f rbac.yaml
kubectl apply -f class.yaml
kubectl apply -f nfs-client-provisioner-deployment.yaml
kubectl apply -f mysql-pvc.yaml
kubectl apply -f mysql-config.yaml
kubectl apply -f mysql-deploy.yaml

执行以下命令卸载

kubectl delete -f mysql-deploy.yaml
kubectl delete -f mysql-config.yaml
kubectl delete -f mysql-pvc.yaml
kubectl delete -f nfs-client-provisioner-deployment.yaml
kubectl delete -f class.yaml
kubectl delete -f rbac.yaml

如果需要设置默认sc（StorageClass），执行如下命令

#managed-nfs-storage为sc名 可通过kubectl get sc命令查看所有sc
# kubectl patch sc managed-nfs-storage -p '"metadata": "annotations": "storageclass.beta.kubernetes.io/is-default-class": "true"'