k8s单机版使用本地存储local-path-provisioner

Posted 2023-02-16 gsls200808

在k8s单机环境中，我们经常使用本机路径作为持久化存储。在k8s官方文档中有两种方式使用本机存储，一种是hostPath，另一种是local volume，这两种都不支持动态扩容，并且程序移植改动比较大，而local-path-provisioner很好的弥补了这一缺陷

local-path-provisioner常见的有两个仓库

rancher版

https://github.com/rancher/local-path-provisioner

kubernetes-sigs版

https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner

注意kubernetes-sigs版不支持动态扩容/动态供给dynamically provisioning，所以建议使用rancher版。这里介绍的都是rancher版。

官方的安装和样例使用

#安装
kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/master/deploy/local-path-storage.yaml
#使用
kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/master/examples/pvc/pvc.yaml
kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/master/examples/pod/pod.yaml

默认的存储路径为/opt/local-path-provisioner，默认安装到local-path-storage命名空间

虽然provisioner是安装到local-path-storage命名空间了，那么别的命名空间能用吗？答案是可以的，因为sc是不区分命名空间的，后续pvc都是向sc申请空间的

这里仍然使用之前部署mysql作为例子部署讲解

local-path-storage.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: local-path-storage

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: local-path-provisioner-service-account
  namespace: local-path-storage

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: local-path-provisioner-role
rules:
  - apiGroups: [ "" ]
    resources: [ "nodes", "persistentvolumeclaims", "configmaps" ]
    verbs: [ "get", "list", "watch" ]
  - apiGroups: [ "" ]
    resources: [ "endpoints", "persistentvolumes", "pods" ]
    verbs: [ "*" ]
  - apiGroups: [ "" ]
    resources: [ "events" ]
    verbs: [ "create", "patch" ]
  - apiGroups: [ "storage.k8s.io" ]
    resources: [ "storageclasses" ]
    verbs: [ "get", "list", "watch" ]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: local-path-provisioner-bind
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: local-path-provisioner-role
subjects:
  - kind: ServiceAccount
    name: local-path-provisioner-service-account
    namespace: local-path-storage

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: local-path-provisioner
  namespace: local-path-storage
spec:
  replicas: 1
  selector:
    matchLabels:
      app: local-path-provisioner
  template:
    metadata:
      labels:
        app: local-path-provisioner
    spec:
      serviceAccountName: local-path-provisioner-service-account
      containers:
        - name: local-path-provisioner
          image: rancher/local-path-provisioner:master-head
          imagePullPolicy: IfNotPresent
          command:
            - local-path-provisioner
            - --debug
            - start
            - --config
            - /etc/config/config.json
          volumeMounts:
            - name: config-volume
              mountPath: /etc/config/
          env:
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
      volumes:
        - name: config-volume
          configMap:
            name: local-path-config

---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: local-path
provisioner: rancher.io/local-path
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Delete

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: local-path-config
  namespace: local-path-storage
data:
  config.json: |-
    
            "nodePathMap":[
            
                    "node":"DEFAULT_PATH_FOR_NON_LISTED_NODES",
                    "paths":["/opt/local-path-provisioner"]
            
            ]
    
  setup: |-
    #!/bin/sh
    set -eu
    mkdir -m 0777 -p "$VOL_DIR"
  teardown: |-
    #!/bin/sh
    set -eu
    rm -rf "$VOL_DIR"
  helperPod.yaml: |-
    apiVersion: v1
    kind: Pod
    metadata:
      name: helper-pod
    spec:
      containers:
      - name: helper-pod
        image: busybox
        imagePullPolicy: IfNotPresent

mysql-pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pvc
  namespace: default
spec:
  accessModes:
    - ReadWriteOnce   #在rancher/local-path-provisioner里，这里不能用ReadWriteMany
  storageClassName: local-path     #此处为你命名的StorageClass name
  resources:
    requests:
      storage: 4Gi

mysql-config.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: mysql-config
  namespace: default
data:
  my.cnf: |
        [client]
        default-character-set=utf8mb4
        [mysql]
        default-character-set=utf8mb4
        [mysqld] 
        max_connections = 2000
        secure_file_priv=/var/lib/mysql
        sql_mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION

mysql-deploy.yaml

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql-deploy
  namespace: default
spec:
  selector:
    matchLabels:
      app: mysql-deploy
  template:
    metadata:
      labels:
        app: mysql-deploy
    spec:
      containers:
      - name: mysql-deploy
        image: mysql:8.0.28
        args:
        - --default_authentication_plugin=mysql_native_password
        - --character-set-server=utf8mb4
        - --collation-server=utf8mb4_unicode_ci
        env:                        
        - name: MYSQL_ROOT_PASSWORD
          value: "12345678"
        ports:
        - containerPort: 3306
        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
        - name: mysql-config
          mountPath: /etc/mysql/conf.d/my.cnf         
      volumes:
      - name: mysql-persistent-storage
        persistentVolumeClaim:
          claimName: mysql-pvc
      - name: mysql-config
        configMap:
          name: mysql-config      
---
apiVersion: v1
kind: Service
metadata:
  name: mysql-deploy
  namespace: default
spec:
  type: NodePort
  ports:
  - port: 3306
    targetPort: 3306
    nodePort: 31111
  selector:
    app: mysql-deploy

安装

kubectl apply -f local-path-storage.yaml
kubectl apply -f mysql-pvc.yaml
kubectl apply -f mysql-config.yaml
kubectl apply -f mysql-deploy.yaml

卸载

kubectl delete -f mysql-deploy.yaml
kubectl delete -f mysql-config.yaml
kubectl delete -f mysql-pvc.yaml
kubectl delete -f local-path-storage.yaml