kubernetes:pod无法删除

Posted 2023-05-04

参考技术A 通过helm删除服务后，pods无法删除:

突如其来的报错让我有点懵，因为定义的这个pvc是跟随 helm 服务一起删除，我赶紧去看了下pvc服务:

pvc正在删除中，但是一直没有删除成功，因为关联了pod，我又查看了相关pod：

看的出来是问题出在了node节点上，继续查看node的状态：

既然kubelet没有问题，那再去看看pod中的容器：

最终解决办法：

Amazon EKS (NFS) 到 Kubernetes pod。无法挂载卷

【中文标题】Amazon EKS (NFS) 到 Kubernetes pod。无法挂载卷

【英文标题】：Amazon EKS (NFS) to Kubernetes pod. Can't mount volume

【发布时间】：2021-12-22 09:20:42

【问题描述】：

我正在使用 terraform 将 Amazon EKS (NFS) 附加到 Kubernetes pod。

一切运行都没有错误并被创建：

Pod victoriametrics 存储类 持久卷 永久卷声明

但是，卷 victoriametrics-data 没有附加到 pod。无论如何，我在吊舱的外壳中看不到一个。 请问有人能帮我理解我错在哪里吗？

我已经为这个问题剪掉了一些不重要的代码来缩短代码。

resource "kubernetes_deployment" "victoriametrics" 
...
      spec 
        container 
          image = var.image
          name  = var.name
          ...
          volume_mount 
              mount_path        = "/data"
              mount_propagation = "None"
              name              = "victoriametrics-data"
              read_only         = false
            
        

        volume 
            name = "victoriametrics-data"
        

      
    
...

resource "kubernetes_csi_driver" "efs" 
  metadata 
    name = "$local.cluster_name-$local.namespace"
    annotations = 
      name = "For store data of $local.namespace."
    
  
  spec 
    attach_required        = true
    pod_info_on_mount      = true
    volume_lifecycle_modes = ["Persistent"]
  

resource "kubernetes_storage_class" "efs" 
  metadata 
    name = "efs-sc"
  
  storage_provisioner = kubernetes_csi_driver.efs.id
  reclaim_policy      = "Retain"
  mount_options       = ["file_mode=0700", "dir_mode=0777", "mfsymlinks", "uid=1000", "gid=1000", "nobrl", "cache=none"]

resource "kubernetes_persistent_volume" "victoriametrics" 
  metadata 
    name = "$local.cluster_name-$local.namespace"
  
  spec 
    storage_class_name               = "efs-sc"
    persistent_volume_reclaim_policy = "Retain"
    volume_mode                      = "Filesystem"
    access_modes                     = ["ReadWriteMany"]
    capacity = 
      storage = var.size_of_persistent_volume_claim
    
    persistent_volume_source 
      nfs 
        path   = "/"
        server = local.eks_iput_target
      
    
  

resource "kubernetes_persistent_volume_claim" "victoriametrics" 
  metadata 
    name      = local.name_persistent_volume_claim
    namespace = local.namespace
  
  spec 
    access_modes       = ["ReadWriteMany"]
    storage_class_name = "efs-sc"
    resources 
      requests = 
        storage = var.size_of_persistent_volume_claim
      
    
    volume_name = kubernetes_persistent_volume.victoriametrics.metadata.0.name
  

kind: Deployment
apiVersion: apps/v1
metadata:
  name: victoriametrics
  namespace: victoriametrics
  labels:
    k8s-app: victoriametrics
    purpose: victoriametrics
  annotations:
    deployment.kubernetes.io/revision: '1'
    name: >-
      VictoriaMetrics - The High Performance Open Source Time Series Database &
      Monitoring Solution.
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: victoriametrics
      purpose: victoriametrics
  template:
    metadata:
      name: victoriametrics
      creationTimestamp: null
      labels:
        k8s-app: victoriametrics
        purpose: victoriametrics
      annotations:
        name: >-
          VictoriaMetrics - The High Performance Open Source Time Series
          Database & Monitoring Solution.
    spec:
      containers:
        - name: victoriametrics
          image: 714154805721.dkr.ecr.us-east-1.amazonaws.com/victoriametrics:v1.68.0
          ports:
            - containerPort: 8428
              protocol: TCP
            - containerPort: 2003
              protocol: TCP
            - containerPort: 2003
              protocol: UDP
          volumeMounts:
            - mountPath: /data
              name: victoriametrics-data
            - mountPath: /var/log
              name: varlog
          env:
            - name: Name
              value: victoriametrics
          resources:
            limits:
              cpu: '1'
              memory: 1Gi
            requests:
              cpu: 500m
              memory: 1Gi
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: IfNotPresent
      volumes:
      - name: victoriametrics-data
        emptyDir: 
      - name: varlog
        emptyDir:     
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      dnsPolicy: ClusterFirst
      automountServiceAccountToken: true
      shareProcessNamespace: false
      securityContext: 
      schedulerName: default-scheduler
      tolerations:
        - key: k8s-app
          operator: Equal
          value: victoriametrics
          effect: NoSchedule
      enableServiceLinks: true
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 25%
      maxSurge: 25%
  minReadySeconds: 15
  revisionHistoryLimit: 10
  progressDeadlineSeconds: 300

【参考方案1】：

您需要在部署中使用您创建的持久卷声明，而不是 emptyDir：

kind: Deployment
apiVersion: apps/v1
metadata:
  name: victoriametrics
...
  volumes:
  - name: victoriametrics-data
      persistentVolumeClaim:
        claimName: <value of local.name_persistent_volume_claim>

【讨论】：

谢谢你gohm'c！现在我可以附加 NFS 卷了。然而，尽管在声称的卷中设置了 21Gi，但该 pod 有 8E（EFS 的全部可能大小）。请您帮我解答这个问题好吗？ ***.com/questions/69917416/…