阿里云K8S使用NAS作为动态存储部署RabbitMQ高可用集群

Posted 2022-04-08 rong341233

一.通过subPath的方式创建动态存储

1.alicloud-nas-subpath.yaml

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: rabbit-ifs
mountOptions:
- nolock,tcp,noresvport
- vers=3
parameters:
  volumeAs: subpath
  server: "xxxxxxxxx.cn-hangzhou.extreme.nas.aliyuncs.com:/share/rabbitmq"   #注意一定要在/share目录下，需要自己手动创建目录
provisioner: nasplugin.csi.alibabacloud.com
reclaimPolicy: Retain

2.rabbit-ifs.yaml

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: rabbit-ifs
spec:
  accessModes:
  - ReadWriteMany
  storageClassName: rabbit-ifs
  resources:
    requests:
      storage: 100Gi

二.创建RabbitMQ服务

1.创建configmap.yaml

kind: ConfigMap
apiVersion: v1
metadata:
  name: rabbitmq-cluster-config
  namespace: default
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
data:
    enabled_plugins: |
      [rabbitmq_management,rabbitmq_peer_discovery_k8s].
    rabbitmq.conf: |
      cluster_formation.peer_discovery_backend = rabbit_peer_discovery_k8s
      cluster_formation.k8s.host = kubernetes.default.svc.cluster.local
      cluster_formation.k8s.address_type = hostname
      cluster_formation.node_cleanup.interval = 30
      cluster_formation.node_cleanup.only_log_warning = true
      cluster_partition_handling = autoheal
      queue_master_locator=min-masters
      loopback_users.guest = false
      cluster_formation.randomized_startup_delay_range.min = 0
      cluster_formation.randomized_startup_delay_range.max = 2
      cluster_formation.k8s.hostname_suffix = .rabbitmq-cluster.default.svc.cluster.local
      vm_memory_high_watermark.absolute = 1GB
      disk_free_limit.absolute = 2GB

2.创建RBAC文件 rabbit-rbac.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: rabbitmq-cluster
  namespace: default
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: rabbitmq-cluster
  namespace: default
rules:
- apiGroups: [""]
  resources: ["endpoints"]
  verbs: ["get"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: rabbitmq-cluster
  namespace: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: rabbitmq-cluster
subjects:
- kind: ServiceAccount
  name: rabbitmq-cluster
  namespace: default

3.创建服务svc.yaml

kind: Service
apiVersion: v1
metadata:
  labels:
    app: rabbitmq-cluster
  name: rabbitmq-cluster
  namespace: default
spec:
  clusterIP: None
  ports:
  - name: rmqport
    port: 5672
    targetPort: 5672
  selector:
    app: rabbitmq-cluster

---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: rabbitmq-cluster
  name: rabbitmq-cluster-manage
  namespace: default
spec:
  ports:
  - name: http
    port: 15672
    protocol: TCP
    targetPort: 15672
  - name: rmqport
    port: 5672
    targetPort: 5672
  selector:
    app: rabbitmq-cluster
  # type: NodePort

4.创建有状态服务rabbit-statefulset.yaml

kind: StatefulSet
apiVersion: apps/v1
metadata:
  labels:
    app: rabbitmq-cluster
  name: rabbitmq-cluster
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: rabbitmq-cluster
  serviceName: rabbitmq-cluster
  template:
    metadata:
      labels:
        app: rabbitmq-cluster
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: "app"
                    operator: In
                    values:
                      - rabbitmq
              topologyKey: "kubernetes.io/hostname"
      containers:
      - args:
        - -c
        - cp -v /etc/rabbitmq/rabbitmq.conf $RABBITMQ_CONFIG_FILE; exec docker-entrypoint.sh
          rabbitmq-server
        command:
        - sh
        env:
        - name: RABBITMQ_DEFAULT_PASS
          value: admin      #这里设置了账号密码，但是实际是还是需要使用guest账号登录,密码也是guest
        - name: RABBITMQ_DEFAULT_USER
          value: admin
        - name: TZ
          value: Asia/Shanghai
        - name: RABBITMQ_ERLANG_COOKIE
          value: SWvCP0Hrqv43NG7GybHC95ntCJKoW8UyNFWnBEWG8TY=
        - name: K8S_SERVICE_NAME
          value: rabbitmq-cluster
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: POD_NAME
          valueFrom记一次阿里云k8s部署-测试存储

 记一次阿里云k8s部署-测试存储
 如何连接Rancher与阿里云NAS存储
 阿里云ask 对nas共享存储压测
 阿里云ask 对nas共享存储压测
 威联通NAS折腾日记(5)-使用阿里云DDNS服务