helm安装mongodb-replicaset遇到的坑及解决方法

Posted 2021-05-01 tcpdump

#如果你安装K8s集群时更改了默认的域名后缀cluster.local那么请看下去

https://github.com/helm/charts/tree/master/stable/mongodb-replicaset

1.手起刀落嚓嚓就是三刀

  改成国内镜像源

helm repo list

helm repo remove stable

helm repo add stable http://mirror.azure.cn/kubernetes/charts/

helm repo update

helm search mongodb-replicaset

helm fetch mongodb-replicaset

2.修改安装时的相关参数与配置

   修改values.yaml和 template下的yaml文件,比如镜像改成国内或自有私服,  安装的ns名称等

添加一个持久化的卷,这里用cephfs,mongodb-storageclass.yaml

kind: StorageClass

apiVersion: storage.k8s.io/v1

metadata:

  name:  mongodbdata

  namespace:  app-db

  annotations:

    storageclass.kubernetes.io/is-default-class: "true"

provisioner: ceph.com/cephfs

reclaimPolicy: Retain

parameters:

    monitors: 192.168.10.1:6789,192.168.10.2:6789,192.168.0.3:6789

    adminId: admin

    adminSecretName: ceph-secret-admin

    adminSecretNamespace: "app-db"

    claimRoot: /mongodbdata

执行创建持久化卷 kubectl apply -f  mongodb-storageclass.yaml

3.嚓嚓安装开始掉坑

  helm install  mongodb-replicaset  --name mongodb  --namespace app-db

等吧,等到了1小时没动静其实1分钟就可以看到了

错误2/3 INIT  怎么都走不下去,同时exec进pod查看初始化容器有执行脚本

kubectl describe pod mongodb-mongodb-replicaset-0  -n app-db

kubectl exec -it  mongodb-mongodb-replicaset-0 -- /bin/bash  -n app-db

ps命令就能看到初始化脚本在执行但一直是hug不动的也没有日志

.排查坑

  去查看GITHUB上的issue详见

https://github.com/helm/charts/pull/17883

https://github.com/helm/charts/issues/8576

同时查看官方此镜像dockerfile

https://github.com/helm/charts/blob/master/stable/mongodb-replicaset/install/Dockerfile

#RUN apk update && apk add bash openssl && wget -qO /peer-finder http://storage.googleapis.com/kubernetes-release/pets/peer-finder

发现这句坑爹的链接下的是不支持domain参数版本

因为官方charts提供的mongo-install镜像用的旧peer-find,此版本不支持domain参数,亦即是如果你改了k8s集

群的cluster.local域名就会有问题,这时我们需要重新打包以便支持识别自定义的domain版本的peer-find文件

##

通过下载最新的

https://github.com/kubernetes-retired/contrib/tree/master/peer-finder  go build生成或者

链接: https://pan.baidu.com/s/1VcaXwisjm_YZVs4oBCoVXg 提取码: sr6k

然后重新构建镜像

 docker build -t mongodb-install:0.8 . -f peer-finder_dockerfile

####peer-finder_dockerfile 内容###########

# Copyright 2016 The Kubernetes Authors All rights reserved.

#

# Licensed under the Apache License, Version 2.0 (the "License");

# you may not use this file except in compliance with the License.

# You may obtain a copy of the License at

#

#     http://www.apache.org/licenses/LICENSE-2.0

#

# Unless required by applicable law or agreed to in writing, software

# distributed under the License is distributed on an "AS IS" BASIS,

# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

# See the License for the specific language governing permissions and

# limitations under the License.

#FROM golang

#ADD https://raw.githubusercontent.com/kubernetes-retired/contrib/master/peer-finder/peer-finder.go .

#ADD peer-finder.go .

#RUN go get -d -v k8s.io/apimachinery/pkg/util/sets

#RUN go get -d -v https://github.com/kubernetes/apimachinery/pkg/util/sets

#RUN CGO_ENABLED=0 go build -a -installsuffix cgo --ldflags '-w' ./peer-finder.go

FROM alpine:3.8

LABEL maintainer="Reinhard<[e-mail]>"

RUN  apk add bash openssl

COPY  peer-finder /

#RUN apk update && apk add bash openssl && wget -qO /peer-finder http://storage.googleapis.com/kubernetes-release/pets/peer-finder

COPY install.sh /

RUN chmod -c 755 /install.sh /peer-finder

ENTRYPOINT ["/install.sh"]

#######结束#######################

打完自验证镜像可以使用docker起个容器看看

docker run --rm=true -it --entrypoint=/peer-finder mongodb-install:0.8  --help

输出应该会有--domain参数

5. 嚓嚓重新release

  把重构建后的镜像传到私有或dockerhub上,或者放到需要布署的node节点上

重新执行relaease即可

helm delete mongodb --purge 

helm install  mongodb-replicaset  --name mongodb  --namespace app-db

过会可以看到正常初化并执行成功

  按官方提示在下个版本3.10.2会修复此问题,最新是3.10.1

https://github.com/helm/charts/commit/77f515d0af2b3a6a8b2948e568b014519982c22a