k8s网络模型从 Calico切换为Canal踩的坑

Posted 2022-01-28

问题描述

在跟着《每天五分钟玩转kubernets》这本书学习到K8S的网络章节时，实验中部署canal网络以演示Network Policy。因为最开始搭建k8s集群是部署的Calico网络（Calico也支持Network Policy，但是为了和教程保持一致，还是切换了），所以这里重新初始化了master，切换网络。

按照书上指示，操作了下面的步骤：
1、首先在k8s集群所有节点执行kubeadm reset命令销毁当前集群
2、在k8s的master上执行命令重新初始化了master：

kubeadm init --kubernetes-version=v1.14.0 --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=10.0.0.101

根据安装k8s集群时的步骤，初始化master之后，也又在master上执行了下面的配置kubectl的三条命令（这一步骤书上在这一环节没有提，只说了要重新init，所以还是怀着忐忑的心情执行的）：

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

3、然后执行canal部署命令：

kubectl apply -f https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/canal/rbac.yaml
kubectl apply -f https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/canal/canal.yaml

然后在执行第一条命令的时候发现报错了：

[[email protected] ~]# kubectl apply -f https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/canal/rbac.yaml
unable to recognize "https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/canal/rbac.yaml": Get https://10.0.0.101:6443/api?timeout=32s: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")
unable to recognize "https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/canal/rbac.yaml": Get https://10.0.0.101:6443/api?timeout=32s: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")
unable to recognize "https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/canal/rbac.yaml": Get https://10.0.0.101:6443/api?timeout=32s: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")
unable to recognize "https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/canal/rbac.yaml": Get https://10.0.0.101:6443/api?timeout=32s: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")
[[email protected] ~]# 

尝试了查看node，发现报错一样的：

[[email protected] ~]# kubectl get nodes
Unable to connect to the server: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")

解决方法

作为k8s初学小白，还看不懂这个报错，不知道啥原因。所以只有求助Google。

看到GitHub上一个类似问题的文章有一个回答：(链接：https://github.com/kubernetes/kubernetes/issues/48378 )

然后想到可能是旧的目录$HOME/.kube和新的目录有什么冲突，于是试了一下把老的目录删除了，再重新配置kubectl（原文链接还有一些其他的解决方法可以尝试，这里用得比较简单粗暴）：

[[email protected] ~]# rm -rf $HOME/.kube
#重新配置kubectl：
[[email protected] ~]#   mkdir -p $HOME/.kube
[[email protected] ~]#   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[[email protected] ~]#   sudo chown $(id -u):$(id -g) $HOME/.kube/config

接着再执行重新部署canal命令，就成功了：

[[email protected] ~]# kubectl apply -f https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/canal/rbac.yaml
clusterrole.rbac.authorization.k8s.io/calico created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/canal-flannel created
clusterrolebinding.rbac.authorization.k8s.io/canal-calico created
[[email protected] ~]# kubectl apply -f https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/canal/canal.yaml
configmap/canal-config created
daemonset.extensions/canal created
serviceaccount/canal created
customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created
[[email protected] ~]#

查看node命令也正常了：

[[email protected] ~]# kubectl get nodes
NAME        STATUS   ROLES    AGE   VERSION
k8smaster   Ready    master   27m   v1.14.0

等把node节点重新加到集群里（一主一从）之后，再查看网络，成功切换到了canal：

[[email protected] ~]# kubectl get --namespace=kube-system daemonset canal
NAME    DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR                 AGE
canal   2         2         2       2            2           beta.kubernetes.io/os=linux   46m
[[email protected] ~]# 
[[email protected] ~]# 
[[email protected] ~]# kubectl get --namespace=kube-system pod -o wide|grep canal
canal-xwbps                         3/3     Running   0          46m    10.0.0.101   k8smaster   <none>           <none>
canal-zwfqj                         3/3     Running   0          2m5s   10.0.0.102   k8snode01   <none>           <none>
[[email protected] ~]#