K8S集群证书已过期且etcd和apiserver已不能正常使用下的恢复方案
Posted aguncn
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了K8S集群证书已过期且etcd和apiserver已不能正常使用下的恢复方案相关的知识，希望对你有一定的参考价值。
在这种比较极端的情况下，要小心翼翼的规划和操作，才不会让集群彻底死翘翘。首先，几个ca根证书是10年期，应该还没有过期。我们可以基于这几个根证书，来重新生成一套可用的各组件认证证书。
前期，先制定以下方案步骤，能否实现，待验证。
一，制作证书的基本文件。
Ca-csr.json(因为根证书是OK的，所以这个文件，可是列在这里，不会用上)
{ "CN": "kubernetes", "key": { "algo": "rsa", "size": 2048 }, "ca": { "expiry": "438000h" } }
Ca-config.json(它用来从自签名根ca.crt和ca.key生成新的证书，可以共用)
{ "signing": { "default": { "expiry": "43800h" }, "profiles": { "kubernetes": { "usages": [ "signing", "key encipherment", "server auth", "client auth" ], "expiry": "43800h" } } } }
二，重新生成etcd系列证书((注意，这是依据/etc/kubernetes/pki/etcd/目录下的ca证书)
Etcd-server.json
{ "CN": "etcdServer", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "O": "etcd", "OU": "etcd Security", "C": "CN", "L": "ShangHai", "ST": "ShangHai" } ] }
cfssl gencert -ca=ca.crt -ca-key=ca.key -config=ca-config.json -hostname=127.0.0.1,localhost,本机ip,小写主机名 -profile=kubernetes etcd-server.json|cfssljson -bare server
etcd-peer.json
{ "CN": "etcdPeer", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "O": "etcd", "OU": "etcd Security", "C": "CN", "L": "ShangHai", "ST": "ShangHai" } ] }
cfssl gencert -ca=ca.crt -ca-key=ca.key -config=ca-config.json -hostname=127.0.0.1,localhost,本机ip,小写主机名 -profile=kubernetes etcd-peer.json|cfssljson -bare peer
etcd-client.json
{ "CN": "etcdClient", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "O": "etcd", "OU": "etcd Security", "C": "CN", "L": "ShangHai", "ST": "ShangHai" } ] }
cfssl gencert -ca=ca.crt -ca-key=ca.key -config=ca-config.json -profile=kubernetes etcd-client.json |cfssljson -bare client
三，重新制作apiserver证书(注意，这是依据/etc/kubernetes/pki目录下的ca证书)
Apiserver.json
{ "CN": "kube-apiserver", "key": { "algo": "rsa", "size": 2048 } }
cfssl gencert -ca=ca.crt -ca-key=ca.key -config=ca-config.json -hostname=127.0.0.1, kubernetes , kubernetes.default, kubernetes.default.svc, kubernetes.default.svc.cluster.local,本机ip,小写主机名 -profile=kubernetes apiserver.json |cfssljson -bare apiserver
apiserver-kubelet-client.json
{ "CN": "kube-apiserver-kubelet-client", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "O": "system:masters" } ] }
cfssl gencert -ca=ca.crt -ca-key=ca.key -config=ca-config.json -profile=kubernetes apiserver-kubelet-client.json |cfssljson -bare apiserver-kubelet-client
三，重新制作front-proxy证书(注意，这是依据/etc/kubernetes/pki目录下的front-proxy-ca证书，它必须和apiserver的ca不一样，牵扯到apiserver的认证顺序，切记)
Front-proxy-client.json
{ "CN": "front-proxy-client", "key": { "algo": "rsa", "size": 2048 } }
cfssl gencert -ca=ca.crt -ca-key=ca.key -config=ca-config.json -profile=kubernetes front-proxy-client.json |cfssljson -bare front-proxy-client
四，制作scheduler，controller-manager,admin,kubelet,bootstrap证书，此证书只存在于主节点。此证书主要用来生成controller-manager.conf, scheduler.conf, admin.conf, kubelet.conf bootstrap-kubelet.conf。
如果/etc/kubernetes/pki目录下的sa.key,sa.pub存在，则无须更新，因为它没有过期概念。
kube-scheduler-csr.json
{ "CN": "system:kube-scheduler", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "O": "system:kube-scheduler", } ] }
cfssl gencert -ca=ca.crt -ca-key=ca.key -config=ca-config.json -hostname=127.0.0.1,localhost,本机ip,小写主机名 -profile=kubernetes kube-scheduler-csr.json|cfssljson -bare kube-scheduler
kube-controller-manager-csr.json
{ "CN": "system:kube-controller-manager", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "O": "system:kube-controller-manager", } ] }
cfssl gencert -ca=ca.crt -ca-key=ca.key -config=ca-config.json -hostname=127.0.0.1,localhost,本机ip,小写主机名 -profile=kubernetes kube-controller-manager-csr.json |cfssljson -bare kube-controller-manager
admin-csr.json
{ "CN": "admin", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "O": "system:masters", } ] }
cfssl gencert -ca=ca.crt -ca-key=ca.key -config=ca-config.json -profile=kubernetes admin-csr.json |cfssljson -bare kube- admin
kubelet-csr.json(这个方法，只适合master上的kubelet运行，不用bootstrap的情况)
{ "CN": "system:node: 小写主机名", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "O": "system:nodes", } ] }
cfssl gencert -ca=ca.crt -ca-key=ca.key -config=ca-config.json -hostname=127.0.0.1,localhost,本机ip,小写主机名 -profile=kubernetes kubelet-csr.json |cfssljson -bare kubelet
如果还需要bootstrap,可以参考下面的url:
https://k2r2bai.com/2018/07/17/kubernetes/deploy/manual-install/
https://www.jianshu.com/p/6650954fa973?tdsourcetag=s_pctim_aiomsg
五，以上文件作好之后，需要根据现在的k8s命令规则改名，还要根据不同的文件，存放于不同的目录。
六，这时，k8s master应该可以启动了。接下来，制作kubeconfig文件，参考url
https://www.cnblogs.com/netsa/p/8134000.html(配置bootstrap及kubelet认证)
https://www.cnblogs.com/charlieroro/p/8489515.html（配置.kube/config文件）
# 设置集群参数
kubectl config set-cluster
# 设置客户端认证参数
kubectl config set-credentials
# 设置上下文参数
kubectl config set-context
# 设置默认上下文
kubectl config use-context
kubectl config set-cluster kubernetes --certificate-authority=/etc/kubernetes/cert/ca.pem --embed-certs=true --server=https://ip:port \\ --kubeconfig=kube-controller-manager.kubeconfig kubectl config set-credentials system:kube-controller-manager --client-certificate=kube-controller-manager.pem --client-key=kube-controller-manager-key.pem --embed-certs=true --kubeconfig=kube-controller-manager.kubeconfig kubectl config set-context system:kube-controller-manager --cluster=kubernetes --user=system:kube-controller-manager --kubeconfig=kube-controller-manager.kubeconfig kubectl config use-context system:kube-controller-manager --kubeconfig=kube-controller-manager.kubeconfig
kubectl config set-cluster kubernetes --certificate-authority=/etc/kubernetes/cert/ca.pem --embed-certs=true --server=https://ip:port \\ --kubeconfig=kube-scheduler.kubeconfig kubectl config set-credentials system:kube-scheduler --client-certificate=kube-scheduler.pem --client-key=kube-scheduler-key.pem --embed-certs=true --kubeconfig=kube-scheduler.kubeconfig kubectl config set-context system:kube-scheduler --cluster=kubernetes --user=system:kube-scheduler --kubeconfig=kube-scheduler.kubeconfig kubectl config use-context system:kube-scheduler --kubeconfig=kube-scheduler.kubeconfig
kubectl config set-cluster kubernetes --certificate-authority=${PKI_DIR}/ca.pem --embed-certs=true --server=https://ip:port \\ --kubeconfig=${K8S_DIR}/admin.conf kubectl config set-credentials kubernetes-admin --client-certificate=${PKI_DIR}/admin.pem --client-key=${PKI_DIR}/admin-key.pem --embed-certs=true --kubeconfig=${K8S_DIR}/admin.conf kubectl config set-context kubernetes-[email protected] --cluster=kubernetes --user=kubernetes-admin --kubeconfig=${K8S_DIR}/admin.conf kubectl config use-context kubernetes-[email protected] --kubeconfig=${K8S_DIR}/admin.conf
kubectl config set-cluster kubernetes --certificate-authority=${PKI_DIR}/ca.pem --embed-certs=true --server=https://ip:port \\ --kubeconfig=${K8S_DIR}/kubelet.conf && kubectl config set-credentials system:node:小写主机名 --client-certificate=${PKI_DIR}/kubelet.pem --client-key=${PKI_DIR}/kubelet-key.pem --embed-certs=true --kubeconfig=${K8S_DIR}/kubelet.conf && kubectl config set-context system:node:小写主机名@kubernetes --cluster=kubernetes --user=system:node:小写主机名 --kubeconfig=${K8S_DIR}/kubelet.conf && kubectl config use-context system:node:小写主机名@kubernetes --kubeconfig=${K8S_DIR}/kubelet.conf
七，当制作好这些文件之后，按k8s安装的位置，分发文件，重启kubelet，应该就可以重新启动好集群了。
以上是关于K8S集群证书已过期且etcd和apiserver已不能正常使用下的恢复方案的主要内容，如果未能解决你的问题，请参考以下文章