[TimLinux] k8s 故障分析全集

Posted 2021-06-12 TimLinux

节点NotReady

Traints:

• node.kubernetes.io/unreachable:NoExecute
• node.kubernetes.io/unreachable:NoSchedule

解决办法：

1. 查看 kube-controller-manager， kube-scheduler， kube-apiserver, kube-proxy 等的日志，都显示：

Error from server (InternalError): Internal error occurred: Authorization error 
(user=kube-apiserver-kubelet-client, verb=get, resource=nodes, subresource=proxy)

2. 尝试登陆进去看下：

# kubectl -n kube-system exec -it kube-proxy-xxxx sh
error: unable to upgrade connection: Authorization error
(user=kube-apiserver-kubelet-client, verb=create, resource=nodes, subresource=proxy)

3. 查看其它节点都 Kube-proxy 日志

kubectl -n kube-system logs kube-proxy-xxx
...
endpoints is forbidden: User "system:serviceaccount:kube-system:kube-proxy" cannot list resource "endpoints" in 
API group "" at the cluster scope: RBAC: [clusterrole.rbac.authorization.k8s.io "system:basic-user" not found,
clusterrole.rbac.authorization.k8s.io "system:node-proxier" not found,
clusterrole.rbac.authorization.k8s.io "system:discovery" not found,
clusterrole.rbac.authorization.k8s.io "system:public-info-viewer" not found]
...