工作节点在 Kubernetes 中始终显示 NotReady 状态

Posted

技术标签:

【中文标题】工作节点在 Kubernetes 中始终显示 NotReady 状态【英文标题】:Worker nodes display always NotReady status in Kubernetes 【发布时间】:2018-04-18 17:50:54 【问题描述】:

当我加入Kubernetes集群的worker节点时,worker节点的STATUS显示为NotReady,节点的ROLE显示为(使用命令kubectl get nodes)。我在主节点和工作节点上使用的 Kubernetes 版本是 v1.8.2。我使用的 Docker 版本是17.03.1-ce。工作节点上的 Kublet 登录如下:

Nov 07 10:57:03 xyz.com kubelet[26351]: E1107 10:57:03.587712   26351 kuberuntime_image.go:126] Remove image "sha256:7b9d24c898ae4ec4882eb8f60f75b9ae3fb753b5e078ecd7b310df8bfbfb9f11" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:03 xyz.com kubelet[26351]: I1107 10:57:03.587729   26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:637ee73c524b815fcb40b6d4056af8ad92cf4adc1e9ecedbb4f98964fae8c9b1" to free 88610277 bytes
Nov 07 10:57:03 xyz.com kubelet[26351]: E1107 10:57:03.590608   26351 remote_image.go:130] RemoveImage "sha256:637ee73c524b815fcb40b6d4056af8ad92cf4adc1e9ecedbb4f98964fae8c9b1" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:03 xyz.com kubelet[26351]: E1107 10:57:03.590641   26351 kuberuntime_image.go:126] Remove image "sha256:637ee73c524b815fcb40b6d4056af8ad92cf4adc1e9ecedbb4f98964fae8c9b1" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:03 xyz.com kubelet[26351]: I1107 10:57:03.590659   26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:b949fa39c923232fc9dc5521d908cc44047d9a951339af5633b057f6a4878dce" to free 274688064 bytes
Nov 07 10:57:03 xyz.com kubelet[26351]: E1107 10:57:03.592550   26351 remote_image.go:130] RemoveImage "sha256:b949fa39c923232fc9dc5521d908cc44047d9a951339af5633b057f6a4878dce" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:03 xyz.com kubelet[26351]: E1107 10:57:03.592577   26351 kuberuntime_image.go:126] Remove image "sha256:b949fa39c923232fc9dc5521d908cc44047d9a951339af5633b057f6a4878dce" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:03 xyz.com kubelet[26351]: I1107 10:57:03.592592   26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:f5bc15a74665edd153f657b9b84fda59b02e3840733abf4d23413e9dc011e202" to free 1076462680 bytes
Nov 07 10:57:03 xyz.com kubelet[26351]: E1107 10:57:03.596231   26351 remote_image.go:130] RemoveImage "sha256:f5bc15a74665edd153f657b9b84fda59b02e3840733abf4d23413e9dc011e202" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:03 xyz.com kubelet[26351]: E1107 10:57:03.596249   26351 kuberuntime_image.go:126] Remove image "sha256:f5bc15a74665edd153f657b9b84fda59b02e3840733abf4d23413e9dc011e202" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:03 xyz.com kubelet[26351]: I1107 10:57:03.596259   26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:05a3bd381fc2470695a35f230afefd7bf978b566253199c4ae5cc96fafa29b37" to free 1840 bytes
Nov 07 10:57:03 xyz.com kubelet[26351]: E1107 10:57:03.598096   26351 remote_image.go:130] RemoveImage "sha256:05a3bd381fc2470695a35f230afefd7bf978b566253199c4ae5cc96fafa29b37" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:03 xyz.com kubelet[26351]: E1107 10:57:03.598128   26351 kuberuntime_image.go:126] Remove image "sha256:05a3bd381fc2470695a35f230afefd7bf978b566253199c4ae5cc96fafa29b37" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:03 xyz.com kubelet[26351]: W1107 10:57:03.598172   26351 eviction_manager.go:435] eviction manager: unexpected error when attempting to reduce nodefs pressure: wanted to free 9223372036854775807 bytes, but freed 0 bytes space with errors in image deletion: [rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist]
Nov 07 10:57:03 xyz.com kubelet[26351]: I1107 10:57:03.598190   26351 eviction_manager.go:346] eviction manager: must evict pod(s) to reclaim nodefs
Nov 07 10:57:03 xyz.com kubelet[26351]: E1107 10:57:03.598205   26351 eviction_manager.go:357] eviction manager: eviction thresholds have been met, but no pods are active to evict
Nov 07 10:57:04 xyz.com kubelet[26351]: W1107 10:57:04.104666   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-lns64_kube-system(827a29d5-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:04 xyz.com kubelet[26351]: W1107 10:57:04.702441   26351 eviction_manager.go:142] Failed to admit pod kube-proxy-ss9r4_kube-system(82d57b1a-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:05 xyz.com kubelet[26351]: W1107 10:57:05.023950   26351 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d
Nov 07 10:57:05 xyz.com kubelet[26351]: E1107 10:57:05.024138   26351 kubelet.go:2095] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Nov 07 10:57:05 xyz.com kubelet[26351]: W1107 10:57:05.303013   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-xsbq9_kube-system(83310557-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:05 xyz.com kubelet[26351]: W1107 10:57:05.906477   26351 eviction_manager.go:142] Failed to admit pod kube-proxy-kwnwq_kube-system(838d0597-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:06 xyz.com kubelet[26351]: W1107 10:57:06.505627   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-2qmxh_kube-system(83e86d90-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:07 xyz.com kubelet[26351]: W1107 10:57:07.103627   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-hgthx_kube-system(8443da6c-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:07 xyz.com kubelet[26351]: W1107 10:57:07.702696   26351 eviction_manager.go:142] Failed to admit pod kube-proxy-mgtpg_kube-system(849f3e0b-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:08 xyz.com kubelet[26351]: W1107 10:57:08.503689   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-vfbkw_kube-system(85197993-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:09 xyz.com kubelet[26351]: W1107 10:57:09.102866   26351 eviction_manager.go:142] Failed to admit pod kube-proxy-wtlt4_kube-system(8574f6cc-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:09 xyz.com kubelet[26351]: W1107 10:57:09.704456   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-9zxvg_kube-system(85d0951b-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:10 xyz.com kubelet[26351]: W1107 10:57:10.025736   26351 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d
Nov 07 10:57:10 xyz.com kubelet[26351]: E1107 10:57:10.025893   26351 kubelet.go:2095] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Nov 07 10:57:10 xyz.com kubelet[26351]: W1107 10:57:10.303805   26351 eviction_manager.go:142] Failed to admit pod kube-proxy-t7gw2_kube-system(862c1c67-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:10 xyz.com kubelet[26351]: W1107 10:57:10.905402   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-hhvbp_kube-system(8687f17a-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:11 xyz.com kubelet[26351]: W1107 10:57:11.501720   26351 eviction_manager.go:142] Failed to admit pod kube-proxy-p5l4m_kube-system(86e2de87-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:12 xyz.com kubelet[26351]: W1107 10:57:12.104620   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-x529x_kube-system(873ee171-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:12 xyz.com kubelet[26351]: W1107 10:57:12.702859   26351 eviction_manager.go:142] Failed to admit pod kube-proxy-b7mc7_kube-system(879a2f7e-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:13 xyz.com kubelet[26351]: W1107 10:57:13.304742   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-td4xn_kube-system(87f609af-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:13 xyz.com kubelet[26351]: W1107 10:57:13.612712   26351 conversion.go:110] Could not get instant cpu stats: different number of cpus
Nov 07 10:57:13 xyz.com kubelet[26351]: W1107 10:57:13.619906   26351 conversion.go:110] Could not get instant cpu stats: different number of cpus
Nov 07 10:57:13 xyz.com kubelet[26351]: W1107 10:57:13.620145   26351 eviction_manager.go:332] eviction manager: attempting to reclaim nodefs
Nov 07 10:57:13 xyz.com kubelet[26351]: I1107 10:57:13.620176   26351 helpers.go:1118] eviction manager: attempting to delete unused containers
Nov 07 10:57:13 xyz.com kubelet[26351]: I1107 10:57:13.622760   26351 helpers.go:1128] eviction manager: attempting to delete unused images
Nov 07 10:57:13 xyz.com kubelet[26351]: I1107 10:57:13.630989   26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:05a3bd381fc2470695a35f230afefd7bf978b566253199c4ae5cc96fafa29b37" to free 1840 bytes
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.634404   26351 remote_image.go:130] RemoveImage "sha256:05a3bd381fc2470695a35f230afefd7bf978b566253199c4ae5cc96fafa29b37" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.634449   26351 kuberuntime_image.go:126] Remove image "sha256:05a3bd381fc2470695a35f230afefd7bf978b566253199c4ae5cc96fafa29b37" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: I1107 10:57:13.634469   26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:637ee73c524b815fcb40b6d4056af8ad92cf4adc1e9ecedbb4f98964fae8c9b1" to free 88610277 bytes
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.638400   26351 remote_image.go:130] RemoveImage "sha256:637ee73c524b815fcb40b6d4056af8ad92cf4adc1e9ecedbb4f98964fae8c9b1" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.638435   26351 kuberuntime_image.go:126] Remove image "sha256:637ee73c524b815fcb40b6d4056af8ad92cf4adc1e9ecedbb4f98964fae8c9b1" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: I1107 10:57:13.638452   26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:b949fa39c923232fc9dc5521d908cc44047d9a951339af5633b057f6a4878dce" to free 274688064 bytes
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.641568   26351 remote_image.go:130] RemoveImage "sha256:b949fa39c923232fc9dc5521d908cc44047d9a951339af5633b057f6a4878dce" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.641601   26351 kuberuntime_image.go:126] Remove image "sha256:b949fa39c923232fc9dc5521d908cc44047d9a951339af5633b057f6a4878dce" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: I1107 10:57:13.641621   26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:f5bc15a74665edd153f657b9b84fda59b02e3840733abf4d23413e9dc011e202" to free 1076462680 bytes
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.645970   26351 remote_image.go:130] RemoveImage "sha256:f5bc15a74665edd153f657b9b84fda59b02e3840733abf4d23413e9dc011e202" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.646004   26351 kuberuntime_image.go:126] Remove image "sha256:f5bc15a74665edd153f657b9b84fda59b02e3840733abf4d23413e9dc011e202" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: I1107 10:57:13.646021   26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:7b9d24c898ae4ec4882eb8f60f75b9ae3fb753b5e078ecd7b310df8bfbfb9f11" to free 965091970 bytes
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.650426   26351 remote_image.go:130] RemoveImage "sha256:7b9d24c898ae4ec4882eb8f60f75b9ae3fb753b5e078ecd7b310df8bfbfb9f11" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.650454   26351 kuberuntime_image.go:126] Remove image "sha256:7b9d24c898ae4ec4882eb8f60f75b9ae3fb753b5e078ecd7b310df8bfbfb9f11" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: I1107 10:57:13.650484   26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:30ee439fbdcf8fdd4bb9b517983bcc386651ccfe20c93e3aaeb3f7f040057b79" to free 79601765 bytes
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.654416   26351 remote_image.go:130] RemoveImage "sha256:30ee439fbdcf8fdd4bb9b517983bcc386651ccfe20c93e3aaeb3f7f040057b79" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.654456   26351 kuberuntime_image.go:126] Remove image "sha256:30ee439fbdcf8fdd4bb9b517983bcc386651ccfe20c93e3aaeb3f7f040057b79" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: I1107 10:57:13.654475   26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:5e89f9aa5754594e83665a15d8ff0faee9c5991f48c143067a53a0e8dcbcef8b" to free 946975195 bytes
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.662062   26351 remote_image.go:130] RemoveImage "sha256:5e89f9aa5754594e83665a15d8ff0faee9c5991f48c143067a53a0e8dcbcef8b" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.662095   26351 kuberuntime_image.go:126] Remove image "sha256:5e89f9aa5754594e83665a15d8ff0faee9c5991f48c143067a53a0e8dcbcef8b" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: W1107 10:57:13.662136   26351 eviction_manager.go:435] eviction manager: unexpected error when attempting to reduce nodefs pressure: wanted to free 9223372036854775807 bytes, but freed 0 bytes space with errors in image deletion: [rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist]
Nov 07 10:57:13 xyz.com kubelet[26351]: I1107 10:57:13.662153   26351 eviction_manager.go:346] eviction manager: must evict pod(s) to reclaim nodefs
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.662167   26351 eviction_manager.go:357] eviction manager: eviction thresholds have been met, but no pods are active to evict
Nov 07 10:57:13 xyz.com kubelet[26351]: W1107 10:57:13.901494   26351 eviction_manager.go:142] Failed to admit pod kube-proxy-msnln_kube-system(88513541-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:14 xyz.com kubelet[26351]: W1107 10:57:14.504867   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-4h2zk_kube-system(88ad16a8-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:15 xyz.com kubelet[26351]: W1107 10:57:15.027416   26351 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d
Nov 07 10:57:15 xyz.com kubelet[26351]: E1107 10:57:15.027629   26351 kubelet.go:2095] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Nov 07 10:57:15 xyz.com kubelet[26351]: W1107 10:57:15.101833   26351 eviction_manager.go:142] Failed to admit pod kube-proxy-wfpjk_kube-system(89084bb6-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:15 xyz.com kubelet[26351]: W1107 10:57:15.708613   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-4mcj7_kube-system(8964d8e8-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:16 xyz.com kubelet[26351]: W1107 10:57:16.304520   26351 eviction_manager.go:142] Failed to admit pod kube-proxy-l2jdt_kube-system(89bfc0a1-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:16 xyz.com kubelet[26351]: W1107 10:57:16.904426   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-6qmgc_kube-system(8a1b38a6-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:17 xyz.com kubelet[26351]: W1107 10:57:17.503529   26351 eviction_manager.go:142] Failed to admit pod kube-proxy-87xmx_kube-system(8a76c081-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:18 xyz.com kubelet[26351]: W1107 10:57:18.303394   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-vmms4_kube-system(8af0ec82-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:18 xyz.com kubelet[26351]: W1107 10:57:18.902607   26351 eviction_manager.go:142] Failed to admit pod kube-proxy-jndnj_kube-system(8b4c3859-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:19 xyz.com kubelet[26351]: W1107 10:57:19.503823   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-znlsg_kube-system(8ba7eed6-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:20 xyz.com kubelet[26351]: W1107 10:57:20.029739   26351 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d
Nov 07 10:57:20 xyz.com kubelet[26351]: E1107 10:57:20.029848   26351 kubelet.go:2095] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Nov 07 10:57:20 xyz.com kubelet[26351]: W1107 10:57:20.103103   26351 eviction_manager.go:142] Failed to admit pod kube-proxy-wb68t_kube-system(8c034e6f-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:20 xyz.com kubelet[26351]: W1107 10:57:20.704166   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-rnlth_kube-system(8c5f0327-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:21 xyz.com kubelet[26351]: W1107 10:57:21.302613   26351 eviction_manager.go:142] Failed to admit pod kube-proxy-pttwm_kube-system(8cba6d18-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:21 xyz.com kubelet[26351]: W1107 10:57:21.904844   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-2zpc6_kube-system(8d1621a3-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:22 xyz.com kubelet[26351]: W1107 10:57:22.503932   26351 eviction_manager.go:142] Failed to admit pod kube-proxy-f2s2b_kube-system(8d718d21-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:23 xyz.com kubelet[26351]: W1107 10:57:23.103857   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-kwttj_kube-system(8dcd5e80-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:23 xyz.com kubelet[26351]: W1107 10:57:23.673069   26351 conversion.go:110] Could not get instant cpu stats: different number of cpus
Nov 07 10:57:23 xyz.com kubelet[26351]: W1107 10:57:23.679490   26351 conversion.go:110] Could not get instant cpu stats: different number of cpus
Nov 07 10:57:23 xyz.com kubelet[26351]: W1107 10:57:23.704826   26351 eviction_manager.go:142] Failed to admit pod kube-proxy-rlx2q_kube-system(8e28ef71-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:24 xyz.com kubelet[26351]: E1107 10:57:24.113346   26351 helpers.go:468] PercpuUsage had 0 cpus, but the actual number is 56; ignoring extra CPUs
Nov 07 10:57:24 xyz.com kubelet[26351]: W1107 10:57:24.312653   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-ffjzb_kube-system(8e8575e3-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:24 xyz.com kubelet[26351]: W1107 10:57:24.903196   26351 eviction_manager.go:142] Failed to admit pod kube-proxy-g57cx_kube-system(8edfc25e-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:25 xyz.com kubelet[26351]: W1107 10:57:25.031318   26351 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d
Nov 07 10:57:25 xyz.com kubelet[26351]: E1107 10:57:25.031532   26351 kubelet.go:2095] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Nov 07 10:57:25 xyz.com kubelet[26351]: W1107 10:57:25.503456   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-rdmwj_kube-system(8f3b764f-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:26 xyz.com kubelet[26351]: W1107 10:57:26.106221   26351 eviction_manager.go:142] Failed to admit pod kube-proxy-qq9k6_kube-system(8f976fdd-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:26 xyz.com kubelet[26351]: W1107 10:57:26.705257   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-n98m7_kube-system(8ff2e041-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:27 xyz.com kubelet[26351]: W1107 10:57:27.302494   26351 eviction_manager.go:142] Failed to admit pod kube-proxy-vgrgk_kube-system(904e0e2d-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:27 xyz.com kubelet[26351]: E1107 10:57:27.760414   26351 helpers.go:468] PercpuUsage had 0 cpus, but the actual number is 56; ignoring extra CPUs
Nov 07 10:57:28 xyz.com kubelet[26351]: W1107 10:57:28.105039   26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-6r89f_kube-system(90c86ef7-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:28 xyz.com kubelet[26351]: W1107 10:57:28.701069   26351 eviction_manager.go:142] Failed to admit pod kube-proxy-5gc85_kube-system(91236eb4-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:30 xyz.com kubelet[26351]: W1107 10:57:30.033674   26351 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d
Nov 07 10:57:30 xyz.com kubelet[26351]: E1107 10:57:30.033845   26351 kubelet.go:2095] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Nov 07 10:57:33 xyz.com kubelet[26351]: E1107 10:57:33.682890   26351 eviction_manager.go:238] eviction manager: unexpected err: failed to get node info: node 'xyz.com' not found
Nov 07 10:57:35 xyz.com kubelet[26351]: W1107 10:57:35.035448   26351 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d
Nov 07 10:57:35 xyz.com kubelet[26351]: E1107 10:57:35.035640   26351 kubelet.go:2095] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Nov 07 10:57:35 xyz.com kubelet[26351]: E1107 10:57:35.158420   26351 helpers.go:468] PercpuUsage had 0 cpus, but the actual number is 56; ignoring extra CPUs
Nov 07 10:57:35 xyz.com kubelet[26351]: E1107 10:57:35.962221   26351 kubelet_node_status.go:390] Error updating node status, will retry: error getting node "xyz.com": nodes "xyz.com" not found
Nov 07 10:57:35 xyz.com kubelet[26351]: E1107 10:57:35.964898   26351 kubelet_node_status.go:390] Error updating node status, will retry: error getting node "xyz.com": nodes "xyz.com" not found
Nov 07 10:57:35 xyz.com kubelet[26351]: E1107 10:57:35.966502   26351 kubelet_node_status.go:390] Error updating node status, will retry: error getting node "xyz.com": nodes "xyz.com" not found
Nov 07 10:57:35 xyz.com kubelet[26351]: E1107 10:57:35.968578   26351 kubelet_node_status.go:390] Error updating node status, will retry: error getting node "xyz.com": nodes "xyz.com" not found
Nov 07 10:57:35 xyz.com kubelet[26351]: E1107 10:57:35.970564   26351 kubelet_node_status.go:390] Error updating node status, will retry: error getting node "xyz.com": nodes "xyz.com" not found
Nov 07 10:57:35 xyz.com kubelet[26351]: E1107 10:57:35.970582   26351 kubelet_node_status.go:382] Unable to update node status: update node status exceeds retry count
Nov 07 10:57:40 xyz.com kubelet[26351]: W1107 10:57:40.037259   26351 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d
Nov 07 10:57:40 xyz.com kubelet[26351]: E1107 10:57:40.037713   26351 kubelet.go:2095] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Nov 07 10:57:40 xyz.com kubelet[26351]: E1107 10:57:40.858703   26351 helpers.go:468] PercpuUsage had 0 cpus, but the actual number is 56; ignoring extra CPUs
Nov 07 10:57:43 xyz.com kubelet[26351]: E1107 10:57:43.683118   26351 eviction_manager.go:238] eviction manager: unexpected err: failed to get node info: node 'xyz.com' not found

可能是什么问题。此外,当我在工作节点上启动 pod 时,它们仍处于 PENDING 状态,而当我在主节点上启动它们时,它们只会被执行

【问题讨论】:

我对您的日志消息被截断感到困惑;您包含了超长主机名和不必要的前导时间戳,但在日志消息中截断了实际有意义的文本。这不适合寻求帮助 @MatthewLDaniel 我已经更新了日志 【参考方案1】:

删除$KUBELET_NETWORK_ARGS 中的$KUBELET_NETWORK_ARGS 似乎可以工作@ 987654323@ Github Issue Link

【讨论】:

以上是关于工作节点在 Kubernetes 中始终显示 NotReady 状态的主要内容,如果未能解决你的问题,请参考以下文章

在 Kubernetes 的工作节点之间共享存储/卷?

Kubernetes 集群主/工作节点

是否有任何工具可以根据 kubernetes 中请求的总 pod 自动扩展 GKE 节点?

Kubernetes-Pod基本概念(六)

kubernetes的master节点和node节点

kubernetes 正在运行但未列出工作节点