工作节点在 Kubernetes 中始终显示 NotReady 状态
Posted
技术标签:
【中文标题】工作节点在 Kubernetes 中始终显示 NotReady 状态【英文标题】:Worker nodes display always NotReady status in Kubernetes 【发布时间】:2018-04-18 17:50:54 【问题描述】:当我加入Kubernetes
集群的worker节点时,worker节点的STATUS显示为NotReady,节点的ROLE显示为无(使用命令kubectl get nodes
)。我在主节点和工作节点上使用的 Kubernetes 版本是 v1.8.2
。我使用的 Docker 版本是17.03.1-ce
。工作节点上的 Kublet 登录如下:
Nov 07 10:57:03 xyz.com kubelet[26351]: E1107 10:57:03.587712 26351 kuberuntime_image.go:126] Remove image "sha256:7b9d24c898ae4ec4882eb8f60f75b9ae3fb753b5e078ecd7b310df8bfbfb9f11" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:03 xyz.com kubelet[26351]: I1107 10:57:03.587729 26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:637ee73c524b815fcb40b6d4056af8ad92cf4adc1e9ecedbb4f98964fae8c9b1" to free 88610277 bytes
Nov 07 10:57:03 xyz.com kubelet[26351]: E1107 10:57:03.590608 26351 remote_image.go:130] RemoveImage "sha256:637ee73c524b815fcb40b6d4056af8ad92cf4adc1e9ecedbb4f98964fae8c9b1" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:03 xyz.com kubelet[26351]: E1107 10:57:03.590641 26351 kuberuntime_image.go:126] Remove image "sha256:637ee73c524b815fcb40b6d4056af8ad92cf4adc1e9ecedbb4f98964fae8c9b1" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:03 xyz.com kubelet[26351]: I1107 10:57:03.590659 26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:b949fa39c923232fc9dc5521d908cc44047d9a951339af5633b057f6a4878dce" to free 274688064 bytes
Nov 07 10:57:03 xyz.com kubelet[26351]: E1107 10:57:03.592550 26351 remote_image.go:130] RemoveImage "sha256:b949fa39c923232fc9dc5521d908cc44047d9a951339af5633b057f6a4878dce" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:03 xyz.com kubelet[26351]: E1107 10:57:03.592577 26351 kuberuntime_image.go:126] Remove image "sha256:b949fa39c923232fc9dc5521d908cc44047d9a951339af5633b057f6a4878dce" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:03 xyz.com kubelet[26351]: I1107 10:57:03.592592 26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:f5bc15a74665edd153f657b9b84fda59b02e3840733abf4d23413e9dc011e202" to free 1076462680 bytes
Nov 07 10:57:03 xyz.com kubelet[26351]: E1107 10:57:03.596231 26351 remote_image.go:130] RemoveImage "sha256:f5bc15a74665edd153f657b9b84fda59b02e3840733abf4d23413e9dc011e202" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:03 xyz.com kubelet[26351]: E1107 10:57:03.596249 26351 kuberuntime_image.go:126] Remove image "sha256:f5bc15a74665edd153f657b9b84fda59b02e3840733abf4d23413e9dc011e202" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:03 xyz.com kubelet[26351]: I1107 10:57:03.596259 26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:05a3bd381fc2470695a35f230afefd7bf978b566253199c4ae5cc96fafa29b37" to free 1840 bytes
Nov 07 10:57:03 xyz.com kubelet[26351]: E1107 10:57:03.598096 26351 remote_image.go:130] RemoveImage "sha256:05a3bd381fc2470695a35f230afefd7bf978b566253199c4ae5cc96fafa29b37" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:03 xyz.com kubelet[26351]: E1107 10:57:03.598128 26351 kuberuntime_image.go:126] Remove image "sha256:05a3bd381fc2470695a35f230afefd7bf978b566253199c4ae5cc96fafa29b37" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:03 xyz.com kubelet[26351]: W1107 10:57:03.598172 26351 eviction_manager.go:435] eviction manager: unexpected error when attempting to reduce nodefs pressure: wanted to free 9223372036854775807 bytes, but freed 0 bytes space with errors in image deletion: [rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist]
Nov 07 10:57:03 xyz.com kubelet[26351]: I1107 10:57:03.598190 26351 eviction_manager.go:346] eviction manager: must evict pod(s) to reclaim nodefs
Nov 07 10:57:03 xyz.com kubelet[26351]: E1107 10:57:03.598205 26351 eviction_manager.go:357] eviction manager: eviction thresholds have been met, but no pods are active to evict
Nov 07 10:57:04 xyz.com kubelet[26351]: W1107 10:57:04.104666 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-lns64_kube-system(827a29d5-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:04 xyz.com kubelet[26351]: W1107 10:57:04.702441 26351 eviction_manager.go:142] Failed to admit pod kube-proxy-ss9r4_kube-system(82d57b1a-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:05 xyz.com kubelet[26351]: W1107 10:57:05.023950 26351 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d
Nov 07 10:57:05 xyz.com kubelet[26351]: E1107 10:57:05.024138 26351 kubelet.go:2095] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Nov 07 10:57:05 xyz.com kubelet[26351]: W1107 10:57:05.303013 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-xsbq9_kube-system(83310557-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:05 xyz.com kubelet[26351]: W1107 10:57:05.906477 26351 eviction_manager.go:142] Failed to admit pod kube-proxy-kwnwq_kube-system(838d0597-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:06 xyz.com kubelet[26351]: W1107 10:57:06.505627 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-2qmxh_kube-system(83e86d90-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:07 xyz.com kubelet[26351]: W1107 10:57:07.103627 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-hgthx_kube-system(8443da6c-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:07 xyz.com kubelet[26351]: W1107 10:57:07.702696 26351 eviction_manager.go:142] Failed to admit pod kube-proxy-mgtpg_kube-system(849f3e0b-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:08 xyz.com kubelet[26351]: W1107 10:57:08.503689 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-vfbkw_kube-system(85197993-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:09 xyz.com kubelet[26351]: W1107 10:57:09.102866 26351 eviction_manager.go:142] Failed to admit pod kube-proxy-wtlt4_kube-system(8574f6cc-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:09 xyz.com kubelet[26351]: W1107 10:57:09.704456 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-9zxvg_kube-system(85d0951b-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:10 xyz.com kubelet[26351]: W1107 10:57:10.025736 26351 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d
Nov 07 10:57:10 xyz.com kubelet[26351]: E1107 10:57:10.025893 26351 kubelet.go:2095] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Nov 07 10:57:10 xyz.com kubelet[26351]: W1107 10:57:10.303805 26351 eviction_manager.go:142] Failed to admit pod kube-proxy-t7gw2_kube-system(862c1c67-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:10 xyz.com kubelet[26351]: W1107 10:57:10.905402 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-hhvbp_kube-system(8687f17a-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:11 xyz.com kubelet[26351]: W1107 10:57:11.501720 26351 eviction_manager.go:142] Failed to admit pod kube-proxy-p5l4m_kube-system(86e2de87-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:12 xyz.com kubelet[26351]: W1107 10:57:12.104620 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-x529x_kube-system(873ee171-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:12 xyz.com kubelet[26351]: W1107 10:57:12.702859 26351 eviction_manager.go:142] Failed to admit pod kube-proxy-b7mc7_kube-system(879a2f7e-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:13 xyz.com kubelet[26351]: W1107 10:57:13.304742 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-td4xn_kube-system(87f609af-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:13 xyz.com kubelet[26351]: W1107 10:57:13.612712 26351 conversion.go:110] Could not get instant cpu stats: different number of cpus
Nov 07 10:57:13 xyz.com kubelet[26351]: W1107 10:57:13.619906 26351 conversion.go:110] Could not get instant cpu stats: different number of cpus
Nov 07 10:57:13 xyz.com kubelet[26351]: W1107 10:57:13.620145 26351 eviction_manager.go:332] eviction manager: attempting to reclaim nodefs
Nov 07 10:57:13 xyz.com kubelet[26351]: I1107 10:57:13.620176 26351 helpers.go:1118] eviction manager: attempting to delete unused containers
Nov 07 10:57:13 xyz.com kubelet[26351]: I1107 10:57:13.622760 26351 helpers.go:1128] eviction manager: attempting to delete unused images
Nov 07 10:57:13 xyz.com kubelet[26351]: I1107 10:57:13.630989 26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:05a3bd381fc2470695a35f230afefd7bf978b566253199c4ae5cc96fafa29b37" to free 1840 bytes
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.634404 26351 remote_image.go:130] RemoveImage "sha256:05a3bd381fc2470695a35f230afefd7bf978b566253199c4ae5cc96fafa29b37" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.634449 26351 kuberuntime_image.go:126] Remove image "sha256:05a3bd381fc2470695a35f230afefd7bf978b566253199c4ae5cc96fafa29b37" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: I1107 10:57:13.634469 26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:637ee73c524b815fcb40b6d4056af8ad92cf4adc1e9ecedbb4f98964fae8c9b1" to free 88610277 bytes
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.638400 26351 remote_image.go:130] RemoveImage "sha256:637ee73c524b815fcb40b6d4056af8ad92cf4adc1e9ecedbb4f98964fae8c9b1" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.638435 26351 kuberuntime_image.go:126] Remove image "sha256:637ee73c524b815fcb40b6d4056af8ad92cf4adc1e9ecedbb4f98964fae8c9b1" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: I1107 10:57:13.638452 26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:b949fa39c923232fc9dc5521d908cc44047d9a951339af5633b057f6a4878dce" to free 274688064 bytes
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.641568 26351 remote_image.go:130] RemoveImage "sha256:b949fa39c923232fc9dc5521d908cc44047d9a951339af5633b057f6a4878dce" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.641601 26351 kuberuntime_image.go:126] Remove image "sha256:b949fa39c923232fc9dc5521d908cc44047d9a951339af5633b057f6a4878dce" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: I1107 10:57:13.641621 26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:f5bc15a74665edd153f657b9b84fda59b02e3840733abf4d23413e9dc011e202" to free 1076462680 bytes
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.645970 26351 remote_image.go:130] RemoveImage "sha256:f5bc15a74665edd153f657b9b84fda59b02e3840733abf4d23413e9dc011e202" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.646004 26351 kuberuntime_image.go:126] Remove image "sha256:f5bc15a74665edd153f657b9b84fda59b02e3840733abf4d23413e9dc011e202" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: I1107 10:57:13.646021 26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:7b9d24c898ae4ec4882eb8f60f75b9ae3fb753b5e078ecd7b310df8bfbfb9f11" to free 965091970 bytes
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.650426 26351 remote_image.go:130] RemoveImage "sha256:7b9d24c898ae4ec4882eb8f60f75b9ae3fb753b5e078ecd7b310df8bfbfb9f11" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.650454 26351 kuberuntime_image.go:126] Remove image "sha256:7b9d24c898ae4ec4882eb8f60f75b9ae3fb753b5e078ecd7b310df8bfbfb9f11" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: I1107 10:57:13.650484 26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:30ee439fbdcf8fdd4bb9b517983bcc386651ccfe20c93e3aaeb3f7f040057b79" to free 79601765 bytes
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.654416 26351 remote_image.go:130] RemoveImage "sha256:30ee439fbdcf8fdd4bb9b517983bcc386651ccfe20c93e3aaeb3f7f040057b79" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.654456 26351 kuberuntime_image.go:126] Remove image "sha256:30ee439fbdcf8fdd4bb9b517983bcc386651ccfe20c93e3aaeb3f7f040057b79" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: I1107 10:57:13.654475 26351 image_gc_manager.go:350] [imageGCManager]: Removing image "sha256:5e89f9aa5754594e83665a15d8ff0faee9c5991f48c143067a53a0e8dcbcef8b" to free 946975195 bytes
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.662062 26351 remote_image.go:130] RemoveImage "sha256:5e89f9aa5754594e83665a15d8ff0faee9c5991f48c143067a53a0e8dcbcef8b" from image service failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.662095 26351 kuberuntime_image.go:126] Remove image "sha256:5e89f9aa5754594e83665a15d8ff0faee9c5991f48c143067a53a0e8dcbcef8b" failed: rpc error: code = Unknown desc = Error response from daemon: reference does not exist
Nov 07 10:57:13 xyz.com kubelet[26351]: W1107 10:57:13.662136 26351 eviction_manager.go:435] eviction manager: unexpected error when attempting to reduce nodefs pressure: wanted to free 9223372036854775807 bytes, but freed 0 bytes space with errors in image deletion: [rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist, rpc error: code = Unknown desc = Error response from daemon: reference does not exist]
Nov 07 10:57:13 xyz.com kubelet[26351]: I1107 10:57:13.662153 26351 eviction_manager.go:346] eviction manager: must evict pod(s) to reclaim nodefs
Nov 07 10:57:13 xyz.com kubelet[26351]: E1107 10:57:13.662167 26351 eviction_manager.go:357] eviction manager: eviction thresholds have been met, but no pods are active to evict
Nov 07 10:57:13 xyz.com kubelet[26351]: W1107 10:57:13.901494 26351 eviction_manager.go:142] Failed to admit pod kube-proxy-msnln_kube-system(88513541-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:14 xyz.com kubelet[26351]: W1107 10:57:14.504867 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-4h2zk_kube-system(88ad16a8-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:15 xyz.com kubelet[26351]: W1107 10:57:15.027416 26351 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d
Nov 07 10:57:15 xyz.com kubelet[26351]: E1107 10:57:15.027629 26351 kubelet.go:2095] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Nov 07 10:57:15 xyz.com kubelet[26351]: W1107 10:57:15.101833 26351 eviction_manager.go:142] Failed to admit pod kube-proxy-wfpjk_kube-system(89084bb6-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:15 xyz.com kubelet[26351]: W1107 10:57:15.708613 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-4mcj7_kube-system(8964d8e8-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:16 xyz.com kubelet[26351]: W1107 10:57:16.304520 26351 eviction_manager.go:142] Failed to admit pod kube-proxy-l2jdt_kube-system(89bfc0a1-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:16 xyz.com kubelet[26351]: W1107 10:57:16.904426 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-6qmgc_kube-system(8a1b38a6-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:17 xyz.com kubelet[26351]: W1107 10:57:17.503529 26351 eviction_manager.go:142] Failed to admit pod kube-proxy-87xmx_kube-system(8a76c081-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:18 xyz.com kubelet[26351]: W1107 10:57:18.303394 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-vmms4_kube-system(8af0ec82-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:18 xyz.com kubelet[26351]: W1107 10:57:18.902607 26351 eviction_manager.go:142] Failed to admit pod kube-proxy-jndnj_kube-system(8b4c3859-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:19 xyz.com kubelet[26351]: W1107 10:57:19.503823 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-znlsg_kube-system(8ba7eed6-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:20 xyz.com kubelet[26351]: W1107 10:57:20.029739 26351 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d
Nov 07 10:57:20 xyz.com kubelet[26351]: E1107 10:57:20.029848 26351 kubelet.go:2095] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Nov 07 10:57:20 xyz.com kubelet[26351]: W1107 10:57:20.103103 26351 eviction_manager.go:142] Failed to admit pod kube-proxy-wb68t_kube-system(8c034e6f-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:20 xyz.com kubelet[26351]: W1107 10:57:20.704166 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-rnlth_kube-system(8c5f0327-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:21 xyz.com kubelet[26351]: W1107 10:57:21.302613 26351 eviction_manager.go:142] Failed to admit pod kube-proxy-pttwm_kube-system(8cba6d18-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:21 xyz.com kubelet[26351]: W1107 10:57:21.904844 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-2zpc6_kube-system(8d1621a3-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:22 xyz.com kubelet[26351]: W1107 10:57:22.503932 26351 eviction_manager.go:142] Failed to admit pod kube-proxy-f2s2b_kube-system(8d718d21-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:23 xyz.com kubelet[26351]: W1107 10:57:23.103857 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-kwttj_kube-system(8dcd5e80-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:23 xyz.com kubelet[26351]: W1107 10:57:23.673069 26351 conversion.go:110] Could not get instant cpu stats: different number of cpus
Nov 07 10:57:23 xyz.com kubelet[26351]: W1107 10:57:23.679490 26351 conversion.go:110] Could not get instant cpu stats: different number of cpus
Nov 07 10:57:23 xyz.com kubelet[26351]: W1107 10:57:23.704826 26351 eviction_manager.go:142] Failed to admit pod kube-proxy-rlx2q_kube-system(8e28ef71-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:24 xyz.com kubelet[26351]: E1107 10:57:24.113346 26351 helpers.go:468] PercpuUsage had 0 cpus, but the actual number is 56; ignoring extra CPUs
Nov 07 10:57:24 xyz.com kubelet[26351]: W1107 10:57:24.312653 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-ffjzb_kube-system(8e8575e3-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:24 xyz.com kubelet[26351]: W1107 10:57:24.903196 26351 eviction_manager.go:142] Failed to admit pod kube-proxy-g57cx_kube-system(8edfc25e-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:25 xyz.com kubelet[26351]: W1107 10:57:25.031318 26351 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d
Nov 07 10:57:25 xyz.com kubelet[26351]: E1107 10:57:25.031532 26351 kubelet.go:2095] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Nov 07 10:57:25 xyz.com kubelet[26351]: W1107 10:57:25.503456 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-rdmwj_kube-system(8f3b764f-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:26 xyz.com kubelet[26351]: W1107 10:57:26.106221 26351 eviction_manager.go:142] Failed to admit pod kube-proxy-qq9k6_kube-system(8f976fdd-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:26 xyz.com kubelet[26351]: W1107 10:57:26.705257 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-n98m7_kube-system(8ff2e041-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:27 xyz.com kubelet[26351]: W1107 10:57:27.302494 26351 eviction_manager.go:142] Failed to admit pod kube-proxy-vgrgk_kube-system(904e0e2d-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:27 xyz.com kubelet[26351]: E1107 10:57:27.760414 26351 helpers.go:468] PercpuUsage had 0 cpus, but the actual number is 56; ignoring extra CPUs
Nov 07 10:57:28 xyz.com kubelet[26351]: W1107 10:57:28.105039 26351 eviction_manager.go:142] Failed to admit pod kube-flannel-ds-6r89f_kube-system(90c86ef7-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:28 xyz.com kubelet[26351]: W1107 10:57:28.701069 26351 eviction_manager.go:142] Failed to admit pod kube-proxy-5gc85_kube-system(91236eb4-c37d-11e7-919f-9418820aee50) - node has conditions: [DiskPressure]
Nov 07 10:57:30 xyz.com kubelet[26351]: W1107 10:57:30.033674 26351 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d
Nov 07 10:57:30 xyz.com kubelet[26351]: E1107 10:57:30.033845 26351 kubelet.go:2095] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Nov 07 10:57:33 xyz.com kubelet[26351]: E1107 10:57:33.682890 26351 eviction_manager.go:238] eviction manager: unexpected err: failed to get node info: node 'xyz.com' not found
Nov 07 10:57:35 xyz.com kubelet[26351]: W1107 10:57:35.035448 26351 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d
Nov 07 10:57:35 xyz.com kubelet[26351]: E1107 10:57:35.035640 26351 kubelet.go:2095] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Nov 07 10:57:35 xyz.com kubelet[26351]: E1107 10:57:35.158420 26351 helpers.go:468] PercpuUsage had 0 cpus, but the actual number is 56; ignoring extra CPUs
Nov 07 10:57:35 xyz.com kubelet[26351]: E1107 10:57:35.962221 26351 kubelet_node_status.go:390] Error updating node status, will retry: error getting node "xyz.com": nodes "xyz.com" not found
Nov 07 10:57:35 xyz.com kubelet[26351]: E1107 10:57:35.964898 26351 kubelet_node_status.go:390] Error updating node status, will retry: error getting node "xyz.com": nodes "xyz.com" not found
Nov 07 10:57:35 xyz.com kubelet[26351]: E1107 10:57:35.966502 26351 kubelet_node_status.go:390] Error updating node status, will retry: error getting node "xyz.com": nodes "xyz.com" not found
Nov 07 10:57:35 xyz.com kubelet[26351]: E1107 10:57:35.968578 26351 kubelet_node_status.go:390] Error updating node status, will retry: error getting node "xyz.com": nodes "xyz.com" not found
Nov 07 10:57:35 xyz.com kubelet[26351]: E1107 10:57:35.970564 26351 kubelet_node_status.go:390] Error updating node status, will retry: error getting node "xyz.com": nodes "xyz.com" not found
Nov 07 10:57:35 xyz.com kubelet[26351]: E1107 10:57:35.970582 26351 kubelet_node_status.go:382] Unable to update node status: update node status exceeds retry count
Nov 07 10:57:40 xyz.com kubelet[26351]: W1107 10:57:40.037259 26351 cni.go:196] Unable to update cni config: No networks found in /etc/cni/net.d
Nov 07 10:57:40 xyz.com kubelet[26351]: E1107 10:57:40.037713 26351 kubelet.go:2095] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Nov 07 10:57:40 xyz.com kubelet[26351]: E1107 10:57:40.858703 26351 helpers.go:468] PercpuUsage had 0 cpus, but the actual number is 56; ignoring extra CPUs
Nov 07 10:57:43 xyz.com kubelet[26351]: E1107 10:57:43.683118 26351 eviction_manager.go:238] eviction manager: unexpected err: failed to get node info: node 'xyz.com' not found
可能是什么问题。此外,当我在工作节点上启动 pod 时,它们仍处于 PENDING
状态,而当我在主节点上启动它们时,它们只会被执行
【问题讨论】:
我对您的日志消息被截断感到困惑;您包含了超长主机名和不必要的前导时间戳,但在日志消息中截断了实际有意义的文本。这不适合寻求帮助 @MatthewLDaniel 我已经更新了日志 【参考方案1】:删除$KUBELET_NETWORK_ARGS
中的$KUBELET_NETWORK_ARGS
似乎可以工作@ 987654323@
Github Issue Link
【讨论】:
以上是关于工作节点在 Kubernetes 中始终显示 NotReady 状态的主要内容,如果未能解决你的问题,请参考以下文章