K8s二进制安装之ETCD错误
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了K8s二进制安装之ETCD错误相关的知识，希望对你有一定的参考价值。
参考技术A etcd v3.4.9使用member list查询etcd状态或者使用endpoint health查询群集状态时
#etcdctl member list
出现如下信息，切记不是报错信息，只是通过客户端访问的时候需要带上证书访问
"level":"warn","ts":"2021-02-23T02:42:32.148-0500","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-633d3464-2a3d-432c-a269-01eb26d31ba0/127.0.0.1:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused\""
Error: context deadline exceeded
#etcdctl endpoint health
"level":"warn","ts":"2021-02-23T02:42:32.148-0500","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-633d3464-2a3d-432c-a269-01eb26d31ba0/127.0.0.1:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused\""
Error: context deadline exceeded
正确的访问方法：
#etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/server.pem --key=/opt/etcd/ssl/server-key.pem \
>--endpoints="https://192.168.100.71:2379,https://192.168.100.72:2379,https://192.168.100.73:2379" member list
结果：
41077e602e1d7711, started, etcd-3, https://192.168.100.73:2380, https://192.168.100.73:2379, false
e3dca3d7a066519b, started, etcd-2, https://192.168.100.72:2380, https://192.168.100.72:2379, false
e8e1060c65b6e78b, started, etcd-1, https://192.168.100.71:2380, https://192.168.100.71:2379, false
etcd单节点安装
本篇安装单个etcd，然后进行扩容etcd节点至2个、3个
二进制安装k8s 1.11.0 实验架构 master: 192.168.0.91 etcd node2: 192.168.0.92 node3: 192.168.0.93 1、环境配置 如下操作在所有节点操作 配置hosts解析 [[email protected]-10-1-1-8 k8s]# hostnamectl set-hostname master [[email protected]-10-1-1-68 ~]# hostnamectl set-hostname node2 [[email protected]-10-1-1-111 ~]# hostnamectl set-hostname node3 cat >>/etc/hosts<<EOF 192.168.0.91 master 192.168.0.92 node2 192.168.0.93 node3 EOF 禁用selinux sed -i ‘s/SELINUX=permissive/SELINUX=disabled/‘ /etc/sysconfig/selinux 关闭swap 注释/etc/fstab文件里swap相关的行 所有节点都重启 开启forward iptables -P FORWARD ACCEPT 配置转发相关参数 cat >> /etc/sysctl.d/k8s.conf <<EOF net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 vm.swappiness=0 EOF sysctl --system 加载ipvs相关内核模块 如果重新开机，需要重新加载 modprobe ip_vs modprobe ip_vs_rr modprobe ip_vs_wrr modprobe ip_vs_sh modprobe nf_conntrack_ipv4 lsmod | grep ip_vs 2、安装docker 如下操作在所有节点操作 v1.11.0版本推荐使用docker v17.03, v1.11,v1.12,v1.13, 也可以使用，再高版本的docker可能无法正常使用。 测试发现17.09无法正常使用，不能使用资源限制(内存CPU) 卸载自带docker yum remove -y docker-ce docker-ce-selinux container-selinux 配置Docker仓库镜像 wget https://download.docker.com/linux/centos/7/x86_64/stable/Packages/docker-ce-17.03.2.ce-1.el7.centos.x86_64.rpm wget https://download.docker.com/linux/centos/7/x86_64/stable/Packages/docker-ce-selinux-17.03.2.ce-1.el7.centos.noarch.rpm 安装 Docker 和依赖包 yum install -y docker-ce-*.rpm 开机启动 systemctl enable docker 启动 docker 服务 systemctl start docker 3、安装CFSSL证书生成工具 只在master节点操作 mkdir -pv /server/software/k8s cd /server/software/k8s wget下载cfssl工具 wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 安装cfssl工具 只要把安装包改下名字，移动到usr/local/bin/下，加上授权即可 mv cfssl-certinfo_linux-amd64 /usr/local/bin/cfssl-certinfo mv cfssl_linux-amd64 /usr/local/bin/cfssl mv cfssljson_linux-amd64 /usr/local/bin/cfssljson chmod +x /usr/local/bin/cfssl* 4、创建CA配置文件：生成其他组件ca证书时需要用到(除了根证书) 只在master节点操作 mkdir -p $HOME/ssl && cd $HOME/ssl cat >ca-config.json<<EOF { "signing": { "default": { "expiry": "87600h" }, "profiles": { "kubernetes": { "usages": [ "signing", "key encipherment", "server auth", "client auth" ], "expiry": "87600h" } } } } EOF 5、生成 ca 根证书和私钥: 生成其他组件ca证书时需要用到 只在master节点操作 cd $HOME/ssl cat >ca-csr.json<<EOF { "CN": "kubernetes", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "ST": "BeiJing", "L": "BeiJing", "O": "k8s", "OU": "System" } ], "ca": { "expiry": "87600h" } } EOF cfssl gencert -initca ca-csr.json | cfssljson -bare ca 查看生成的证书和私钥 ca-key.pem ca.pem 把根证书和私钥复制到一个目录里面 mkdir -p /etc/kubernetes/cert/ cp ca*.pem /etc/kubernetes/cert/ 6、安装、配置、启动etcd 只在master节点上操作 6.1、生成etcd的ca证书和私钥 cd $HOME/ssl cat >etcd-csr.json<<EOF { "CN": "etcd", "hosts": [ "127.0.0.1", "192.168.0.91", "192.168.0.92", "192.168.0.93" ], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "ST": "BeiJing", "L": "BeiJing", "O": "etcd", "OU": "Etcd Security" } ] } EOF cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes etcd-csr.json | cfssljson -bare etcd 查看生成的证书和私钥 etcd-key.pem etcd.pem 把etcd证书复制到一个目录里面 mkdir -p /etc/etcd/cert/ cp etcd*.pem /etc/etcd/cert/ 6.2、安装etcd mkdir -p /server/software/k8s mkdir -p /opt/k8s/bin cd /server/software/k8s wget https://github.com/coreos/etcd/releases/download/v3.2.18/etcd-v3.2.18-linux-amd64.tar.gz tar -xf etcd-v3.2.18-linux-amd64.tar.gz mv etcd-v3.2.18-linux-amd64/etcd* /opt/k8s/bin chmod +x /opt/k8s/bin/* ln -s /opt/k8s/bin/etcd /usr/bin/etcd etcd --version 6.3 配置etcd启动脚本 cat >> /etc/profile << EOF export ETCD_NAME=$(hostname) export INTERNAL_IP=$(hostname -i | awk ‘{print $NF}‘) export ECTD_CLUSTER=‘master=https://192.168.0.91:2380‘ EOF source /etc/profile mkdir -p /data/etcd cat > /etc/systemd/system/etcd.service <<EOF [Unit] Description=Etcd Server After=network.target After=network-online.target Wants=network-online.target Documentation=https://github.com/coreos [Service] Type=notify WorkingDirectory=/data/etcd EnvironmentFile=-/etc/etcd/etcd.conf ExecStart=/opt/k8s/bin/etcd --name $ETCD_NAME --cert-file=/etc/etcd/cert/etcd.pem --key-file=/etc/etcd/cert/etcd-key.pem --peer-cert-file=/etc/etcd/cert/etcd.pem --peer-key-file=/etc/etcd/cert/etcd-key.pem --trusted-ca-file=/etc/kubernetes/cert/ca.pem --peer-trusted-ca-file=/etc/kubernetes/cert/ca.pem --initial-advertise-peer-urls https://${INTERNAL_IP}:2380 --listen-peer-urls https://${INTERNAL_IP}:2380 --listen-client-urls https://${INTERNAL_IP}:2379,http://127.0.0.1:2379 --advertise-client-urls https://${INTERNAL_IP}:2379 --initial-cluster-token my-etcd-token --initial-cluster $ECTD_CLUSTER --initial-cluster-state new --data-dir=/data/etcd Restart=on-failure RestartSec=5 LimitNOFILE=65536 [Install] WantedBy=multi-user.target EOF 6.4、启动etctd、设置开机启动 systemctl daemon-reload #一定要执行，否则报错 systemctl start etcd systemctl status etcd systemctl enable etcd systemctl stop etcd 6.5、查看单个etcd集群状态 [[email protected] ~]# etcdctl cluster-health member 42f7141ed6110de1 is healthy: got healthy result from https://192.168.0.91:2379 cluster is healthy
以上是关于K8s二进制安装之ETCD错误的主要内容，如果未能解决你的问题，请参考以下文章
Kubernetes 集群 之 二进制安装部署（单Master节点）