linux运维架构之路-Kubernetes1.13离线集群部署双向认证
Posted yanxinjiang
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了linux运维架构之路-Kubernetes1.13离线集群部署双向认证相关的知识，希望对你有一定的参考价值。
一、部署环境
1、服务器规划
|
IP地址 |
主机名 |
CPU |
内存 |
|
10.0.0.100 |
k8s-master01 |
2C |
2G |
|
10.0.0.101 |
k8s-node01 |
2C |
2G |
|
10.0.0.102 |
k8s-node02 |
2C |
2G |
2、Kubernetes 1.13包下载
https://github.com/kubernetes/kubernetes/releases
所用到包的版本：
3、Kubernetes工作流程
4、Kubernetes部署前准备
①系统环境
[root@k8s-master01 ~]# cat /etc/redhat-release CentOS Linux release 7.6.1810 (Core) [root@k8s-master01 ~]# uname -r 3.10.0-957.21.3.el7.x86_64 [root@k8s-master01 ~]# systemctl status firewalld.service ● firewalld.service - firewalld - dynamic firewall daemon Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled) Active: inactive (dead) Docs: man:firewalld(1) [root@k8s-master01 ~]# getenforce Disabled
②关闭Swap
swapoff -a && sysctl -w vm.swappiness=0 vim /etc/fstab UUID=b6ff8dda-f2db-49ad-a091-bdd4e82cdace / xfs defaults 0 0 UUID=f4eaa64a-f9c7-4e11-a78b-13d423ac65d2 /boot xfs defaults 0 0 #UUID=bc4d148a-56e0-4fa8-96a2-d10e5470963c swap swap defaults 0 0
③设置Docker所需参数
cat << EOF | tee /etc/sysctl.d/k8s.conf net.ipv4.ip_forward = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 EOF sysctl -p /etc/sysctl.d/k8s.conf
二、部署 Docker(所有节点)
1、安装
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo yum list docker-ce --showduplicates | sort -r yum install docker-ce -y systemctl start docker && systemctl enable docker
2、创建安装所需目录
mkdir /k8s/etcd/bin,cfg,ssl -p
mkdir /k8s/kubernetes/bin,cfg,ssl -p
3、安装生成证书工具CFSSL(Master节点)
mkdir tools && cd tools wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 chmod +x cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64 mv cfssl_linux-amd64 /usr/local/bin/cfssl mv cfssljson_linux-amd64 /usr/local/bin/cfssljson mv cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo
4、创建认证证书(Master节点)
①创建 ETCD 证书
cat << EOF | tee ca-config.json "signing": "default": "expiry": "87600h" , "profiles": "www": "expiry": "87600h", "usages": [ "signing", "key encipherment", "server auth", "client auth" ] EOF
创建 ETCD CA 配置文件
cat << EOF | tee ca-csr.json "CN": "etcd CA", "key": "algo": "rsa", "size": 2048 , "names": [ "C": "CN", "L": "Shenzhen", "ST": "Shenzhen" ] EOF
创建 ETCD Server 证书
cat << EOF | tee server-csr.json "CN": "etcd", "hosts": [ "10.0.0.100", "10.0.0.101", "10.0.0.102" ], "key": "algo": "rsa", "size": 2048 , "names": [ "C": "CN", "L": "Shenzhen", "ST": "Shenzhen" ] EOF
生成 ETCD CA 证书和私钥
cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server
②创建 Kubernetes CA 证书
cat << EOF | tee ca-config.json "signing": "default": "expiry": "87600h" , "profiles": "kubernetes": "expiry": "87600h", "usages": [ "signing", "key encipherment", "server auth", "client auth" ] EOF
cat << EOF | tee ca-csr.json "CN": "kubernetes", "key": "algo": "rsa", "size": 2048 , "names": [ "C": "CN", "L": "Shenzhen", "ST": "Shenzhen", "O": "k8s", "OU": "System" ] EOF
生成Kubernetes 证书和秘钥
cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
③生成API_SERVER证书
cat << EOF | tee server-csr.json "CN": "kubernetes", "hosts": [ "10.0.0.1", "127.0.0.1", "10.0.0.100", "kubernetes", "kubernetes.default", "kubernetes.default.svc", "kubernetes.default.svc.cluster", "kubernetes.default.svc.cluster.local" ], "key": "algo": "rsa", "size": 2048 , "names": [ "C": "CN", "L": "Shenzhen", "ST": "Shenzhen", "O": "k8s", "OU": "System" ] EOF
生成API_SERVER证书和秘钥
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson -bare server
④创建 Kubernetes Proxy 证书
cat << EOF | tee kube-proxy-csr.json "CN": "system:kube-proxy", "hosts": [], "key": "algo": "rsa", "size": 2048 , "names": [ "C": "CN", "L": "Shenzhen", "ST": "Shenzhen", "O": "k8s", "OU": "System" ] EOF
生成Kubernetes Proxy 证书和秘钥
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy
5、ssh-key免秘钥认证
ssh-keygen
ssh-copy-id 10.0.0.101 ssh-copy-id 10.0.0.102
三、部署ETCD服务
1、上传解压文件
tar xf etcd-v3.3.10-linux-amd64.tar.gz cd etcd-v3.3.10-linux-amd64/ cp etcd etcdctl /k8s/etcd/bin/
2、创建ETCD配置文件
vim /k8s/etcd/cfg/etcd #[Member] ETCD_NAME="etcd01" ETCD_DATA_DIR="/var/lib/etcd/default.etcd" ETCD_LISTEN_PEER_URLS="https://10.0.0.100:2380" ETCD_LISTEN_CLIENT_URLS="https://10.0.0.100:2379" #[Clustering] ETCD_INITIAL_ADVERTISE_PEER_URLS="https://10.0.0.100:2380" ETCD_ADVERTISE_CLIENT_URLS="https://10.0.0.100:2379" ETCD_INITIAL_CLUSTER="etcd01=https://10.0.0.100:2380,etcd02=https://10.0.0.101:2380,etcd03=https://10.0.0.102:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new"
创建 etcd的 systemd unit 文件
vim /usr/lib/systemd/system/etcd.service [Unit] Description=Etcd Server After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify EnvironmentFile=/k8s/etcd/cfg/etcd ExecStart=/k8s/etcd/bin/etcd --name=$ETCD_NAME --data-dir=$ETCD_DATA_DIR --listen-peer-urls=$ETCD_LISTEN_PEER_URLS --listen-client-urls=$ETCD_LISTEN_CLIENT_URLS,http://127.0.0.1:2379 \\ --advertise-client-urls=$ETCD_ADVERTISE_CLIENT_URLS --initial-advertise-peer-urls=$ETCD_INITIAL_ADVERTISE_PEER_URLS --initial-cluster=$ETCD_INITIAL_CLUSTER --initial-cluster-token=$ETCD_INITIAL_CLUSTER_TOKEN --initial-cluster-state=new --cert-file=/k8s/etcd/ssl/server.pem --key-file=/k8s/etcd/ssl/server-key.pem --peer-cert-file=/k8s/etcd/ssl/server.pem --peer-key-file=/k8s/etcd/ssl/server-key.pem --trusted-ca-file=/k8s/etcd/ssl/ca.pem --peer-trusted-ca-file=/k8s/etcd/ssl/ca.pem Restart=on-failure LimitNOFILE=65536 [Install] WantedBy=multi-user.target
拷贝证书文件
cd tools
cp ca*pem server*pem /k8s/etcd/ssl
将启动文件、配置文件分发到 节点1、节点2
cd /k8s scp -r etcd 10.0.0.101:/k8s/ scp -r etcd 10.0.0.102:/k8s/ scp /usr/lib/systemd/system/etcd.service 10.0.0.101:/usr/lib/systemd/system/etcd.service scp /usr/lib/systemd/system/etcd.service 10.0.0.102:/usr/lib/systemd/system/etcd.service
修改节点1、节点2配置文件
vim /k8s/etcd/cfg/etcd #[Member] ETCD_NAME="etcd02" ETCD_DATA_DIR="/var/lib/etcd/default.etcd" ETCD_LISTEN_PEER_URLS="https://10.0.0.101:2380" ETCD_LISTEN_CLIENT_URLS="https://10.0.0.101:2379" #[Clustering] ETCD_INITIAL_ADVERTISE_PEER_URLS="https://10.0.0.101:2380" ETCD_ADVERTISE_CLIENT_URLS="https://10.0.0.101:2379" ETCD_INITIAL_CLUSTER="etcd01=https://10.0.0.100:2380,etcd02=https://10.0.0.101:2380,etcd03=https://10.0.0.102:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new"
vim /k8s/etcd/cfg/etcd #[Member] ETCD_NAME="etcd03" ETCD_DATA_DIR="/var/lib/etcd/default.etcd" ETCD_LISTEN_PEER_URLS="https://10.0.0.102:2380" ETCD_LISTEN_CLIENT_URLS="https://10.0.0.102:2379" #[Clustering] ETCD_INITIAL_ADVERTISE_PEER_URLS="https://10.0.0.102:2380" ETCD_ADVERTISE_CLIENT_URLS="https://10.0.0.102:2379" ETCD_INITIAL_CLUSTER="etcd01=https://10.0.0.100:2380,etcd02=https://10.0.0.101:2380,etcd03=https://10.0.0.102:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new"
3、启动ETCD服务
systemctl daemon-reload
systemctl enable etcd
systemctl start etcd
4、验证集群是否正常运行
/k8s/etcd/bin/etcdctl --ca-file=/k8s/etcd/ssl/ca.pem --cert-file=/k8s/etcd/ssl/server.pem --key-file=/k8s/etcd/ssl/server-key.pem --endpoints="https://10.0.0.100:2379,\\ https://10.0.0.101:2379,\\ https://10.0.0.102:2379" cluster-health
member 5db3ea816863435 is healthy: got healthy result from https://10.0.0.102:2379 member 991b5845cecb31b is healthy: got healthy result from https://10.0.0.101:2379 member c67ee2780d64a0d4 is healthy: got healthy result from https://10.0.0.100:2379 cluster is healthy 注意： 启动ETCD集群同时启动二个节点，启动一个节点集群是无法正常启动的；
以上是关于linux运维架构之路-Kubernetes1.13离线集群部署双向认证的主要内容，如果未能解决你的问题，请参考以下文章