kubernetes安装(二进制安装)

Posted givenchy_yzl

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了kubernetes安装(二进制安装)相关的知识,希望对你有一定的参考价值。

kubernetes

k8s和docker之间的关系?
k8s是一个容器化管理平台,docker是一个容器,

二进制部署kubernetes

集群角色

  • master节点: 管理集群
  • node节点: 主要用来部署应用
    Master节点部署插件
  • kube-apiserver : 中央管理器,调度管理集群
  • kube-controller-manager :控制器: 管理容器,监控容器
  • kube-scheduler:调度器:调度容器
  • flannel : 提供集群间网络
  • etcd:数据库
  • kubelet
  • kube-proxy
    Node节点部署插件
  • kubelet : 部署容器,监控容器
  • kube-proxy : 提供容器间的网络

节点规划

192.168.1.81 kubernetes-master-01 m1
192.168.1.82 kubernetes-master-02 m2
192.168.1.83 kubernetes-master-03 m3
192.168.1.84 kubernetes-node-01 n1
192.168.1.85 kubernetes-node-02 n2

#虚拟ip
192.168.1.86 kubernetes-master-vip vip

插件规划

# Master节点规划
kube-apiserver
kube-controller-manager
kube-scheduler
flannel
etcd
kubelet
kube-proxy

# Node节点规划
kubelet
kube-proxy

系统优化(全部都做)

# 关闭selinux
sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/selinux/config 

# 关闭防火墙
systemctl disable --now firewalld

# 关闭swap分区
swapoff -a
修改/etc/fstab
echo 'KUBELET_EXTRA_ARGS="--fail-swap-on=false"' > /etc/sysconfig/kubelet   # kubelet忽略swap

# 做好ip解析
[root@kubernetes-master-01 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.81 kubernetes-master-01 m1
192.168.1.82 kubernetes-master-02 m2
192.168.1.83 kubernetes-master-03 m3
192.168.1.84 kubernetes-node-01 n1
192.168.1.85 kubernetes-node-02 n2
#虚拟ip
192.168.1.86 kubernetes-master-vip vip

# 设置主机名
[root@kubernetes-master-01 ~]# hostnamectl set-hostname kubernetes-master-01


# 做免密登录
[root@kubernetes-master-01 ~]#  ssh-keygen -t rsa
[root@kubernetes-master-01 ~]#  for i in m1 m2 m3 n1 n2;do  ssh-copy-id -i ~/.ssh/id_rsa.pub root@$i; done

# 同步集群时间
ntpdate ntp1.aliyun.com
hwclock --systohc

# 配置镜像源
[root@kubernetes-master-01 ~]#  curl  -o /etc/yum.repos.d/CentOS-Base.repo https://repo.huaweicloud.com/repository/conf/CentOS-7-reg.repo
[root@kubernetes-master-01 ~]# yum clean all
[root@kubernetes-master-01 ~]# yum makecache

# 更新系统
[root@kubernetes-master-01 ~]# yum update -y --exclud=kernel*

# 安装基础常用软件
[root@kubernetes-master-01 ~]# yum install wget expect vim net-tools ntp bash-completion ipvsadm ipset jq iptables conntrack sysstat libseccomp -y

# 更新系统内核(docker 对系统内核要求比较高,最好使用4.4+)
[root@kubernetes-master-01 ~]# wget https://elrepo.org/linux/kernel/el7/x86_64/RPMS/kernel-lt-5.4.130-1.el7.elrepo.x86_64.rpm
[root@kubernetes-master-01 ~]# wget https://elrepo.org/linux/kernel/el7/x86_64/RPMS/kernel-lt-devel-5.4.130-1.el7.elrepo.x86_64.rpm

###   注意这里因为linux内核版本一直在更新,所以5.4.130很有可能会找不到,此时进入https://elrepo.org/linux/kernel/el7/x86_64/RPMS选择你对应的版本即可

    ## 安装系统内容
[root@kubernetes-master-01 ~]# yum localinstall -y kernel-lt*
    ## 调到默认启动
[root@kubernetes-master-01 ~]# grub2-set-default  0 && grub2-mkconfig -o /etc/grub2.cfg
    ## 查看当前默认启动的内核
[root@kubernetes-master-01 ~]# grubby --default-kernel
    ## 重启
[root@kubernetes-master-01 ~]# reboot

# 安装IPVS
    yum install -y conntrack-tools ipvsadm ipset conntrack libseccomp

   	## 加载IPVS模块
cat > /etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
ipvs_modules="ip_vs ip_vs_lc ip_vs_wlc ip_vs_rr ip_vs_wrr ip_vs_lblc ip_vs_lblcr ip_vs_dh ip_vs_sh ip_vs_fo ip_vs_nq ip_vs_sed ip_vs_ftp nf_conntrack"
for kernel_module in \\${ipvs_modules}; do
/sbin/modinfo -F filename \\${kernel_module} > /dev/null 2>&1
if [ $? -eq 0 ]; then
/sbin/modprobe \\${kernel_module}
fi
done
EOF

    chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep ip_vs

# 修改内核启动参数
cat > /etc/sysctl.d/k8s.conf << EOF
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
fs.may_detach_mounts = 1
vm.overcommit_memory=1
vm.panic_on_oom=0
fs.inotify.max_user_watches=89100
fs.file-max=52706963
fs.nr_open=52706963
net.ipv4.tcp_keepalive_time = 600
net.ipv4.tcp.keepaliv.probes = 3
net.ipv4.tcp_keepalive_intvl = 15
net.ipv4.tcp.max_tw_buckets = 36000
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp.max_orphans = 327680
net.ipv4.tcp_orphan_retries = 3
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.ip_conntrack_max = 65536
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.top_timestamps = 0
net.core.somaxconn = 16384
EOF

# 立即生效
sysctl --system

安装docker

# 卸载之前安装过得docker
[root@kubernetes-master-01 ~]#  sudo yum remove docker docker-common docker-selinux docker-engine

# 安装docker需要的依赖包
[root@kubernetes-master-01 ~]#  sudo yum install -y yum-utils device-mapper-persistent-data lvm2

# 安装dockeryum源
[root@kubernetes-master-01 ~]#  wget -O /etc/yum.repos.d/docker-ce.repo https://repo.huaweicloud.com/docker-ce/linux/centos/docker-ce.repo

# 安装docker
[root@kubernetes-master-01 ~]#  yum install docker-ce -y

# 设置开机自启动
[root@kubernetes-master-01 ~]#  systemctl enable --now docker.service

集群证书

# 以下命令只需要在master01执行即可

# 安装证书生成工具
wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64

# 设置执行权限
chmod +x cfssljson_linux-amd64
chmod +x cfssl_linux-amd64

# 移动到/usr/local/bin
mv cfssljson_linux-amd64 cfssljson
mv cfssl_linux-amd64 cfssl
mv cfssljson cfssl /usr/local/bin

生成根证书

mkdir -p /opt/cert/ca
cat > /opt/cert/ca/ca-config.json <<EOF
{
  "signing": {
    "default": {
      "expiry": "8760h"
    },
    "profiles": {
      "kubernetes": {
        "usages": [
          "signing",
          "key encipherment",
          "server auth",
          "client auth"
        ],
           "expiry": "8760h"
      }
    }
  }
}
EOF

生成根证书请求文件

cat > /opt/cert/ca/ca-csr.json << EOF
{
  "CN": "kubernetes",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names":[{
    "C": "CN",
    "ST": "ShangHai",
    "L": "ShangHai"
  }]
}
EOF

生成根证书


[root@kubernetes-master-01 ~]# cd /opt/cert/ca/
[root@kubernetes-master-01 /opt/cert/ca]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
2021/03/26 17:34:55 [INFO] generating a new CA key and certificate from CSR
2021/03/26 17:34:55 [INFO] generate received request
2021/03/26 17:34:55 [INFO] received CSR
2021/03/26 17:34:55 [INFO] generating key: rsa-2048
2021/03/26 17:34:56 [INFO] encoded CSR
2021/03/26 17:34:56 [INFO] signed certificate with serial number 661764636777400005196465272245416169967628201792
[root@kubernetes-master-01 /opt/cert/ca]# ll
total 20
-rw-r--r-- 1 root root  285 Mar 26 17:34 ca-config.json
-rw-r--r-- 1 root root  960 Mar 26 17:34 ca.csr
-rw-r--r-- 1 root root  153 Mar 26 17:34 ca-csr.json
-rw------- 1 root root 1675 Mar 26 17:34 ca-key.pem
-rw-r--r-- 1 root root 1281 Mar 26 17:34 ca.pem

部署ETCD集群

节点规划

192.168.1.81 etcd-01
192.168.1.82 etcd-01
192.168.1.83 etcd-01

创建ETCD集群证书

mkdir -p /opt/cert/etcd
cd /opt/cert/etcd

cat > etcd-csr.json << EOF
{
    "CN": "etcd",
    "hosts": [
        "127.0.0.1",
        "192.168.1.81",
        "192.168.1.82",
        "192.168.1.83",
        "192.168.1.84",
        "192.168.1.85",
        "192.168.1.86"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
          "C": "CN",
          "ST": "ShangHai",
          "L": "ShangHai"
        }
    ]
}
EOF
生成ETCD证书
[root@kubernetes-master-01 /opt/cert/etcd]# cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -profile=kubernetes etcd-csr.json | cfssljson -bare etcd
2021/03/26 17:38:57 [INFO] generate received request
2021/03/26 17:38:57 [INFO] received CSR
2021/03/26 17:38:57 [INFO] generating key: rsa-2048
2021/03/26 17:38:58 [INFO] encoded CSR
2021/03/26 17:38:58 [INFO] signed certificate with serial number 179909685000914921289186132666286329014949215773
2021/03/26 17:38:58 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
分发ETCD证书
[root@kubernetes-master-01 /opt/cert/etcd]# for ip in m1 m2 m3;do 
   ssh root@${ip} "mkdir -pv /etc/etcd/ssl"
   scp ../ca/ca*.pem  root@${ip}:/etc/etcd/ssl
   scp ./etcd*.pem  root@${ip}:/etc/etcd/ssl   
 done
 
mkdir: created directory ‘/etc/etcd’
mkdir: created directory ‘/etc/etcd/ssl’
ca-key.pem                                                      100% 1675   299.2KB/s   00:00    
ca.pem                                                          100% 1281   232.3KB/s   00:00    
etcd-key.pem                                                    100% 1675     1.4MB/s   00:00    
etcd.pem                                                        100% 1379   991.0KB/s   00:00    
mkdir: created directory ‘/etc/etcd’
mkdir: created directory ‘/etc/etcd/ssl’
ca-key.pem                                                      100% 1675     1.1MB/s   00:00    
ca.pem                                                          100% 1281   650.8KB/s   00:00    
etcd-key.pem                                                    100% 1675   507.7KB/s   00:00    
etcd.pem                                                        100% 1379   166.7KB/s   00:00    
mkdir: created directory ‘/etc/etcd’
mkdir: created directory ‘/etc/etcd/ssl’
ca-key.pem                                                      100% 1675   109.1KB/s   00:00    
ca.pem                                                          100% 1281   252.9KB/s   00:00    
etcd-key.pem                                                    100% 1675   121.0KB/s   00:00    
etcd.pem                                                        100% 1379   180.4KB/s   00:00    
[root@kubernetes-master-01 /opt/cert/etcd]# ll /etc/etcd/ssl/
total 16
-rw------- 1 root root 1675 Mar 26 17:41 ca-key.pem
-rw-r--r-- 1 root root 1281 Mar 26 17:41 ca.pem
-rw------- 1 root root 1675 Mar 26 17:41 etcd-key.pem
-rw-r--r-- 1 root root 1379 Mar 26 17:41 etcd.pem

部署ETCD

# 下载ETCD安装包
wget https://mirrors.huaweicloud.com/etcd/v3.3.24/etcd-v3.3.24-linux-amd64.tar.gz

# 解压
tar xf etcd-v3.3.24-linux-amd64.tar.gz

# 分发至其他节点
for i in m1 m2 m3
do
    scp ./etcd-v3.3.24-linux-amd64/etcd* root@$i:/usr/local/bin/
done
[root@kubernetes-master-01 /opt/etcd-v3.3.24-linux-amd64]# etcd --version
etcd Version: 3.3.24
Git SHA: bdd57848d
Go Version: go1.12.17
Go OS/Arch: linux/amd64

注册ETCD服务

# 在三台master节点上执行
mkdir -pv /etc/kubernetes/conf/etcd
[root@kubernetes-master-01 etcd]# hostnamectl set-hostname kubernetes-master-01
ETCD_NAME=`hostname`
INTERNAL_IP=`hostname -i`
INITIAL_CLUSTER=kubernetes-master-01=https://192.168.1.81:2380,kubernetes-master-02=https://192.168.1.82:2380,kubernetes-master-03=https://192.168.1.83:2380

cat << EOF | sudo tee /usr/lib/systemd/system/etcd.service
[Unit]
Description=etcd
Documentation=https://github.com/coreos

[Service]
ExecStart=/usr/local/bin/etcd \\\\
  --name $ETCD_NAME \\\\
  --cert-file=/etc/etcd/ssl/etcd.pem \\\\
  --key-file=/etc/etcd/ssl/etcd-key.pem \\\\
  --peer-cert-file=/etc/etcd/ssl/etcd.pem \\\\
  --peer-key-file=/etc/etcd/ssl/etcd-key.pem \\\\
  --trusted-ca-file=/etc/etcd/ssl/ca.pem \\\\
  --peer-trusted-ca-file=/etc/etcd/ssl/ca.pem \\\\
  --peer-client-cert-auth \\\\
  --client-cert-auth \\\\
  --initial-advertise-peer-urls https://${INTERNAL_IP}:2380 \\\\
  --listen-peer-urls https://${INTERNAL_IP}:2380 \\\\
  --listen-client-urls https://${INTERNAL_IP}:2379,https://127.0.0.1:2379 \\\\
  --advertise-client-urls https://${INTERNAL_IP}:2379 \\\\
  --initial-cluster-token etcd-cluster \\\\
  --initial-cluster ${INITIAL_CLUSTER} \\\\
  --initial-cluster-state new \\\\
  --data-dir=/var/lib/etcd
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

# 启动ETCD服务
systemctl enable --now etcd

测试ETCD服务

# 第一种方式
ETCDCTL_API=3 etcdctl \\
--cacert=/etc/etcd/ssl/etcd.pem \\
--cert=/etc/etcd/ssl/etcd.pem \\
--key=/etc/etcd/ssl/etcd-key.pem \\
--endpoints="https://192.168.1.81:2379,https://192.168.1.82:2379,https://192.168.1.83:2379" \\
endpoint status --write-out='table'


# 第二种方式
ETCDCTL_API=3 etcdctl \\
--cacert=/etc/etcd/ssl/etcd.pem \\
--cert=/etc/etcd/ssl/etcd.pem \\
--key=/etc/etcd/ssl/etcd-key.pem \\
--endpoints="https://192.168.1.81:2379,https://192.168.1.82:2379,https://192.168.1.83:2379" \\
member list --write-out='table'

部署master节点

主要把master节点上的各个组件部署成功。

集群规划

192.168.1.81 172.16.1.81 kubernetes-master-01 m1 
192.168.1.82 172.16.1.82 kubernetes-master-02 m2
192.168.1.83 172.16.1.83 kubernetes-master-03 m3

kube-apiserver、控制器、调度器、flannel、etcd、kubelet、kube-proxy、DNS

创建证书

创建集群证书

创建集群CA证书
# 只需要在master01上执行
[root@kubernetes-master-01 k8s]# mkdir /opt/cert/k8s
[root@kubernetes-master-01 k8s]# cd /opt/cert/k8s
[root@kubernetes-master-01 k8s]# pwd
/opt/cert/k8s
[root@kubernetes-master-01 k8s]# cat > ca-config.json << EOF
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "kubernetes": {
         "expiry": "87600h",
         "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ]
      }
    }
  }
}
EOF
[root@kubernetes-master-01 k8s]# cat > ca-csr.json << EOF
{
    "CN": "kubernetes",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "ShangHai",
            "ST": "ShangHai"
        }
    ]
}
EOF
[root@kubernetes-master-01 k8s]# ll
total 8
-rw-r--r-- 1 root root 294 Sep 13 19:59 ca-config.json
-rw-r--r-- 1 root root 212 Sep 13 20:01 ca-csr.json
[root@kubernetes-master-01 k8s]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
2020/09/13 20:01:45 [INFO] generating a new CA key and certificate from CSR
2020/09/13 20:01:45 [INFO] generate received request
2020/09/13 20:01:45 [INFO] received CSR
2020/09/13 20:01:45 [INFO] generating key: rsa-2048
2020/09/13 20:01:46 [INFO] encoded CSR
2020/09/13 20:01:46 [INFO] signed certificate with serial number 588993429584840635805985813644877690042550093427
[root@kubernetes-master-01 k8s]# ll
total 20
-rw-r--r-- 1 root root  294 Sep 13 19:59 ca-config.json
-rw-r--r-- 1 root root  960 Sep 13 20:01 ca.csr
-rw-r--r-- 1 root root  212 Sep 13 20:01 ca-csr.json
-rw------- 1 root root 1679 Sep 13 20:01 ca-key.pem
-rw-r--r-- 1 root root 1273 Sep 13 20:01 ca.pem
创建集群普通证书

创建集群各个组件之间的证书

创建kube-apiserver的证书
[root@k8s-m-01 /opt/cert/k8s]# mkdir /opt/cert/k8s
[root@k8s-m-01 /opt/cert/k8s]# cd /opt/cert/k8s
[root@k8s-m-01 /opt/cert/k8s]# cat > server-csr.json << EOF
{
    "CN": "kubernetes",
    "hosts": [
        "127.0.0.1",
        "192.168.1.81",
        "192.168.1.82",
        "192.168.1.83",
        "192.168.1.84",
        "192.168.1.85",
        "192.168.1.86",
        "10.96.0.1",
        "kubernetes",
        "kubernetes.default",
        "kubernetes.default.svc",
        "kubernetes.default.svc.cluster",
        "kubernetes.default.svc.cluster.local"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "ShangHai",
            "ST": "ShangHai"
        }
    ]
}
EOF

[root@k8s-m-01 /opt/cert/k8s]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson -bare server
2021/03/29 09:31:02 [INFO] generate received request
2021/03/29 09:31:02 [INFO] received CSR
2021/03/29 09:31:02 [INFO] generating key: rsa-2048
2021/03/29 09:31:02 [INFO] encoded CSR
2021/03/29 09:31:02 [INFO] signed certificate with serial number 475285860832876170844498652484239182294052997083
2021/03/29 09:31:02 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
[root@k8s-m-01 /opt/cert/k8s]# ll
total 36
-rw-r--r-- 1 root root  294 Mar 29 09:13 ca-config.json
-rw-r--r-- 1 root root  960 Mar 29 09:16 ca.csr
-rw-r--r-- 1 root root  214 Mar 29 09:14 ca-csr.json
-rw------- 1 root root 1675 Mar 29 09:16 ca-key.pem
-rw-r--r-- 1 root root 1281 Mar 29 09:16 ca.pem
-rw-r--r-- 1 root root 1245 Mar 29 09:31 server.csr
-rw-r--r-- 1 root root  603 Mar 29 09:29 server-csr.json
-rw------- 1 root root 1675 Mar 29 09:31 server-key.pem
-rw-r--r-- 1 root root 1574 Mar 29 09:31 server.pem

创建controller-manager的证书
[kubernetes-master-01 /opt/cert/k8s]# cat > kube-controller-manager-csr.json << EOF
{ 
    "CN": "system:kube-controller-manager", 
    "hosts": [ 
        "127.0.0.1",
        "192.168.1.83", 
        "192.168.1.81", 
        "192.168.1.82", 
        "192.168.1.84",
        "192.168.1.85",
        "192.168.1.86"
    ],
    "key": {
        "algo": "rsa", 
        "size": 2048
    },
    "names": [ 
        { 
            "C": "CN", 
            "L": "ShangHai", 
            "ST": "ShangHai", 
            "O": "system:kube-controller-manager",
            "OU": "System"
        } 
    ] 
}
EOF
[kubernetes-master-01 /opt/cert/k8s]# vim kube-controller-manager-csr.json 
[root@kubernetes-master-01 /opt/cert/k8s]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager
2021/03/29 09:33:31 [INFO] generate received request
2021/03/29 09:33:31 [INFO] received CSR
2021/03/29 09:33:31 [INFO] generating key: rsa-2048
2021/03/29 09:33:31 [INFO] encoded CSR
2021/03/29 09:33:31 [INFO] signed certificate with serial number 159207911625502250093013220742142932946474251607
2021/03/29 09:33:31 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").

创建kube-scheduler的证书
[root@kubernetes-master-01 /opt/cert/k8s]# cat > kube-scheduler-csr.json << EOF
{ 
    "CN": "system:kube-scheduler", 
    "hosts": [ 
        "127.0.0.1",
        "192.168.1.83", 
        "192.168.1.81", 
        "192.168.1.82", 
        "192.168.1.84",
        "192.168.1.85",
        "192.168.1.86"
    ],
    "key": {
        "algo": "rsa", 
        "size": 2048
    },
    "names": [ 
        { 
            "C": "CN", 
            "L": "ShangHai", 
            "ST": "ShangHai", 
            "O": "system:kube-scheduler",
            "OU": "System"
        } 
    ] 
}
EOF

[root@kubernetes-master-01 /opt/cert/k8s]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-scheduler-csr.json | cfssljson -bare kube-scheduler
2021/03/29 09:34:57 [INFO] generate received request
2021/03/29 09:34:57 [INFO] received CSR
2021/03/29 09:34:57 [INFO] generating key: rsa-2048
2021/03/29 09:34:58 [INFO] encoded CSR
2021/03/29 09:34:58 [INFO] signed certificate with serial number 38647006614878532408684142936672497501281226307
2021/03/29 09:34:58 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").

创建kube-proxy证书
[root@kubernetes-master-01 /opt/cert/k8s]# cat > kube-proxy-csr.json << EOF
{
    "CN":"system:kube-proxy",
    "hosts":[],
    "key":{
        "algo":"rsa",
        "size":2048
    },
    "names":[
        {
            "C":"CN",
            "L":"ShangHai",
            "ST":"ShangHai",
            "O":"system:kube-proxy",
            "OU":"System"
        }
    ]
}
EOF
[root@kubernetes-master-01 /opt/cert/k8s]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy
2021/03/29 09:37:44 [INFO] generate received request
2021/03/29 09:37:44 [INFO] received CSR
2021/03/29 09:37:44 [INFO] generating key: rsa-2048
2021/03/29 09:37:44 [INFO] encoded CSR
2021/03/29 09:37:44 [INFO] signed certificate with serial number 703321465371340829919693910125364764243453439484
2021/03/29 09:37:44 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").

创建集群管理员证书
[root@kubernetes-master-01 /opt/cert/k8s]# cat > admin-csr.json << EOF
{
    "CN":"admin",
    "key":{
        "algo":"rsa",
        "size":2048
    },
    "names":[
        {
            "C":"CN",
            "L":"ShangHai",
            "ST":"ShangHai",
            "O":"system:masters",
            "OU":"System"
        }
    ]
}
EOF
[root@kubernetes-master-01 /opt/cert/k8s]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin
2021/03/29 09:36:26 [INFO] generate received request
2021/03/29 09:36:26 [INFO] received CSR
2021/03/29 09:36:26 [INFO] generating key: rsa-2048
2021/03/29 09:36:26 [INFO] encoded CSR
2021/03/29 09:36:26 [INFO] signed certificate with serial number 258862825289855717894394114308507213391711602858
2021/03/29 09:36:26 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").

颁发证书
[root@kubernetes-master-01 /opt/cert/k8s]# mkdir -pv /etc/kubernetes/ssl
[root@kubernetes-master-01 /opt/cert/k8s]# cp -p ./{ca*pem,server*pem,kube-controller-manager*pem,kube-scheduler*.pem,kube-proxy*pem,admin*.pem} /etc/kubernetes/ssl

[root@kubernetes-master-01 /opt/cert/k8s]# for i in m1 m2 m3;do
ssh root@$i "mkdir -pv /etc/kubernetes/ssl"
以上是关于kubernetes安装(二进制安装)的主要内容,如果未能解决你的问题,请参考以下文章

kubernetes安装(二进制安装)

Kubernetes-1.18.4二进制高可用安装

centos离线二进制安装kubernetes和docker

kubernetes 二进制安装(v1.20.15)加塞一个工作节点

kubernetes二进制安装

kubernetes 二进制安装(v1.20.16)环境准备