OpenShift 4.2 添加RHEL节点

Posted ericnie

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了OpenShift 4.2 添加RHEL节点相关的知识，希望对你有一定的参考价值。

OpenShift 4.2版本下如何加入RHEL 7.6的节点。

部署架构图

技术图片

 

 

1.worker3所在的物理机

  • 建立一个helper-woker03.cfg文件用于节点虚机的建立和启动，注意nameserver字段应该指到helper也就是dns和负载均衡所在的机器。
[root@localhost data]# cat helper-worker03.cfg 
# System authorization information
auth --enableshadow --passalgo=sha512
# Use CDROM installation media
cdrom
# Use graphical install
text
# Run the Setup Agent on first boot
firstboot --enable
ignoredisk --only-use=vda
# Keyboard layouts
keyboard --vckeymap=us --xlayouts=‘us‘
# System language
lang en_US.UTF-8

# Network information
network  --bootproto=static --device=eth0 --gateway=192.168.7.1 --ip=192.168.7.19  --netmask=255.255.255.0 --nameserver=192.168.7.11  --ipv6=auto --activate
network  --hostname=worker-3.ocp4.redhat.ren

# Root password
rootpw --plaintext redhat
# System services
services --enabled="chronyd"
# System timezone
timezone Asia/Shanghai --isUtc --ntpservers=0.centos.pool.ntp.org,1.centos.pool.ntp.org,2.centos.pool.ntp.org,3.centos.pool.ntp.org
# System bootloader configuration
bootloader --append=" crashkernel=auto" --location=mbr --boot-drive=vda
# Partition clearing information
clearpart --none --initlabel
# Disk partitioning information
part pv.156 --fstype="lvmpv" --ondisk=vda --size=229695
part /boot --fstype="xfs" --ondisk=vda --size=1024
volgroup vg0 --pesize=4096 pv.156
logvol /  --fstype="xfs" --size=229184 --name=root --vgname=vg0
logvol swap  --fstype="swap" --size=508 --name=swap --vgname=vg0
reboot

%packages
@^minimal
@core
chrony
kexec-tools

%end

%addon com_redhat_kdump --enable --reserve-mb=‘auto‘

%end

%anaconda
pwpolicy root --minlen=6 --minquality=1 --notstrict --nochanges --notempty
pwpolicy user --minlen=6 --minquality=1 --notstrict --nochanges --emptyok
pwpolicy luks --minlen=6 --minquality=1 --notstrict --nochanges --notempty
%end

 

  • 然后建立虚机
virt-install --name="ocp4-worker3" --vcpus=2 --ram=8192 --disk path=/data/kvm/ocp4-worker3.qcow2,bus=virtio,size=230 --os-variant centos7.0 --network bridge=br0,model=virtio --boot menu=on --location /data/rhel-server-7.6-x86_64-dvd.iso --initrd-inject helper-worker03.cfg --extra-args "inst.ks=file:/helper-worker03.cfg" --noautoconsole
virsh start ocp4-worker3

 

2.woker3节点

  • 登录新建的虚机，添加yum源
[base]
name=base
baseurl=http://192.168.7.1:8080/repo/rhel-7-server-rpms/
enabled=1
gpgcheck=0
[ansible]
name=ansible
baseurl=http://192.168.7.1:8080/repo/rhel-7-server-ansible-2.8-rpms/
enabled=1
gpgcheck=0
[extra]
name=extra
baseurl=http://192.168.7.1:8080/repo/rhel-7-server-extras-rpms/
enabled=1
gpgcheck=0
[ose]
name=ose
baseurl=http://192.168.7.1:8080/repo/rhel-7-server-ose-4.2-rpms/
enabled=1
gpgcheck=0
[tmp]
name=tmp
baseurl=http://192.168.7.1:8080/repo/tmp/
enabled=1
gpgcheck=0

 

  • 先安装openshift-clients-4.2.0和openshift-hyperkube-4.2.0版本(因为同步的repository只有4.2.1,会报错)

在同步repository的过程中会发现没有4.2.0这个版本，只有4.2.1版本，这时候通过命令将这两个包所需要的所有rpm拉到本地。(这一步坑得比较惨)

repotrack -p ./tmp/  openshift-hyperkube-4.2.0
repotrack -p ./tmp/  openshift-clients-4.2.0
yum install -y openshift-clients-4.2.0 openshift-hyperkube-4.2.0

安装完成后为了避免和其他包的冲突把tmp的enabled修改为0。

  • 修改/etc/hosts,把镜像仓库地址加入,因为nameserver解析不了vm.redhat.com和registry.redhat.com,所以干脆直接写在/etc/hosts上。
[root@worker-3 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.7.1   vm.redhat.ren
192.168.7.1   registry.redhat.ren

 

3.Helper节点

  • 修改named server能够解析woker-3.ocp4.redhat.ren
  • 修改zonefile.db
[root@helper named]# cat /var/named/zonefile.db 
$TTL 1W
@    IN    SOA    ns1.ocp4.redhat.ren.    root (
            2019102900    ; serial
            3H        ; refresh (3 hours)
            30M        ; retry (30 minutes)
            2W        ; expiry (2 weeks)
            1W )        ; minimum (1 week)
    IN    NS    ns1.ocp4.redhat.ren.
    IN    MX 10    smtp.ocp4.redhat.ren.
;
; 
ns1    IN    A    192.168.7.11
smtp    IN    A    192.168.7.11
;
helper    IN    A    192.168.7.11
helper    IN    A    192.168.7.11
;
; The api points to the IP of your load balancer
api        IN    A    192.168.7.11
api-int        IN    A    192.168.7.11
;
; The wildcard also points to the load balancer
*.apps        IN    A    192.168.7.11
;
; Create entry for the bootstrap host
bootstrap    IN    A    192.168.7.12
;
; Create entries for the master hosts
master-0        IN    A    192.168.7.13
master-1        IN    A    192.168.7.14
master-2        IN    A    192.168.7.15
;
; Create entries for the worker hosts
worker-0        IN    A    192.168.7.16
worker-1        IN    A    192.168.7.17
worker-2        IN    A    192.168.7.18
worker-3        IN    A    192.168.7.19
;
; The ETCd cluster lives on the masters...so point these to the IP of the masters
etcd-0    IN    A    192.168.7.13
etcd-1    IN    A    192.168.7.14
etcd-2    IN    A    192.168.7.15
;
; The SRV records are IMPORTANT....make sure you get these right...note the trailing dot at the end...
_etcd-server-ssl._tcp    IN    SRV    0 10 2380 etcd-0.ocp4.redhat.ren.
_etcd-server-ssl._tcp    IN    SRV    0 10 2380 etcd-1.ocp4.redhat.ren.
_etcd-server-ssl._tcp    IN    SRV    0 10 2380 etcd-2.ocp4.redhat.ren.
;
;EOF
  • 修改反向解析文件reverse.db
[root@helper named]# cat /var/named/reverse.db 
$TTL 1W
@    IN    SOA    ns1.ocp4.redhat.ren.    root (
            2019102900    ; serial
            3H        ; refresh (3 hours)
            30M        ; retry (30 minutes)
            2W        ; expiry (2 weeks)
            1W )        ; minimum (1 week)
    IN    NS    ns1.ocp4.redhat.ren.
;
; syntax is "last octet" and the host must have fqdn with trailing dot
13    IN    PTR    master-0.ocp4.redhat.ren.
14    IN    PTR    master-1.ocp4.redhat.ren.
15    IN    PTR    master-2.ocp4.redhat.ren.
;
12    IN    PTR    bootstrap.ocp4.redhat.ren.
;
11    IN    PTR    api.ocp4.redhat.ren.
11    IN    PTR    api-int.ocp4.redhat.ren.
;
16    IN    PTR    worker-0.ocp4.redhat.ren.
17    IN    PTR    worker-1.ocp4.redhat.ren.
18    IN    PTR    worker-2.ocp4.redhat.ren.
19    IN    PTR    worker-3.ocp4.redhat.ren.
;
;EOF
  • 重启服务
systemctl start named
systemctl status named

通过/usr/local/bin/helpernodecheck或者ping来进行验证解析正确。

 

  • ssh-copy-id root@worker-3.ocp4.redhat.ren

 

  • 更新或者下载yum源
# subscription-manager repos     --enable="rhel-7-server-rpms"     --enable="rhel-7-server-extras-rpms"     --enable="rhel-7-server-ansible-2.8-rpms"     --enable="rhel-7-server-ose-4.2-rpms

 

  • 安装软件
yum install openshift-ansible openshift-clients jq

 

  • Ansible Inventory file和运行安装
[root@helper openshift-ansible]# cat /etc/ansible/hosts
[all:vars]
ansible_user=root 
#ansible_become=True 

openshift_kubeconfig_path="/root/ocp4/auth/kubeconfig" 

[new_workers] 
worker-3.ocp4.redhat.ren
cd /usr/share/ansible/openshift-ansible
ansible-playbook -i /etc/ansible/hosts playbooks/scaleup.yml

 

TASK [openshift_node : Approve node CSR] ***********************************************************************************************************************************************
Thursday 31 October 2019  20:41:32 +0800 (0:00:00.472)       0:01:50.089 ****** 
FAILED - RETRYING: Approve node CSR (6 retries left).
FAILED - RETRYING: Approve node CSR (5 retries left).
FAILED - RETRYING: Approve node CSR (4 retries left).
changed: [worker-3.ocp4.redhat.ren -> localhost] => (item=worker-3.ocp4.redhat.ren)

TASK [openshift_node : Wait for nodes to report ready] *********************************************************************************************************************************
Thursday 31 October 2019  20:41:48 +0800 (0:00:15.940)       0:02:06.030 ****** 
FAILED - RETRYING: Wait for nodes to report ready (36 retries left).
FAILED - RETRYING: Wait for nodes to report ready (35 retries left).
FAILED - RETRYING: Wait for nodes to report ready (34 retries left).
FAILED - RETRYING: Wait for nodes to report ready (33 retries left).
FAILED - RETRYING: Wait for nodes to report ready (32 retries left).
FAILED - RETRYING: Wait for nodes to report ready (31 retries left).
FAILED - RETRYING: Wait for nodes to report ready (30 retries left).
FAILED - RETRYING: Wait for nodes to report ready (29 retries left).
FAILED - RETRYING: Wait for nodes to report ready (28 retries left).
changed: [worker-3.ocp4.redhat.ren -> localhost] => (item=worker-3.ocp4.redhat.ren)

PLAY RECAP *****************************************************************************************************************************************************************************
localhost                  : ok=1    changed=1    unreachable=0    failed=0    skipped=3    rescued=0    ignored=0   
worker-3.ocp4.redhat.ren   : ok=39   changed=29   unreachable=0    failed=0    skipped=4    rescued=0    ignored=0   

Thursday 31 October 2019  20:42:35 +0800 (0:00:46.976)       0:02:53.007 ****** 
=============================================================================== 
openshift_node : Wait for nodes to report ready -------------------------------------------------------------------------------------------------------------------------------- 46.98s
openshift_node : Install openshift support packages ---------------------------------------------------------------------------------------------------------------------------- 30.90s
openshift_node : Install openshift packages ------------------------------------------------------------------------------------------------------------------------------------ 30.44s
openshift_node : Reboot the host and wait for it to come back ------------------------------------------------------------------------------------------------------------------ 20.51s
openshift_node : Approve node CSR ---------------------------------------------------------------------------------------------------------------------------------------------- 15.94s
openshift_node : Pull release image --------------------------------------------------------------------------------------------------------------------------------------------- 9.34s
openshift_node : Pull MCD image ------------------------------------------------------------------------------------------------------------------------------------------------- 4.19s
openshift_node : Apply ignition manifest ---------------------------------------------------------------------------------------------------------------------------------------- 1.65s
openshift_node : Get machine controller daemon image from release image --------------------------------------------------------------------------------------------------------- 1.08s
openshift_node : Get cluster nodes ---------------------------------------------------------------------------------------------------------------------------------------------- 0.87s
openshift_node : Setting sebool container_manage_cgroup ------------------------------------------------------------------------------------------------------------------------- 0.82s
openshift_node : Update CA trust ------------------------------------------------------------------------------------------------------------------------------------------------ 0.78s
openshift_node : Check for cluster no proxy ------------------------------------------------------------------------------------------------------------------------------------- 0.73s
openshift_node : Write /etc/containers/registries.conf -------------------------------------------------------------------------------------------------------------------------- 0.68s
openshift_node : Check for cluster http proxy ----------------------------------------------------------------------------------------------------------------------------------- 0.67s
openshift_node : Check for cluster https proxy ---------------------------------------------------------------------------------------------------------------------------------- 0.66s
openshift_node : Wait for bootstrap endpoint to show up ------------------------------------------------------------------------------------------------------------------------- 0.63s
openshift_node : Enable the CRI-O service --------------------------------------------------------------------------------------------------------------------------------------- 0.59s
openshift_node : Create CNI dirs for crio --------------------------------------------------------------------------------------------------------------------------------------- 0.54s
openshift_node : Approve node-bootstrapper CSR ---------------------------------------------------------------------------------------------------------------------------------- 0.47s
[root@helper openshift-ansible]#

查看csr

[root@helper auth]# oc get csr
NAME        AGE     REQUESTOR                                                                   CONDITION
csr-4hhc9   3m47s   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Approved,Issued
csr-bn8ww   3m33s   system:node:worker-3.ocp4.redhat.ren                                        Approved,Issued


[root@helper auth]# oc get nodes
NAME                       STATUS   ROLES    AGE    VERSION
master-0.ocp4.redhat.ren   Ready    master   34h    v1.14.6+c07e432da
master-1.ocp4.redhat.ren   Ready    master   34h    v1.14.6+c07e432da
master-2.ocp4.redhat.ren   Ready    master   34h    v1.14.6+c07e432da
worker-0.ocp4.redhat.ren   Ready    worker   34h    v1.14.6+c07e432da
worker-1.ocp4.redhat.ren   Ready    worker   34h    v1.14.6+c07e432da
worker-2.ocp4.redhat.ren   Ready    worker   34h    v1.14.6+c07e432da
worker-3.ocp4.redhat.ren   Ready    worker   118s   v1.14.6+c07e432da

安装过程中会把新加入的节点重启一次，启动完成后发送csr请求以及自动审批，然后加入集群中。

 

以上是关于OpenShift 4.2 添加RHEL节点的主要内容，如果未能解决你的问题，请参考以下文章

AWS RHEL 7快速安装配置OpenShift

OpenShift 4.2 Service Mesh

OpenShift 4.2环境离线部署Operatorhub

OpenShift 4.2 离线安装补充记录

OpenShift 4.1 探索(持续更新)

Openshift 和Harbor的集成