JenkinsJenkins在CentOS和k8s(minikube)上的部署记录
Posted 小雨青年
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了JenkinsJenkins在CentOS和k8s(minikube)上的部署记录相关的知识，希望对你有一定的参考价值。
目录
一、版本说明
-
CentOS Linux 7
-
Jenkins 2.332.2
-
minikube version: v1.25.2
-
MacOS 12.3.1
二、部署步骤
sudo wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo --no-check-certificate
这里注意，pkg.jenkins.io的证书过期了，我们要加上--no-check-certificate
[xiaoyu@localhost ~]$ sudo wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo --no-check-certificate
--2022-04-22 19:54:23-- https://pkg.jenkins.io/redhat-stable/jenkins.repo
正在解析主机 pkg.jenkins.io (pkg.jenkins.io)... 151.101.74.133, 2a04:4e42:1a::645
正在连接 pkg.jenkins.io (pkg.jenkins.io)|151.101.74.133|:443... 已连接。
警告: 无法验证 pkg.jenkins.io 的由 “/C=US/O=Let's Encrypt/CN=R3” 颁发的证书:
颁发的证书已经过期。
已发出 HTTP 请求，正在等待回应... 200 OK
长度：85
正在保存至: “/etc/yum.repos.d/jenkins.repo”
100%[=========================================================================================>] 85 --.-K/s 用时 0s
2022-04-22 19:54:24 (6.71 MB/s) - 已保存 “/etc/yum.repos.d/jenkins.repo” [85/85])
sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key
sudo yum install fontconfig java-11-openjdk
sudo yum install jenkins
sudo systemctl daemon-reload
上面三条没有什么问题，注意的是给root权限即可。
三、启动
以下命令分别为
- 设置jenkins开机启动
- 启动jenkins服务
- 查看jenkins服务状态
[xiaoyu@localhost ~]$ sudo systemctl enable jenkins
Created symlink from /etc/systemd/system/multi-user.target.wants/jenkins.service to /usr/lib/systemd/system/jenkins.service.
[xiaoyu@localhost ~]$ sudo systemctl start jenkins
[xiaoyu@localhost ~]$ sudo systemctl status jenkins
● jenkins.service - Jenkins Continuous Integration Server
Loaded: loaded (/usr/lib/systemd/system/jenkins.service; enabled; vendor preset: disabled)
Active: active (running) since 五 2022-04-22 20:19:45 CST; 10s ago
Main PID: 20521 (java)
Tasks: 42
CGroup: /system.slice/jenkins.service
└─20521 /usr/bin/java -Djava.awt.headless=true -jar /usr/share/java/jenkins.war --webroot=%C/jenkins/war --httpPort=8...
4月 22 20:19:28 localhost.localdomain jenkins[20521]: This may also be found at: /var/lib/jenkins/secrets/initialAdminPassword
4月 22 20:19:28 localhost.localdomain jenkins[20521]: *************************************************************
4月 22 20:19:28 localhost.localdomain jenkins[20521]: *************************************************************
4月 22 20:19:28 localhost.localdomain jenkins[20521]: *************************************************************
4月 22 20:19:45 localhost.localdomain jenkins[20521]: 2022-04-22 12:19:45.012+0000 [id=43] INFO h.m.Download...aller
4月 22 20:19:45 localhost.localdomain jenkins[20521]: 2022-04-22 12:19:45.012+0000 [id=43] INFO hudson.util....pt #1
4月 22 20:19:45 localhost.localdomain jenkins[20521]: 2022-04-22 12:19:45.014+0000 [id=43] INFO hudson.model...23 ms
4月 22 20:19:45 localhost.localdomain jenkins[20521]: 2022-04-22 12:19:45.867+0000 [id=28] INFO jenkins.Init...ation
4月 22 20:19:45 localhost.localdomain jenkins[20521]: 2022-04-22 12:19:45.884+0000 [id=20] INFO hudson.lifec...nning
4月 22 20:19:45 localhost.localdomain systemd[1]: Started Jenkins Continuous Integration Server.
Hint: Some lines were ellipsized, use -l to show in full.
通过增加-l查看完整信息，我们可以看到jenkins用的是8080端口，初始密码在/var/lib/jenkins/secrets/initialAdminPassword中，需要root权限查看。
[xiaoyu@localhost ~]$ sudo systemctl status jenkins -l
● jenkins.service - Jenkins Continuous Integration Server
Loaded: loaded (/usr/lib/systemd/system/jenkins.service; enabled; vendor preset: disabled)
Active: active (running) since 五 2022-04-22 20:19:45 CST; 2min 44s ago
Main PID: 20521 (java)
Tasks: 35
CGroup: /system.slice/jenkins.service
└─20521 /usr/bin/java -Djava.awt.headless=true -jar /usr/share/java/jenkins.war --webroot=%C/jenkins/war --httpPort=8080
4月 22 20:19:28 localhost.localdomain jenkins[20521]: This may also be found at: /var/lib/jenkins/secrets/initialAdminPassword
4月 22 20:19:28 localhost.localdomain jenkins[20521]: *************************************************************
4月 22 20:19:28 localhost.localdomain jenkins[20521]: *************************************************************
4月 22 20:19:28 localhost.localdomain jenkins[20521]: *************************************************************
4月 22 20:19:45 localhost.localdomain jenkins[20521]: 2022-04-22 12:19:45.012+0000 [id=43] INFO h.m.DownloadService$Downloadable#load: Obtained the updated data file for hudson.tasks.Maven.MavenInstaller
4月 22 20:19:45 localhost.localdomain jenkins[20521]: 2022-04-22 12:19:45.012+0000 [id=43] INFO hudson.util.Retrier#start: Performed the action check updates server successfully at the attempt #1
4月 22 20:19:45 localhost.localdomain jenkins[20521]: 2022-04-22 12:19:45.014+0000 [id=43] INFO hudson.model.AsyncPeriodicWork#lambda$doRun$1: Finished Download metadata. 16,623 ms
4月 22 20:19:45 localhost.localdomain jenkins[20521]: 2022-04-22 12:19:45.867+0000 [id=28] INFO jenkins.InitReactorRunner$1#onAttained: Completed initialization
4月 22 20:19:45 localhost.localdomain jenkins[20521]: 2022-04-22 12:19:45.884+0000 [id=20] INFO hudson.lifecycle.Lifecycle#onReady: Jenkins is fully up and running
4月 22 20:19:45 localhost.localdomain systemd[1]: Started Jenkins Continuous Integration Server.
[xiaoyu@localhost ~]$ sudo cat /var/lib/jenkins/secrets/initialAdminPassword
9e87859e96cc49b39276d6a63f80df1b
云主机注意需要放行8080端口，不然无法访问。
四、初始配置
浏览器访问服务器的8080端口，即可访问jenkins。
直接选择推荐的插件即可。
创建第一个用户。
下一步是配置地址，建议做一层nginx，生产环境使用域名解析，我这里用的本地虚拟机就这样了。
到此，jenkins安装完成。
五、k8s下的安装
本文采用MacOS下本地安装，基于minikebe。
启动minikube
(base) xiaoyu@localhost ~ % minikube start
😄 Darwin 12.3.1 上的 minikube v1.25.2
✨ 根据现有的配置文件使用 hyperkit 驱动程序
👍 Starting control plane node minikube in cluster minikube
🏃 Updating the running hyperkit "minikube" VM ...
❗ This VM is having trouble accessing https://k8s.gcr.io
💡 To pull new external images, you may need to configure a proxy: https://minikube.sigs.k8s.io/docs/reference/networking/proxy/
🐳 正在 Docker 20.10.12 中准备 Kubernetes v1.23.3…
▪ kubelet.housekeeping-interval=5m
🔎 Verifying Kubernetes components...
▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
▪ Using image kubernetesui/metrics-scraper:v1.0.7
▪ Using image kubernetesui/dashboard:v2.3.1
🌟 Enabled addons: default-storageclass, storage-provisioner, dashboard
❗ /usr/local/bin/kubectl is version 1.21.2, which may have incompatibilites with Kubernetes 1.23.3.
▪ Want kubectl v1.23.3? Try 'minikube kubectl -- get pods -A'
🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
创建命名空间
(base) xiaoyu@localhost ~ % kubectl create namespace jenkins
namespace/jenkins created
确定命名空间创建完成。
通过Helm安装jenkins
Helm 是 Kubernetes 的包管理器，可以简化jenkins的安装。
brew install helm
helm repo add jenkinsci https://charts.jenkins.iohelm repo update可以找到jenkinss包(base) xiaoyu@localhost ~ % helm search repo jenkinsciNAME CHART VERSION APP VERSION DESCRIPTION jenkinsci/jenkins 3.12.0 2.332.2 Jenkins - Build great things at any scale!
创建持久卷存储数据
我们采用持久卷的方式挂载数据，防止每次minikube重启的时候丢失数据。
以下内容存放在jenkins-volume.yaml中，注意，hostPath下path为绝对路径，需要根据实际情况更改。
apiVersion: v1
kind: PersistentVolume
metadata:
name: jenkins-pv
namespace: jenkins
spec:
storageClassName: jenkins-pv
accessModes:
- ReadWriteOnce
capacity:
storage: 20Gi
persistentVolumeReclaimPolicy: Retain
hostPath:
path: /Users/xiaoyu/develop/jenkins-pv/data/
在文件当前目录执行kubectl apply -f jenkins-volume.yaml。提示已经创建。
(base) xiaoyu@localhost jenkinss-pv % kubectl apply -f jenkins-volume.yamlpersistentvolume/jenkins-pv created
配置权限
(base) xiaoyu@localhost ~ % minikube ssh _ _ _ _ ( ) ( ) ___ ___ (_) ___ (_)| |/') _ _ | |_ __ /' _ ` _ `\\| |/' _ `\\| || , < ( ) ( )| '_`\\ /'__`\\| ( ) ( ) || || ( ) || || |\\`\\ | (_) || |_) )( ___/(_) (_) (_)(_)(_) (_)(_)(_) (_)`\\___/'(_,__/'`\\____)$ sudo chown -R 1000:1000 /Users/xiaoyu/develop/jenkinss-pv/data
创建jenkins用户
为了保证安全性，一般我们为某个单独的服务配置单独的管理员用户，用来处理单独的工作。在k8s中，默认配置一个和命名空间一致的用户作为这个命名空间内操作的授权用户。
基于文件即配置的思想，我们依然创建一个文件，名称是jenkins-sa.yaml,当然名称不是固定的，这样比较好区分。并将下面内容写入文件内，我们可以看到k8s是如何配置用户并分配权限的。
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: jenkins
namespace: jenkins
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: jenkins
rules:
- apiGroups:
- '*'
resources:
- statefulsets
- services
- replicationcontrollers
- replicasets
- podtemplates
- podsecuritypolicies
- pods
- pods/log
- pods/exec
- podpreset
- poddisruptionbudget
- persistentvolumes
- persistentvolumeclaims
- jobs
- endpoints
- deployments
- deployments/scale
- daemonsets
- cronjobs
- configmaps
- namespaces
- events
- secrets
verbs:
- create
- get
- watch
- delete
- list
- patch
- update
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- list
- watch
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: jenkins
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: jenkins
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:serviceaccounts:jenkins
(base) xiaoyu@localhost jenkinss-pv % kubectl apply -f jenkins-sa.yamlserviceaccount/jenkins createdclusterrole.rbac.authorization.k8s.io/jenkins createdclusterrolebinding.rbac.authorization.k8s.io/jenkins created
以上操作分别完成了创建用户、创建权限、绑定权限操作。
安装 Jenkins
和上面操作一样，新建文件jenkins-values.yaml。
# Default values for jenkins.
# This is a YAML-formatted file.
# Declare name/value pairs to be passed into your templates.
# name: value
## Overrides for generated resource names
# See templates/_helpers.tpl
# nameOverride:
# fullnameOverride:
# namespaceOverride:
# For FQDN resolving of the controller service. Change this value to match your existing configuration.
# ref: https://github.com/kubernetes/dns/blob/master/docs/specification.md
clusterZone: "cluster.local"
renderHelmLabels: true
controller:
# Used for label app.kubernetes.io/component
componentName: "jenkins-controller"
image: "jenkins/jenkins"
# tag: "2.332.2-jdk11"
tagLabel: jdk11
imagePullPolicy: "Always"
imagePullSecretName:
# Optionally configure lifetime for controller-container
lifecycle:
# postStart:
# exec:
# command:
# - "uname"
# - "-a"
disableRememberMe: false
numExecutors: 0
# configures the executor mode of the Jenkins node. Possible values are: NORMAL or EXCLUSIVE
executorMode: "NORMAL"
# This is ignored if enableRawhtmlMarkupFormatter is true
markupFormatter: plainText
customJenkinsLabels: []
# The default configuration uses this secret to configure an admin user
# If you don't need that user or use a different security realm then you can disable it
adminSecret: true
hostNetworking: false
# When enabling LDAP or another non-Jenkins identity source, the built-in admin account will no longer exist.
# If you disable the non-Jenkins identity store and instead use the Jenkins internal one,
# you should revert controller.adminUser to your preferred admin user:
adminUser: "admin"
# adminPassword: <defaults to random>
admin:
existingSecret: ""
userKey: xiaoyuqingnian
passwordKey: xiaoyuqingnian
# This values should not be changed unless you use your custom image of jenkins or any devired from. If you want to use
# Cloudbees Jenkins Distribution docker, you should set jenkinsHome: "/var/cloudbees-jenkins-distribution"
jenkinsHome: "/var/jenkins_home"
# This values should not be changed unless you use your custom image of jenkins or any devired from. If you want to use
# Cloudbees Jenkins Distribution docker, you should set jenkinsRef: "/usr/share/cloudbees-jenkins-distribution/ref"
jenkinsRef: "/usr/share/jenkins/ref"
# Path to the jenkins war file which is used by jenkins-plugin-cli.
jenkinsWar: "/usr/share/jenkins/jenkins.war"
# Overrides the default arguments passed to the war
# overrideArgs:
# - --httpPort=8080
resources:
requests:
cpu: "50m"
memory: "256Mi"
limits:
cpu: "2000m"
memory: "4096Mi"
# Overrides the init container default values
# initContainerResources:
# requests:
# cpu: "50m"
# memory: "256Mi"
# limits:
# cpu: "2000m"
# memory: "4096Mi"
# Environment variables that get added to the init container (useful for e.g. http_proxy)
# initContainerEnv:
# - name: http_proxy
# value: "http://192.168.64.1:3128"
# containerEnv:
# - name: http_proxy
# value: "http://192.168.64.1:3128"
# Set min/max heap here if needed with:
# javaOpts: "-Xms512m -Xmx512m"
# jenkinsOpts: ""
# If you are using the ingress definitions provided by this chart via the `controller.ingress` block the configured hostname will be the ingress hostname starting with `https://` or `http://` depending on the `tls` configuration.
# The Protocol can be overwritten by specifying `controller.jenkinsUrlProtocol`.
# jenkinsUrlProtocol: "https"
# If you are not using the provided ingress you can specify `controller.jenkinsUrl` to change the url definition.
# jenkinsUrl: ""
# If you set this prefix and use ingress controller then you might want to set the ingress path below
# jenkinsUriPrefix: "/jenkins"
# Enable pod security context (must be `true` if podSecurityContextOverride, runAsUser or fsGroup are set)
usePodSecurityContext: true
# Note that `runAsUser`, `fsGroup`, and `securityContextCapabilities` are
# being deprecated and replaced by `podSecurityContextOverride`.
# Set runAsUser to 1000 to let Jenkins run as non-root user 'jenkins' which exists in 'jenkins/jenkins' docker image.
# When setting runAsUser to a different value than 0 also set fsGroup to the same value:
runAsUser: 1000
fsGroup: 1000
# If you have PodSecurityPolicies that require dropping of capabilities as suggested by CIS K8s benchmark, put them here
securityContextCapabilities:
# drop:
# - NET_RAW
# Completely overwrites the contents of the `securityContext`, ignoring the
# values provided for the deprecated fields: `runAsUser`, `fsGroup`, and
# `securityContextCapabilities`. In the case of mounting an ext4 filesystem,
# it might be desirable to use `supplementalGroups` instead of `fsGroup` in
# the `securityContext` block: https://github.com/kubernetes/kubernetes/issues/67014#issuecomment-589915496
# podSecurityContextOverride:
# runAsUser: 1000
# runAsNonRoot: true
# supplementalGroups: [1000]
# # capabilities:
# Container securityContext
containerSecurityContext:
runAsUser: 1000
runAsGroup: 1000
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
servicePort: 8080
targetPort: 8080
# For minikube, set this to NodePort, elsewhere use LoadBalancer
# Use ClusterIP if your setup includes ingress controller
serviceType: ClusterIP
# Use Local to preserve the client source IP and avoids a second hop for LoadBalancer and Nodeport type services,
# but risks potentially imbalanced traffic spreading.
serviceExternalTrafficPolicy:
# Jenkins controller service annotations
serviceAnnotations:
# Jenkins controller custom labels
statefulSetLabels:
# foo: bar
# bar: foo
# Jenkins controller service labels
serviceLabels:
# service.beta.kubernetes.io/aws-load-balancer-backend-protocol: https
# Put labels on Jenkins controller pod
podLabels:
# Used to create Ingress record (should used with ServiceType: ClusterIP)
# nodePort: <to set explicitly, choose port between 30000-32767
# Enable Kubernetes Startup, Liveness and Readiness Probes
# if Startup Probe is supported, enable it too
# ~ 2 minutes to allow Jenkins to restart when upgrading plugins. Set ReadinessTimeout to be shorter than LivenessTimeout.
healthProbes: true
probes:
startupProbe:
httpGet:
path: ' default "" .Values.controller.jenkinsUriPrefix /login'
port: http
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 12
livenessProbe:
failureThreshold: 5
httpGet:
path: ' default "" .Values.controller.jenkinsUriPrefix /login'
port: http
periodSeconds: 10
timeoutSeconds: 5
# If Startup Probe is not supported on your Kubernetes cluster, you might want to use "initialDelaySeconds" instead.
# It delays the initial liveness probe while Jenkins is starting
# initialDelaySeconds: 60
readinessProbe:
failureThreshold: 3
httpGet:
path: ' default "" .Values.controller.jenkinsUriPrefix /login'
port: http
periodSeconds: 10
timeoutSeconds: 5
# If Startup Probe is not supported on your Kubernetes cluster, you might want to use "initialDelaySeconds" instead.
# It delays the initial readyness probe while Jenkins is starting
# initialDelaySeconds: 60
# PodDisruptionBudget config
podDisruptionBudget:
enabled: false
# For Kubernetes v1.5+, use 'policy/v1beta1'
# For Kubernetes v1.21+, use 'policy/v1'
apiVersion: "policy/v1beta1"
annotations:
labels:
# maxUnavailable: "0"
agentListenerEnabled: true
agentListenerPort: 50000
agentListenerHostPort:
agentListenerNodePort:
agentListenerExternalTrafficPolicy:
agentListenerLoadBalancerSourceRanges:
- 0.0.0.0/0
disabledAgentProtocols:
- JNLP-connect
- JNLP2-connect
csrf:
defaultCrumbIssuer:
enabled: true
proxyCompatability: true
# Kubernetes service type for the JNLP agent service
# agentListenerServiceType is the Kubernetes Service type for the JNLP agent service,
# either 'LoadBalancer', 'NodePort', or 'ClusterIP'
# Note if you set this to 'LoadBalancer', you *must* define annotations to secure it. By default
# this will be an external load balancer and allowing inbound 0.0.0.0/0, a HUGE
# security risk: https://github.com/kubernetes/charts/issues/1341
agentListenerServiceType: "ClusterIP"
# Optionally assign an IP to the LoadBalancer agentListenerService LoadBalancer
# GKE users: only regional static IPs will work for Service Load balancer.
agentListenerLoadBalancerIP:
agentListenerServiceAnnotations:
# Example of 'LoadBalancer' type of agent listener with annotations securing it
# agentListenerServiceType: LoadBalancer
# agentListenerServiceAnnotations:
# service.beta.kubernetes.io/aws-load-balancer-internal: "True"
# service.beta.kubernetes.io/load-balancer-source-ranges: "172.0.0.0/8, 10.0.0.0/8"
# LoadBalancerSourcesRange is a list of allowed CIDR values, which are combined with ServicePort to
# set allowed inbound rules on the security group assigned to the controller load balancer
loadBalancerSourceRanges:
- 0.0.0.0/0
# Optionally assign a known public LB IP
# loadBalancerIP: 1.2.3.4
# Optionally configure a JMX port
# requires additional javaOpts, ie
# javaOpts: >
# -Dcom.sun.management.jmxremote.port=4000
# -Dcom.sun.management.jmxremote.authenticate=false
# -Dcom.sun.management.jmxremote.ssl=false
# jmxPort: 4000
# Optionally configure other ports to expose in the controller container
extraPorts: []
# - name: BuildInfoProxy
# port: 9000
# List of plugins to be install during Jenkins controller start
installPlugins:
- kubernetes:1.31.3
- workflow-aggregator:2.6
- git:4.10.2
- configuration-as-code:1414.v878271fc496f
# Set to false to download the minimum required version of all dependencies.
installLatestPlugins: true
# Set to true to download latest dependencies of any plugin that is requested to have the latest version.
installLatestSpecifiedPlugins: false
# List of plugins to install in addition to those listed in controller.installPlugins
additionalPlugins: []
# Enable to initialize the Jenkins controller only once on initial installation.
# Without this, whenever the controller gets restarted (Evicted, etc.) it will fetch plugin updates which has the potential to cause breakage.
# Note that for this to work, `persistence.enabled` needs to be set to `true`
initializeOnce: false
# Enable to always override the installed plugins with the values of 'controller.installPlugins' on upgrade or redeployment.
# overwritePlugins: true
# Configures if plugins bundled with `controller.image` should be overwritten with the values of 'controller.installPlugins' on upgrade or redeployment.
overwritePluginsFromImage: true
# Enable HTML parsing using OWASP Markup Formatter Plugin (antisamy-markup-formatter), useful with ghprb plugin.
# The plugin is not installed by default, please update controller.installPlugins.
enableRawHtmlMarkupFormatter: false
# Used to approve a list of groovy functions in pipelines used the script-security plugin. Can be viewed under /scriptApproval
scriptApproval: []
# - "method groovy.json.JsonSlurperClassic parseText java.lang.String"
# - "new groovy.json.JsonSlurperClassic"
# List of groovy init scripts to be executed during Jenkins controller start
initScripts<以上是关于JenkinsJenkins在CentOS和k8s(minikube)上的部署记录的主要内容，如果未能解决你的问题，请参考以下文章