如何安排执行 kubectl 命令的 cronjob？
Posted
技术标签:
【中文标题】如何安排执行 kubectl 命令的 cronjob？【英文标题】：How to schedule a cronjob which executes a kubectl command? 【发布时间】：2019-07-21 02:12:40 【问题描述】：我想每 5 分钟运行一次以下 kubectl 命令：
kubectl patch deployment runners -p '"spec":"template":"spec":"containers":["name":"jp-runner","env":["name":"START_TIME","value":"'$(date +%s)'"]]' -n jp-test
为此，我创建了一个 cronjob，如下所示：
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: hello
spec:
schedule: "*/5 * * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: hello
image: busybox
args:
- /bin/sh
- -c
- kubectl patch deployment runners -p '"spec":"template":"spec":"containers":["name":"jp-runner","env":["name":"START_TIME","value":"'$(date +%s)'"]]' -n jp-test
restartPolicy: OnFailure
但它无法启动容器，显示消息：
Back-off restarting failed container
错误代码为 127：
State: Terminated
Reason: Error
Exit Code: 127
根据我的检查，错误代码 127 表示该命令不存在。我如何将 kubectl 命令作为 cron 作业运行？我错过了什么吗？
注意：我已经发布了一个类似的问题 (Scheduled restart of Kubernetes pod without downtime)，但这更多是将主要部署本身作为 cronjob，这里我正在尝试运行 kubectl 命令（它会重新启动）使用 CronJob - 所以我认为单独发布会更好
kubectl 描述 cronjob hello -n jp-test:
Name: hello
Namespace: jp-test
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration="apiVersion":"batch/v1beta1","kind":"CronJob","metadata":"annotations":,"name":"hello","namespace":"jp-test","spec":"jobTemplate":"spec":"templ...
Schedule: */5 * * * *
Concurrency Policy: Allow
Suspend: False
Starting Deadline Seconds: <unset>
Selector: <unset>
Parallelism: <unset>
Completions: <unset>
Pod Template:
Labels: <none>
Containers:
hello:
Image: busybox
Port: <none>
Host Port: <none>
Args:
/bin/sh
-c
kubectl patch deployment runners -p '"spec":"template":"spec":"containers":["name":"jp-runner","env":["name":"START_TIME","value":"'$(date +%s)'"]]' -n jp-test
Environment: <none>
Mounts: <none>
Volumes: <none>
Last Schedule Time: Wed, 27 Feb 2019 14:10:00 +0100
Active Jobs: hello-1551273000
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulCreate 6m cronjob-controller Created job hello-1551272700
Normal SuccessfulCreate 1m cronjob-controller Created job hello-1551273000
Normal SawCompletedJob 16s cronjob-controller Saw completed job: hello-1551272700
kubectl describe job hello -v=5 -n jp-test
Name: hello-1551276000
Namespace: jp-test
Selector: controller-uid=fa009d78-3a97-11e9-ae31-ac1f6b1a0950
Labels: controller-uid=fa009d78-3a97-11e9-ae31-ac1f6b1a0950
job-name=hello-1551276000
Annotations: <none>
Controlled By: CronJob/hello
Parallelism: 1
Completions: 1
Start Time: Wed, 27 Feb 2019 15:00:02 +0100
Pods Statuses: 0 Running / 0 Succeeded / 0 Failed
Pod Template:
Labels: controller-uid=fa009d78-3a97-11e9-ae31-ac1f6b1a0950
job-name=hello-1551276000
Containers:
hello:
Image: busybox
Port: <none>
Host Port: <none>
Args:
/bin/sh
-c
kubectl patch deployment runners -p '"spec":"template":"spec":"containers":["name":"jp-runner","env":["name":"START_TIME","value":"'$(date +%s)'"]]' -n jp-test
Environment: <none>
Mounts: <none>
Volumes: <none>
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulCreate 7m job-controller Created pod: hello-1551276000-lz4dp
Normal SuccessfulDelete 1m job-controller Deleted pod: hello-1551276000-lz4dp
Warning BackoffLimitExceeded 1m (x2 over 1m) job-controller Job has reached the specified backoff limit
Name: hello-1551276300
Namespace: jp-test
Selector: controller-uid=ad52e87a-3a98-11e9-ae31-ac1f6b1a0950
Labels: controller-uid=ad52e87a-3a98-11e9-ae31-ac1f6b1a0950
job-name=hello-1551276300
Annotations: <none>
Controlled By: CronJob/hello
Parallelism: 1
Completions: 1
Start Time: Wed, 27 Feb 2019 15:05:02 +0100
Pods Statuses: 1 Running / 0 Succeeded / 0 Failed
Pod Template:
Labels: controller-uid=ad52e87a-3a98-11e9-ae31-ac1f6b1a0950
job-name=hello-1551276300
Containers:
hello:
Image: busybox
Port: <none>
Host Port: <none>
Args:
/bin/sh
-c
kubectl patch deployment runners -p '"spec":"template":"spec":"containers":["name":"jp-runner","env":["name":"START_TIME","value":"'$(date +%s)'"]]' -n jp-test
Environment: <none>
Mounts: <none>
Volumes: <none>
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulCreate 2m job-controller Created pod: hello-1551276300-8d5df
【问题讨论】：
您好，kubectl 需要使用 pod 内的服务帐户向 apiserver 进行身份验证。可能是错误 你能提供kubectl describe job <cron_job_name> -v=5吗？
@Crou 用详细信息更新了我的问题
@Chillax，是的，但我认为工作需要积极主动。也许删除 cronjob 并再次应用它并等待作业处于活动状态。
【参考方案1】：
长话短说BusyBox 没有安装kubectl。
您可以使用 kubectl run -i --tty busybox --image=busybox -- sh 自己检查它，它会将 BusyBox pod 作为交互式 shell 运行。
我建议使用bitnami/kubectl:latest。
还请记住，您需要设置正确的RBAC，因为您将获得Error from server (Forbidden): services is forbidden
你可以这样使用：
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: jp-test
name: jp-runner
rules:
- apiGroups:
- extensions
- apps
resources:
- deployments
verbs:
- 'patch'
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: jp-runner
namespace: jp-test
subjects:
- kind: ServiceAccount
name: sa-jp-runner
namespace: jp-test
roleRef:
kind: Role
name: jp-runner
apiGroup: ""
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: sa-jp-runner
namespace: jp-test
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: hello
spec:
schedule: "*/5 * * * *"
jobTemplate:
spec:
template:
spec:
serviceAccountName: sa-jp-runner
containers:
- name: hello
image: bitnami/kubectl:latest
command:
- /bin/sh
- -c
- kubectl patch deployment runners -p '"spec":"template":"spec":"containers":["name":"jp-runner","env":["name":"START_TIME","value":"'$(date +%s)'"]]' -n jp-test
restartPolicy: OnFailure
【讨论】：
【参考方案2】：您需要制作 CronJob 的容器来下载集群配置，以便您可以针对它运行 kubectl 命令。这是一个例子：
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: drupal-cron
spec:
schedule: "*/5 * * * *"
concurrencyPolicy: Forbid
jobTemplate:
spec:
template:
spec:
containers:
- name: drupal-cron
image: juampynr/digital-ocean-cronjob:latest
env:
- name: DIGITALOCEAN_ACCESS_TOKEN
valueFrom:
secretKeyRef:
name: api
key: key
command: ["/bin/bash","-c"]
args:
- doctl kubernetes cluster kubeconfig save drupster;
POD_NAME=$(kubectl get pods -l tier=frontend -o=jsonpath='.items[0].metadata.name');
kubectl exec $POD_NAME -c drupal -- vendor/bin/drush core:cron;
restartPolicy: OnFailure
我在另一个线程中发布了一个描述我是如何做到这一点的答案：https://***.com/a/62321138/1120652
【讨论】：
以上是关于如何安排执行 kubectl 命令的 cronjob？的主要内容，如果未能解决你的问题，请参考以下文章