k8s篇备份之velero
Posted 运维技术前线
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了k8s篇备份之velero相关的知识，希望对你有一定的参考价值。
velero简介
k8s备份工具之velero, velero 是一个云原生的灾难恢复和迁移工具，它本身也是开源的, 采用 Go 语言编写，可以安全的备份、恢复和迁移Kubernetes集群资源和持久卷。
与 Etcd 备份的区别
与 Etcd 备份相比，直接备份 Etcd 是将集群的全部资源备份起来。而 Velero 就是可以对 Kubernetes 集群内对象级别进行备份。除了对 Kubernetes 集群进行整体备份外，Velero 还可以通过对 Type、Namespace、Label 等对象进行分类备份或者恢复。
Velero 目前包含以下特性
支持 Kubernetes 集群数据备份和恢复
支持复制当前 Kubernetes 集群的资源到其它 Kubernetes 集群
支持复制生产环境到开发以及测试环境(不同命名空间的备份还原)
Velero 组件
Velero 组件一共分两部分，分别是服务端和客户端。服务端运行在你 Kubernetes 的集群中，客户端是一些运行在本地的命令行的工具。
Velero 支持的备份存储
AWS S3 以及兼容 S3 的存储，比如：Minio
Azure BloB 存储
Google Cloud 存储
备份常规需要的资源
将复制的Kubernetes对象的压缩文件上传到云对象存储中(aws s3等云存储)
调用云提供程序API以创建持久卷的磁盘快照（如果指定）
参考链接
https://velero.io/docs/v1.5/contributions/minio/
组件版本
aws eks 1.19
velero 1.5
minio模拟本地s3存储
安装velero命令行
curl -LJO https://github.com/vmware-tanzu/velero/releases/download/v1.5.3/velero-v1.5.3-linux-amd64.tar.gz
tar -zxvf velero-v1.5.3-linux-amd64.tar.gz
cp -rf velero-v1.5.3-linux-amd64/velero /usr/local/bin
ubuntu@ip-172-16-10-57:~/velero-v1.5.3-linux-amd64/examples$ velero version
Client:
Version: v1.5.3
Git commit: 123109a3bcac11dbb6783d2758207bac0d0817cb
<error getting server version: no matches for kind "ServerStatusRequest" in version "velero.io/v1">
安装velero服务器端
安装minio测试例子
velero命令行获取备份描述的时候，需要从外部访问，开启公网访问(简单使用nodeport)
cd velero-v1.5.3-linux-amd64
kubectl apply -f examples/minio/00-minio-deployment.yaml
安装velero服务器端
credentials-velero 配置文件
[default]
aws_access_key_id = minio
aws_secret_access_key = minio123
velero install \ --provider aws \ --plugins velero/velero-plugin-for-aws:v1.0.0 \ --bucket velero \ --secret-file ./credentials-velero \ --use-volume-snapshots=false \ --backup-location-config region=minio,s3ForcePathStyle="true",s3Url=http://minio.velero.svc:9000
kubectl patch -n velero backupstoragelocation default --type merge -p '{"spec":{"config":{"publicUrl":"http://13.231.49.246:32143"}}}'
ubuntu@ip-172-16-10-57:~$ velero install \
> --provider aws \
> --plugins velero/velero-plugin-for-aws:v1.0.0 \
> --bucket velero \
> --secret-file ./credentials-velero \
> --use-volume-snapshots=false \
> --backup-location-config region=minio,s3ForcePathStyle="true",s3Url=http://minio.velero.svc:9000
CustomResourceDefinition/backups.velero.io: attempting to create resource
CustomResourceDefinition/backups.velero.io: created
CustomResourceDefinition/backupstoragelocations.velero.io: attempting to create resource
CustomResourceDefinition/backupstoragelocations.velero.io: created
CustomResourceDefinition/deletebackuprequests.velero.io: attempting to create resource
CustomResourceDefinition/deletebackuprequests.velero.io: created
CustomResourceDefinition/downloadrequests.velero.io: attempting to create resource
CustomResourceDefinition/downloadrequests.velero.io: created
CustomResourceDefinition/podvolumebackups.velero.io: attempting to create resource
CustomResourceDefinition/podvolumebackups.velero.io: created
CustomResourceDefinition/podvolumerestores.velero.io: attempting to create resource
CustomResourceDefinition/podvolumerestores.velero.io: created
CustomResourceDefinition/resticrepositories.velero.io: attempting to create resource
CustomResourceDefinition/resticrepositories.velero.io: created
CustomResourceDefinition/restores.velero.io: attempting to create resource
CustomResourceDefinition/restores.velero.io: created
CustomResourceDefinition/schedules.velero.io: attempting to create resource
CustomResourceDefinition/schedules.velero.io: created
CustomResourceDefinition/serverstatusrequests.velero.io: attempting to create resource
CustomResourceDefinition/serverstatusrequests.velero.io: created
CustomResourceDefinition/volumesnapshotlocations.velero.io: attempting to create resource
CustomResourceDefinition/volumesnapshotlocations.velero.io: created
Waiting for resources to be ready in cluster...
Namespace/velero: attempting to create resource
Namespace/velero: already exists, proceeding
Namespace/velero: created
ClusterRoleBinding/velero: attempting to create resource
ClusterRoleBinding/velero: created
ServiceAccount/velero: attempting to create resource
ServiceAccount/velero: created
Secret/cloud-credentials: attempting to create resource
Secret/cloud-credentials: created
BackupStorageLocation/default: attempting to create resource
BackupStorageLocation/default: created
Deployment/velero: attempting to create resource
Deployment/velero: created
Velero is installed! ⛵ Use 'kubectl logs deployment/velero -n velero' to view the status.
测试还原备份
部署测试应用
kubectl apply -f examples/nginx-app/base.yaml
备份指定应用
velero backup create nginx-backup --selector app=nginx
只备份指定命名空间上的指定应用
velero backup create nginx-backup --include-namespaces nginx-example --selector app=nginx
ubuntu@ip-172-16-10-57:~/velero-v1.5.3-linux-amd64$ velero backup create nginx-backup --selector app=nginx
Backup request "nginx-backup" submitted successfully.
Run `velero backup describe nginx-backup` or `velero backup logs nginx-backup` for more details.
查看备份状态
velero backup describe nginx-backup
ubuntu@ip-172-16-10-57:~/velero-v1.5.3-linux-amd64$ velero backup describe nginx-backup
Name: nginx-backup
Namespace: velero
Labels: velero.io/storage-location=default
Annotations: velero.io/source-cluster-k8s-gitversion=v1.19.6-eks-49a6c0
velero.io/source-cluster-k8s-major-version=1
velero.io/source-cluster-k8s-minor-version=19+
Phase: Completed
Errors: 0
Warnings: 0
Namespaces:
Included: *
Excluded: <none>
Resources:
Included: *
Excluded: <none>
Cluster-scoped: auto
Label selector: app=nginx
Storage Location: default
Velero-Native Snapshot PVs: auto
TTL: 720h0m0s
Hooks: <none>
Backup Format Version: 1.1.0
Started: 2021-02-20 08:04:50 +0000 UTC
Completed: 2021-02-20 08:04:53 +0000 UTC
Expiration: 2021-03-22 08:04:50 +0000 UTC
Total items to be backed up: 6
Items backed up: 6
Velero-Native Snapshots: <none included>
还原应用
还原到原命名空间
velero restore create --from-backup nginx-backup
还原到特定备份空间
kubectl create ns nginx-restore
velero restore create nginx-backup \ --include-resources deployments \ --include-namespaces nginx-example \ --selector app=nginx \ --from-backup nginx-backup \ --namespace-mappings nginx-example:nginx-restore
ubuntu@ip-172-16-10-57:~/velero-v1.5.3-linux-amd64$ kubectl create ns nginx-restore
namespace/nginx-restore created
ubuntu@ip-172-16-10-57:~$ velero restore create nginx-backup \
> --include-resources deployments \
> --include-namespaces nginx-example \
> --selector app=nginx \
> --from-backup nginx-backup \
> --namespace-mappings nginx-example:nginx-restore
Restore request "nginx-backup" submitted successfully.
Run `velero restore describe nginx-backup` or `velero restore logs nginx-backup` for more details.
查看还原状态
velero restore describe nginx-backup
ubuntu@ip-172-16-10-57:~/velero-v1.5.3-linux-amd64$ velero restore describe nginx-backup
Name: nginx-backup
Namespace: velero
Labels: <none>
Annotations: <none>
Phase: Completed
Started: 2021-02-20 08:22:10 +0000 UTC
Completed: 2021-02-20 08:22:11 +0000 UTC
Backup: nginx-backup
Namespaces:
Included: all namespaces found in the backup
Excluded: <none>
Resources:
Included: *
Excluded: nodes, events, events.events.k8s.io, backups.velero.io, restores.velero.io, resticrepositories.velero.io
Cluster-scoped: auto
Namespace mappings: nginx-example=nginx-restore
Label selector: <none>
Restore PVs: auto
ubuntu@ip-172-16-10-57:~/velero-v1.5.3-linux-amd64$ velero restore get
NAME BACKUP STATUS STARTED COMPLETED ERRORS WARNINGS CREATED SELECTOR
nginx-backup nginx-backup Completed 2021-02-20 08:22:10 +0000 UTC 2021-02-20 08:22:11 +0000 UTC 0 0 2021-02-20 08:22:10 +0000 UTC <none>
查看应用状态
ubuntu@ip-172-16-10-57:~/velero-v1.5.3-linux-amd64$ kubectl get pods,svc -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/aws-node-nn6xw 1/1 Running 0 3h18m
kube-system pod/aws-node-vswfd 1/1 Running 0 3h18m
kube-system pod/coredns-59847d77c8-29ss8 1/1 Running 0 3h24m
kube-system pod/coredns-59847d77c8-c7vtf 1/1 Running 0 3h24m
kube-system pod/kube-proxy-7f9ps 1/1 Running 0 3h18m
kube-system pod/kube-proxy-zx4w7 1/1 Running 0 3h18m
nginx-example pod/nginx-deployment-57d5dcb68-6q8dq 1/1 Running 0 23m
nginx-example pod/nginx-deployment-57d5dcb68-l4dv6 1/1 Running 0 23m
nginx-restore pod/nginx-deployment-57d5dcb68-6q8dq 1/1 Running 0 109s
nginx-restore pod/nginx-deployment-57d5dcb68-l4dv6 1/1 Running 0 109s
velero pod/minio-5b84955bdd-6r4mr 1/1 Running 0 32m
velero pod/minio-setup-r8rfr 0/1 Completed 0 32m
velero pod/velero-58d7945c98-2zl85 1/1 Running 0 25m
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.100.0.1 <none> 443/TCP 3h24m
kube-system service/kube-dns ClusterIP 10.100.0.10 <none> 53/UDP,53/TCP 3h24m
nginx-example service/my-nginx LoadBalancer 10.100.115.250 a7992dfca1b0746b4a23b1aa20604bc9-1115673395.ap-northeast-1.elb.amazonaws.com 80:32717/TCP 23m
nginx-restore service/my-nginx LoadBalancer 10.100.87.57 a3a9b59526a6c44fa8b58e435c3ff830-873159261.ap-northeast-1.elb.amazonaws.com 80:32414/TCP 110s
velero service/minio NodePort 10.100.170.114 <none> 9000:32143/TCP 32m
清理备份应用的资源
velero get backupvelero backup delete nginx-backup
清理备份资源，会一同清理还原产生的日志数据
ubuntu@ip-172-16-10-57:~/velero-v1.5.3-linux-amd64$ velero get backup
NAME STATUS ERRORS WARNINGS CREATED EXPIRES STORAGE LOCATION SELECTOR
nginx-backup Completed 0 0 2021-02-20 08:04:50 +0000 UTC 29d default app=nginx
ubuntu@ip-172-16-10-57:~/velero-v1.5.3-linux-amd64$ velero backup delete nginx-backup
Are you sure you want to continue (Y/N)? y
Request to delete backup "nginx-backup" submitted successfully.
The backup will be fully deleted after all associated data (disk snapshots, backup files, restores) are removed.
删除velero
kubectl delete namespace/velero clusterrolebinding/velerokubectl delete crds -l component=velero
minio部署文件00-minio-deployment.yaml
# Copyright 2017 the Velero contributors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
---
apiVersion: v1
kind: Namespace
metadata:
name: velero
---
apiVersion: apps/v1
kind: Deployment
metadata:
namespace: velero
name: minio
labels:
component: minio
spec:
strategy:
type: Recreate
selector:
matchLabels:
component: minio
template:
metadata:
labels:
component: minio
spec:
volumes:
- name: storage
emptyDir: {}
- name: config
emptyDir: {}
containers:
- name: minio
image: minio/minio:latest
imagePullPolicy: IfNotPresent
args:
- server
- /storage
- --config-dir=/config
env:
- name: MINIO_ACCESS_KEY
value: "minio"
- name: MINIO_SECRET_KEY
value: "minio123"
ports:
- containerPort: 9000
volumeMounts:
- name: storage
mountPath: "/storage"
- name: config
mountPath: "/config"
---
apiVersion: v1
kind: Service
metadata:
namespace: velero
name: minio
labels:
component: minio
spec:
# ClusterIP is recommended for production environments.
# Change to NodePort if needed per documentation,
# but only if you run Minio in a test/trial environment, for example with Minikube.
type: NodePort
ports:
- port: 9000
targetPort: 9000
protocol: TCP
selector:
component: minio
---
apiVersion: batch/v1
kind: Job
metadata:
namespace: velero
name: minio-setup
labels:
component: minio
spec:
template:
metadata:
name: minio-setup
spec:
restartPolicy: OnFailure
volumes:
- name: config
emptyDir: {}
containers:
- name: mc
image: minio/mc:latest
imagePullPolicy: IfNotPresent
command:
- /bin/sh
- -c
- "mc --config-dir=/config config host add velero http://minio:9000 minio minio123 && mc --config-dir=/config mb -p velero/velero"
volumeMounts:
- name: config
mountPath: "/config"
nginx测试无存储.base.yaml
# Copyright 2017 the Velero contributors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
---
apiVersion: v1
kind: Namespace
metadata:
name: nginx-example
labels:
app: nginx
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: nginx-example
labels:
app: nginx
spec:
replicas: 2
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx:1.17.6
name: nginx
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
labels:
app: nginx
name: my-nginx
namespace: nginx-example
spec:
ports:
- port: 80
targetPort: 80
selector:
app: nginx
type: LoadBalancer
以上是关于k8s篇备份之velero的主要内容，如果未能解决你的问题，请参考以下文章
详解kubernetes备份恢复利器 Velero - 深入了解Carina系列第三期