alertmanager 钉钉告警配置

Posted 2023-05-10

参考技术A

集成了这个项目，用于钉钉机器人推送告警信息
https://github.com/timonwong/prometheus-webhook-dingtalk

prometheus-webhook-dingtalk部署

alert配置

cat alertmanager.yml

参考博客
https://theo.im/blog/2017/10/16/release-prometheus-alertmanager-webhook-for-dingtalk/

Alertmanager配置短信告警

Prometheus是以operator方式部署
这里仅仅提供一个思路，万变不离其宗。
使用短信告警之前需要自己购买短信服务，然后定义好短信模板，一般都有现成的sdk，自己简单包装一下就可以使用了。

思路：通过自定义webhook的方式进行发送。
我简单写了一个webhook，项目地址：https://github.com/cool-ops/prometheus-alert-sms.git

部署

1、下载代码

git clone https://github.com/cool-ops/prometheus-alert-sms.git

2、编译代码

cd prometheus-alert-sms/
sh build.sh

3、打包镜像

docker build -t registry.cn-hangzhou.aliyuncs.com/rookieops/prometheus-alert-sms:v0.0.7 .

注：镜像地址更换成自己的仓库地址
4、推送镜像到镜像仓库

docker push registry.cn-hangzhou.aliyuncs.com/rookieops/prometheus-alert-sms:v0.0.7

5、修改项目目录下的prometheus-alert-sms.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: sms-conf
  namespace: monitoring
data:
  sms.yaml: |
    adapter:
      adapter_name: "RongLianYun"
    RongLianYun:
      baseUrl : "https://app.cloopen.com:8883"
      accountSid : "xxxxxx"
      appToken   : "xxxxxx"
      appId      : "xxxxx"
      templateId : "xxx"
      phones : ["11111111111","22222222222"]

    AliYun:
      aliRegion: "cn-hangzhou"
      accessKeyId: "xxxx"
      accessSecret: "xxxx"
      phoneNumbers: "11111111111,22222222222"
      signName: "xxxx"
      templateCode: "xxxx"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: prometheus-alert-sms
  namespace: monitoring
spec:
  replicas: 1
  selector:
    matchLabels:
      app: prometheus-alert-sms
  template:
    metadata:
      labels:
        app: prometheus-alert-sms
    spec:
      containers:
        - name: prometheus-alert-sms
          image: registry.cn-hangzhou.aliyuncs.com/rookieops/prometheus-alert-sms:v0.0.7
          imagePullPolicy: IfNotPresent
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthCheck
              port: tcp-9000
              scheme: HTTP
            initialDelaySeconds: 30
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 2
          readinessProbe:
              failureThreshold: 3
              httpGet:
                path: /healthCheck
                port: tcp-9000
                scheme: HTTP
              initialDelaySeconds: 30
              periodSeconds: 10
              successThreshold: 1
              timeoutSeconds: 2
          env:
            - name: CONFIG_PATH
              value: /app/conf/sms.yaml
          ports:
            - name: app-port
              containerPort: 9000
              protocol: TCP
          resources:
            limits:
              cpu: 500m
              memory: 1Gi
            requests:
              cpu: 500m
              memory: 1Gi
          volumeMounts:
            - name: sms-conf
              mountPath: /app/conf/sms.yaml
              subPath: sms.yaml
      volumes:
        - name: sms-conf
          configMap:
            name: sms-conf
---
apiVersion: v1
kind: Service
metadata:
  name: prometheus-alter-sms
  namespace: monitoring
spec:
  selector:
    app: prometheus-alert-sms
  ports:
    - name: app-port
      port: 9000
      targetPort: 9000
      protocol: TCP

到自己购买的短信服务获取对应的信息。
7、部署yaml文件

kubectl apply -f prometheus-alert-sms.yaml

8、修改alertmanager的报警媒介

 ......
      - receiver: sms 
        group_wait: 10s
        match:
          filesystem: node
    receivers:
    - name: ‘sms‘
      webhook_configs:
      - url: "http://prometheus-alter-sms.monitoring.svc:9000"
        send_resolved: true
......

然后如果有报警就可以正常接受到报警了。

PS：因为这个是按照我自己的短信模板进行开发的，可能并不适合你，你可以做相应的修改。