三 saltstack 任务管理和集群

Posted 2020-11-05

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了三 saltstack 任务管理和集群相关的知识，希望对你有一定的参考价值。

1 Job管理和Runner

https://www.unixhot.com/docs/saltstack/ref/modules/all/salt.modules.saltutil.html#module-salt.modules.saltutil

1 任务管理

1.1 查看正在运行的任务

[[email protected] ~]# salt ‘*‘ saltutil.running
salt-node4.damaicha.org-204:
    |_
      ----------
      arg:
      fun:
          state.highstate
      jid:
          20170318082805017980
      pid:
          52413
      ret:
      tgt:
          *
      tgt_type:
          glob
      user:
          root
test-node3.damaicha.org-203:
    |_
      ----------
      arg:
      fun:
          state.highstate
      jid:
          20170318082805017980
      pid:
          44995
      ret:
      tgt:
          *
      tgt_type:
          glob
      user:
          root
[[email protected] ~]#

1.2 停止正在运行中的任务

[[email protected] ~]# salt ‘*‘ saltutil.kill_job  20170318082805017980

2 将master上返回的结果写在数据库里。

前面使用了插件，让Minnion的数据写到数据库里。
现在让master上的数据直接写在文件里。
返回的数据在这里/var/cache/salt/master/jobs 是他自己的保存方式。

安装 mysql-python

[[email protected] ~]# yum -y install MySQL-python

修改master配置文件，在文件结尾处添加如下：
salt库里的相关的建表语句请看salt一第一部分总结笔记

mysql.host: ‘127.0.0.1‘
mysql.user: ‘salt‘
mysql.pass: ‘[email protected]‘
mysql.db: ‘salt‘
mysql.port: 3306

重启master服务，让配置文件生效。

systemctl restart salt-master

3、测试。

[[email protected] /srv]# mysql  -usalt [email protected]
...
MariaDB [(none)]> select * from salt.salt_returns;
...

3 job 和 runner

https://www.unixhot.com/docs/saltstack/ref/modules/all/salt.modules.saltutil.html#module-salt.modules.saltutil

3.1 job

查看job cache里的数据

 salt-run jobs.list_jobs

查看之前执行job的返回内容

salt-run jobs.lookup_jid jid号

例子：

[[email protected] /srv]#   salt-run jobs.list_jobs
20170318125522221559:
    ----------
    Arguments:
        - ifconfig
    Function:
        cmd.run
    StartTime:
        2017, Mar 18 12:55:22.221559
    Target:
        *
    Target-type:
        glob
    User:
        root
20170318125618022528:
    ----------
    Arguments:
        - ifconfig
    Function:
        cmd.run
    StartTime:
        2017, Mar 18 12:56:18.022528
    Target:
        test-*
    Target-type:
        glob
    User:
        root
20170318125633831619:
    ----------
    Arguments:
        - ifconfig
    Function:
        cmd.run
    StartTime:
        2017, Mar 18 12:56:33.831619
    Target:
        test-*
    Target-type:
        glob
    User:
        root
[[email protected] /srv]# salt-run jobs.lookup_jid 20170318125633831619
test-node3.damaicha.org-203:
    eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
            inet 10.0.0.203  netmask 255.255.255.0  broadcast 10.0.0.255
            inet6 fe80::20c:29ff:fe79:7aa7  prefixlen 64  scopeid 0x20<link>
            ether 00:0c:29:79:7a:a7  txqueuelen 1000  (Ethernet)
            RX packets 608231  bytes 219740452 (209.5 MiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 452636  bytes 99942459 (95.3 MiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
            inet 127.0.0.1  netmask 255.0.0.0
            inet6 ::1  prefixlen 128  scopeid 0x10<host>
            loop  txqueuelen 0  (Local Loopback)
            RX packets 187130  bytes 13223278 (12.6 MiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 187130  bytes 13223278 (12.6 MiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

3.2 runner

查看minion的状态

[[email protected] /srv]# salt-run manage.status
down:
up:
    - test-node3.damaicha.org-203
    - salt-node4.damaicha.org-204

查看正常连接的minion

[[email protected] /srv]# salt-run manage.up
- test-node3.damaicha.org-203
- salt-node4.damaicha.org-204

查看down掉的minion

[[email protected] /srv]# salt-run manage.down

查看minion的版本

[[email protected] /srv]# salt-run manage.versions
Master:
    2016.11.3
Up to date:
    ----------
    salt-node4.damaicha.org-204:
        2016.11.3
    test-node3.damaicha.org-203:
        2016.11.3

2 无Master和多Master

1 masterless 无master

files_roots pillar_roots 和master的一样，区别在预不从master获取数据，改成了从本地。

1 修改minion的配置文件。

[[email protected] ~]# vim /etc/salt/minion
..
530 file_client: local
..

[[email protected] ~]# systemctl restart salt-minion

2 测试

[[email protected] ~]# salt-call test.ping
local:
    True

2 多master

需要注意的事项。

1 需要共享的东西（使用nfs）,结合git做版本管理。

<br/>keys： master minion<br/>file_roots里的所有文件<br/>pillar_root <br/>

####2 minion的配置。

- 10.0.0.203
- 10.0.0.204

3 Syndic

1 逻辑图

技术分享图片

2 原理

发布任务：将先生要干一件事，他将要做的事情传给sydic（小秘），sydic再将要做的事情给陈浩南，陈浩南收到信息后再将事情指派给下面的小弟去做。
返回数据：小弟干完事情后，将结果反馈给陈浩南，此时小秘(sydic）和陈浩南在一块，小秘听到后将消息转给蒋先生（不需要陈浩南反馈）

3 应用场景

多机房

注意：syndic的file_roots pilar_roots必须和saltmaster的保持高度一致。

4 配置安装

4.1角色部署

10.0.0.204 saltmaster+syndic minion
10.0.0.203 saltmaster 将先生

4.2 安装salt-syndic 10.0.0.204

[[email protected] ~]# yum -y install salt-syndic

4.3 修改master配置文件，告诉陈浩南老大是将先生。10.0.0.204

[[email protected] ~]# vim /etc/salt/maste
...
865 syndic_master: 10.0.0.203
...
[[email protected] ~]# systemctl restart salt-master
[[email protected] ~]# systemctl start salt-syndic

4.4 修改master配置文件，告诉将先生，小弟是陈浩南。10.0.0.203

[[email protected] ~]# vim /etc/salt/master
...
857 order_masters: True
...
[[email protected] ~]# systemctl restart salt-master

4.5 接受saltmaster+syndic 发来的key(收小弟）

[[email protected] ~]# salt-key -A
The following keys are going to be accepted:
Unaccepted Keys:
salt-node4.damaicha.org-204
Proceed? [n/Y] y
Key for minion salt-node

测试
[[email protected] ~]# salt ‘*‘ test.ping
salt-node4.damaicha.org-204:
    True
test-node3.damaicha.org-203:
    True

##4 saltstack-SSH

1 安装salt-ssh

[[email protected] ~]# yum -y install salt-ssh

2 编辑配置文件/etc/salt/roster

[[email protected] ~]# vim /etc/salt/roster 
#添加如下：
test-node3.damaicha.org-203:
  host: 10.0.0.204
  user: root
  passwd: 123..abc
  port: 52113

salt-node4.damaicha.org-204:
  host: 10.0.0.203
  user: root
  passwd: 123..abc
  port: 52113

3 测试

# 执行相当于c/s模式时的cmd.run （第一次使用时，这么玩。）
[[email protected] ~]# salt-ssh ‘*‘ -r ‘ifconfig‘ -i

# 执行高级状态
[[email protected] ~]# salt-ssh ‘*‘ state.highstate

拓展

拓展：
关闭ssh,key更换时提示的错误信息。
cd ~/.ssh
echo StricHostKeyChecking no >config

5 API

配置步骤

https 证书
配置文件
验证。使用pam验证
启动salt-api

https://www.unixhot.com/docs/saltstack/ref/netapi/all/salt.netapi.rest_cherrypy.html#a-rest-api-for-salt

1 安装https证书（生产环境是需要购买的）

# 新建本地用户，后面调用api的时候会用到
[[email protected] ~]# useradd  -M -s /sbin/nologin saltapi   
[[email protected] ~]# echo ‘saltapi‘|passwd --stdin saltapi

# 本机产生https证书
[[email protected] ~]# cd /etc/pki/tls/certs/
[[email protected] /etc/pki/tls/certs]# make testcert   密码123123  然后一路回车
[[email protected] /etc/pki/tls/certs]# cd /etc/pki/tls/private/
[[email protected] /etc/pki/tls/private]#  openssl rsa -in localhost.key -out salt_nopass.key    密码是123123
[[email protected] /etc/pki/tls/private]# ls
localhost.key  salt_nopass.key

2 安装CherryPy 和saltapi

[[email protected] ~]# yum -y install python-pip   salt-api
[[email protected] ~]# pip install --upgrade pip
[[email protected] ~]# pip install CherryPy==3.2.6   ``# ps: 如果不能安装这个版本就用yum来进行安装 yum install python-cherry*``

我遇到的梗：
如果salt-api启动报错，记得看日志。如果是cherrypy的问题，就重装pip install CherryPy

拓展：

[[email protected] ~]# vim .pip/pip.conf 
#更新pip源为淘宝的：
[global]
index-url = http://mirrors.aliyun.com/pypi/simple/
[install]
trusted-host=mirrors.aliyun.com

3 修改master配置文件

[[email protected] ~]# vim /etc/salt/master
...
  12 default_include: master.d/*.conf
...

[[email protected] ~]# cd /etc/salt/master.d/
[[email protected] /etc/salt/master.d]# vim api.conf
rest_cherrypy:
  host: 10.0.0.204
  port: 8000
  ssl_crt: /etc/pki/tls/certs/localhost.crt
  ssl_key: /etc/pki/tls/private/salt_nopass.key

[[email protected] /etc/salt/master.d]# vim eauth.conf
[[email protected] /etc/salt/master.d]# cat eauth.conf 
external_auth:
  pam:
    saltapi:
      - .*  # 代表能执行所有模块
      - ‘@wheel‘  # 代表salt-key
      - ‘@runner‘  ＃ runner看机器是否存活。

重启saltmaster 和api
[[email protected] /etc/salt/master.d]# systemctl restart salt-master
[[email protected] /etc/salt/master.d]# systemctl restart salt-api

4 获取token

[[email protected] master.d]# curl -k https://10.0.0.204:8000/login  -H "Accept: application/x-yaml"  -d username=‘saltapi‘ -d password=‘saltapi‘ -d eauth=‘pam‘

-----
return:
- eauth: pam
  expire: 1490041767.050187
  perms:
  - .*
  - ‘@wheel‘
  - ‘@runner‘
  start: 1489998567.050187
  token: 3d625f75a04cb066e7d1f975d140ff5f96a56a47
  user: saltapi

获得token后，我们使用获得的token查询minion的10.0.0.203的信息。它返回的是一个字典
curl -k https://10.0.0.204:8000/minions/test-node3.damaicha.org-203 -H "Accept: application/x-yaml" -H ‘X-Auth-Token: 3d625f75a04cb066e7d1f975d140ff5f96a56a47‘

通过runner查看那些机器活着
curl -k https://10.0.0.204:8000 -H "Accept: application/x-yaml" -H ‘X-Auth-Token: ef0b60cceb33fd66ab1ab3a8344671c461445d1c‘ -d client=‘runner‘ -d fun=‘manage.status‘
--------------
return:
- down: []
  up:
  - test-node3.damaicha.org-203
  - salt-node4.damaicha.org-204

test.ping 
[[email protected] sysconfig]# curl -k https://10.0.0.204:8000 -H "Accept: application/x-yaml" -H ‘X-Auth-Token: ef0b60cceb33fd66ab1ab3a8344671c461445d1c‘ -d client=‘local‘ -d tgt=‘*‘ -d fun=‘test.ping‘
----
return:
- salt-node4.damaicha.org-204: true
  test-node3.damaicha.org-203: true

执行高级状态，有同步和异步，下面的是同步的比较慢。异步的是async
curl -k https://10.0.0.204:8000 -H "Accept: application/x-yaml" -H ‘X-Auth-Token: ef0b60cceb33fd66ab1ab3a8344671c461445d1c‘ -d client=‘local‘ -d tgt=‘*‘ -d fun=‘state.highstate‘

查看jobs
curl -k https://10.0.0.204:8000/jobs -H "Accept: application/x-yaml" -H ‘X-Auth-Token: ef0b60cceb33fd66ab1ab3a8344671c461445d1c‘

查看指定的jid 的执行内容
curl -k https://10.0.0.204:8000/jobs/20170320163206321875 -H "Accept: application/x-yaml" -H ‘X-Auth-Token: 3d625f75a04cb066e7d1f975d140ff5f96a56a47‘

推荐oms
https://github.com/binbin91/oms

dashboard推荐
https://github.com/yueyongyue/saltshaker

以上是关于三 saltstack 任务管理和集群的主要内容，如果未能解决你的问题，请参考以下文章