SaltStack——Pillar篇

Posted 2020-11-19 zuoyang

数据中心管理——Pillar

Pillar is an interface for Salt designed to offer global values that can be distributed to minions. Pillar data is managed in a similar way as the Salt State Tree.(Pillar was added to Salt in version 0.9.8)

Pillar也是SaltStack组件中非常重要的组件之一，是数据管理中心，我们经常配合states在大规模的配置管理工作中使用它，Pillar在SaltStack中主要的作用就是存储和定义配置管理中需要的一些数据，比如软件版本号，用户名密码等信息，它的定义存储格式跟Grains类似，都是YAML格式。

Storing sensitive data
Pillar data is compiled on the master. Additionally, pillar data for a given minion is only accessible by the minion for which it is targeted in the pillar configuration. This makes pillar useful for storing sensitive data specific to a particular minion.

声明Master的Pillar

The Salt Master server maintains a pillar_roots setup that matches the structure of the file_roots used in the Salt file server. Like file_roots, the pillar_roots option maps environments to directories. The pillar data is then mapped to minions based on matchers in a top file which is laid out in the same way as the state top file. Salt pillars can use the same matcher types as the standard top file.

conf_master:pillar_roots is configured just like file_roots. For example:

在Master配置文件中有一段Pillar settings选项专门定义Pillar相关的参数:

pillar_roots:
  base:
    - /srv/pillar

This example configuration declares that the base environment will be located in the /srv/pillar directory. It must not be in a subdirectory of the state tree.

这里我们只需要了解pillar_roots相关的配置即可，默认Base环境下Pillar的工作目录在/srv/pillar目录下。

The top file used matches the name of the top file used for States, and has the same structure:

我这里只使用默认的配置，首先去pillar工作目录新建top.sls文件，Target匹配所有‘*‘，同时引用packages.sls，如下所示:

/srv/pillar/top.sls

base:
  ‘*‘:
    - packages

In the above top file, it is declared that in the base environment, the glob matching all minions will have the pillar data found in the packages pillar available to it. Assuming the pillar_roots value of /srv/pillar taken from above, the packages pillar would be located at /srv/pillar/packages.sls.

Any number of matchers can be added to the base environment. For example, here is an expanded version of the Pillar top file stated above:

如果你想定义多个环境不同的Pillar工作目录，只需要修改这样修改配置文件即可。

/srv/pillar/top.sls:

base:
  ‘*‘:
    - packages
  ‘web*‘:
    - vim

In this expanded top file, minions that match web* will have access to the /srv/pillar/packages.sls file, as well as the /srv/pillar/vim.sls file.

Another example shows how to use other standard top matching types to deliver specific salt pillar data to minions with different properties.

Here is an example using the grains matcher to target pillars to minions by their os grain:

dev:
  ‘os:Debian‘:
    - match: grain
    - servers

/srv/pillar/packages.sls

{% if grains[‘os‘] == ‘RedHat‘ %}
apache: httpd
git: git
{% elif grains[‘os‘] == ‘Debian‘ %}
apache: apache2
git: git-core
{% endif %}

company: Foo Industries

Warning: https://docs.saltstack.com/en/latest/faq.html#faq-grain-security

See Is Targeting using Grain Data Secure? for important security information.

The above pillar sets two key/value pairs. If a minion is running RedHat, then the apache key is set to httpd and the git key is set to the value of git. If the minion is running Debian, those values are changed to apache2 and git-core respectively. All minions that have this pillar targeting to them via a top file will have the key of company with a value of Foo Industries.

Consequently this data can be used from within modules, renderers, State SLS files, and more via the shared pillar dictionary:

apache:
  pkg.installed:
    - name: {{ pillar[‘apache‘] }}

git:
  pkg.installed:
    - name: {{ pillar[‘git‘] }}

Finally, the above states can utilize the values provided to them via Pillar. All pillar values targeted to a minion are available via the ‘pillar‘ dictionary. As seen in the above example, Jinja substitution can then be utilized to access the keys and values in the Pillar dictionary.

Note that you cannot just list key/value-information in top.sls. Instead, target a minion to a pillar file and then list the keys and values in the pillar. Here is an example top file that illustrates this point:

base:
  ‘*‘:
     - common_pillar

And the actual pillar file at ‘/srv/pillar/common_pillar.sls‘:

foo: bar
boo: baz

When working with multiple pillar environments, assuming that each pillar environment has its own top file, the jinja placeholder {{ saltenv }} can be used in place of the environment name:

{{ saltenv }}:
  ‘*‘:
     - common_pillar

Yes, this is {{ saltenv }}, and not {{ pillarenv }}. The reason for this is because the Pillar top files are parsed using some of the same code which parses top files when running states, so the pillar environment takes the place of {{ saltenv }} in the jinja context.

dynamic pillar environments New in version 2017.7.5,2018.3.1

If environment __env__ is specified in pillar_roots, all environments that are not explicitly specified in pillar_roots will map to the directories from __env__. This allows one to use dynamic git branch based environments for state/pillar files with the same file-based pillar applying to all environments. For example:

这个功能只能在‘New in version 2017.7.5,2018.3.1‘之后才能使用，我个人觉得很好。

pillar_roots:
  __env__:
    - /srv/pillar

ext_pillar:
  - git:
    - __env__ https://example.com/git-pillar.git

The separate pillar SLS files all merge down into a single dictionary of key-value pairs. When the same key is defined in multiple SLS files, this can result in unexpected behavior if care is not taken to how the pillar SLS files are laid out.

单独的pillar sls文件合并成一个single dictionary of key-value pairs时候，key要保持唯一性，不然会意外情况发生。

比如：Then a request for the bind pillar key will only return named. The bind9 value will be lost, because services.sls was evaluated later.

For example, given a top.sls containing the following:

base:
  ‘*‘:
    - packages
    - services

with packages.sls containing:

bind: bind9

and services.sls containing:

bind: named

 PILLAR DICTIONARY MERGING

If the same pillar key is defined in multiple pillar SLS files, and the keys in both files refer to nested dictionaries, then the content from these dictionaries will be recursively merged.

如果多个pillar sls文件里有同样的key，并且在这些文件中的key有字典嵌套，那么这些字典文本将会被递归合并。

For example, keeping the top.sls the same, assume the following modifications to the pillar SLS files:

packages.sls:

bind:
  package-name: bind9
  version: 9.9.5

services.sls:

bind:
  port: 53
  listen-on: any

结果会变成:

$ salt-call pillar.get bind
local:
    ----------
    listen-on:
        any
    package-name:
        bind9
    port:
        53
    version:
        9.9.5

INCLUDING OTHER PILLARS New in version 0.16.0

Pillar SLS files may include other pillar files, similar to State files. Two syntaxes are available for this purpose. The simple form simply includes the additional pillar as if it were part of the same file:

include:
  - users

The full include form allows two additional options -- passing default values to the templating engine for the included pillar file as well as an optional key under which to nest the results of the included pillar:

include:
  - users:
      defaults:
          sudo: [‘bob‘, ‘paul‘]
      key: users

With this form, the included file (users.sls) will be nested within the ‘users‘ key of the compiled pillar. Additionally, the ‘sudo‘ value will be available as a template variable to users.sls.

IN-MEMORY PILLAR DATA VS ON-DEMAND PILLAR DATA

Since compiling pillar data is computationally expensive, the minion will maintain a copy of the pillar data in memory to avoid needing to ask the master to recompile and send it a copy of the pillar data each time pillar data is requested. This in-memory pillar data is what is returned by the pillar.item, pillar.get, and pillar.raw functions.

Also, for those writing custom execution modules, or contributing to Salt‘s existing execution modules, the in-memory pillar data is available as the __pillar__ dunder dictionary.

The in-memory pillar data is generated on minion start, and can be refreshed using the saltutil.refresh_pillar function:

salt ‘*‘ saltutil.refresh_pillar

This function triggers the minion to asynchronously refresh the in-memory pillar data and will always return None.

In contrast to in-memory pillar data, certain actions trigger pillar data to be compiled to ensure that the most up-to-date pillar data is available. These actions include:

• Running states
• Running pillar.items

Performing these actions will not refresh the in-memory pillar data. So, if pillar data is modified, and then states are run, the states will see the updated pillar data, but pillar.item,pillar.get, and pillar.raw will not see this data unless refreshed using saltutil.refresh_pillar.