一如何通过patroni如何管理3个postgres节点

Posted caidingyu

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了一如何通过patroni如何管理3个postgres节点相关的知识,希望对你有一定的参考价值。

环境: CentOS Linux release 7.6.1810 (Core) 内核版本:3.10.0-957.10.1.el7.x86_64

node1:192.168.216.130

node2:192.168.216.132

node3:192.168.216.134

一、安装postgres

yum install https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7-x86_64/pgdg-centos11-11-2.noarch.rpm
yum install postgresql11
yum install postgresql11-server
yum install postgresql11-libs
yum install postgresql11-contrib 
yum install postgresql11-devel
可以参考:https://www.jianshu.com/p/b4a759c2208f

 安装完成后可以查询下rpm -qa|grep postgres安装了哪些包

postgresql11-libs-11.5-1PGDG.rhel7.x86_64
postgresql10-libs-10.10-1PGDG.rhel7.x86_64
postgresql11-11.5-1PGDG.rhel7.x86_64
postgresql11-contrib-11.5-1PGDG.rhel7.x86_64
postgresql11-server-11.5-1PGDG.rhel7.x86_64
postgresql11-devel-11.5-1PGDG.rhel7.x86_64

 安装后不需要初始化,可由patroni来完成初始化操作,如果已经初始化完成,不需要patroni来初始化操作,可以修改patroni配置文件的以下参数来指定data目录和安装目录

data_dir: /var/lib/pgsql/11/data
bin_dir: /usr/pgsql-11/bin
config_dir: /var/lib/pgsql/11/data

 二、安装patroni,这里建议先修改pip源为国内,否则在安装过程中可能遇到大量超时问题

可参考:https://www.cnblogs.com/caidingyu/p/11566690.html

yum install gcc
yum install python-devel.x86_64
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python get-pip.py
pip install psycopg2-binary
pip install patroni[etcd,consul]

三、安装etcd服务

可参考:https://www.cnblogs.com/caidingyu/p/11408389.html

四、创建patroni的配置文件

node1:patroni配置文件如下

[root@localhost tmp]# cat /etc/patroni/patroni.yml 

scope: postgres-cluster
name: pgnode01
namespace: /service/

restapi:
  listen: 192.168.216.130:8008
  connect_address: 192.168.216.130:8008
#  certfile: /etc/ssl/certs/ssl-cert-snakeoil.pem
#  keyfile: /etc/ssl/private/ssl-cert-snakeoil.key
#  authentication:
#    username: username
#    password: password

etcd:
  hosts: 192.168.216.130:2379,192.168.216.132:2379,192.168.216.134:2379

bootstrap:
  # this section will be written into Etcd:/<namespace>/<scope>/config after initializing new cluster
  # and all other cluster members will use it as a `global configuration`
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576
    master_start_timeout: 300
    synchronous_mode: false
    synchronous_mode_strict: false
    #standby_cluster:
      #host: 127.0.0.1
      #port: 1111
      #primary_slot_name: patroni 
    postgresql:
      use_pg_rewind: true
      use_slots: true
      parameters:
        max_connections: 100
        superuser_reserved_connections: 5
        max_locks_per_transaction: 64
        max_prepared_transactions: 0
        huge_pages: try
        shared_buffers: 512MB
        work_mem: 128MB
        maintenance_work_mem: 256MB
        effective_cache_size: 4GB
        checkpoint_timeout: 15min
        checkpoint_completion_target: 0.9
        min_wal_size: 2GB
        max_wal_size: 4GB
        wal_buffers: 32MB
        default_statistics_target: 1000
        seq_page_cost: 1
        random_page_cost: 4
        effective_io_concurrency: 2
        synchronous_commit: on
        autovacuum: on
        autovacuum_max_workers: 5
        autovacuum_vacuum_scale_factor: 0.01
        autovacuum_analyze_scale_factor: 0.02
        autovacuum_vacuum_cost_limit: 200
        autovacuum_vacuum_cost_delay: 20
        autovacuum_naptime: 1s
        max_files_per_process: 4096
        archive_mode: on
        archive_timeout: 1800s
        archive_command: cd .
        wal_level: replica
        wal_keep_segments: 130
        max_wal_senders: 10
        max_replication_slots: 10
        hot_standby: on
        wal_log_hints: on
        shared_preload_libraries: pg_stat_statements,auto_explain
        pg_stat_statements.max: 10000
        pg_stat_statements.track: all
        pg_stat_statements.save: off
        auto_explain.log_min_duration: 10s
        auto_explain.log_analyze: true
        auto_explain.log_buffers: true
        auto_explain.log_timing: false
        auto_explain.log_triggers: true
        auto_explain.log_verbose: true
        auto_explain.log_nested_statements: true
        track_io_timing: on
        log_lock_waits: on
        log_temp_files: 0
        track_activities: on
        track_counts: on
        track_functions: all
        log_checkpoints: on
        logging_collector: on
        log_truncate_on_rotation: on
        log_rotation_age: 1d
        log_rotation_size: 0
        log_line_prefix: ‘%t [%p-%l] %r %q%u@%d ‘
        log_filename: ‘postgresql-%a.log‘
        log_directory: /var/log/postgresql
 
 #      recovery_conf:
#        restore_command: cp ../wal_archive/%f %p
 
  # some desired options for ‘initdb‘
  initdb:  # Note: It needs to be a list (some options need values, others are switches)
  - encoding: UTF8
  - locale: en_US.UTF-8
  - data-checksums
 
  pg_hba:  # Add following lines to pg_hba.conf after running ‘initdb‘
  - host replication replicator 127.0.0.1/32 md5
  - host all all 0.0.0.0/0 md5

 # Additional script to be launched after initial cluster creation (will be passed the connection URL as parameter)
# post_init: /usr/local/bin/setup_cluster.sh
 
  # Some additional users which needs to be created after initializing new cluster
#  users:
#    admin:
#      password: admin-pass
#      options:
#        - createrole
#        - createdb

  
postgresql:
  listen: 192.168.216.130,127.0.0.1:5432
  connect_address: 192.168.216.130:5432
  use_unix_socket: true
  data_dir: /var/lib/pgsql/11/data
  bin_dir: /usr/pgsql-11/bin
  config_dir: /var/lib/pgsql/11/data
  pgpass: /var/lib/pgsql/.pgpass
  authentication:
    replication:
      username: replicator
      password: replicator-pass
    superuser:
      username: postgres
      password: postgres-pass
#    rewind:  # Has no effect on postgres 10 and lower
#      username: rewind_user
#      password: rewind_password
  parameters:
    unix_socket_directories: /var/run/postgresql
    stats_temp_directory: /var/lib/pgsql_stats_tmp
 
#  callbacks:
#    on_start:
#    on_stop:
#    on_restart:
#    on_reload:
#    on_role_change:
 
  create_replica_methods:
#   - pgbackrest
#   - wal_e
   - basebackup
# pgbackrest:
#   command: /usr/bin/pgbackrest --stanza=<Stanza_Name> --delta restore
#   keep_data: True
#   no_params: True
#  wal_e
#    command: patroni_wale_restore
#    no_master: 1
#    envdir: /etc/wal_e/envdir
#    use_iam: 1
  basebackup:
    max-rate: ‘100M‘
 
 #watchdog:
#  mode: automatic # Allowed values: off, automatic, required
#  device: /dev/watchdog
#  safety_margin: 5
 
tags:
    nofailover: false
    noloadbalance: false
    clonefrom: false
    nosync: false
# specify a node to replicate from. This can be used to implement a cascading replication.
#    replicatefrom: (node name)

node2:patroni配置文件如下

[root@localhost postgresql]# cat /etc/patroni/patroni.yml 

scope: postgres-cluster
name: pgnode02
namespace: /service/

restapi:
  listen: 192.168.216.132:8008
  connect_address: 192.168.216.132:8008
#  certfile: /etc/ssl/certs/ssl-cert-snakeoil.pem
#  keyfile: /etc/ssl/private/ssl-cert-snakeoil.key
#  authentication:
#    username: username
#    password: password

etcd:
  hosts: 192.168.216.130:2379,192.168.216.132:2379,192.168.216.134:2379

bootstrap:
  # this section will be written into Etcd:/<namespace>/<scope>/config after initializing new cluster
  # and all other cluster members will use it as a `global configuration`
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576
    master_start_timeout: 300
    synchronous_mode: false
    synchronous_mode_strict: false
    #standby_cluster:
      #host: 127.0.0.1
      #port: 1111
      #primary_slot_name: patroni 
    postgresql:
      use_pg_rewind: true
      use_slots: true
      parameters:
        max_connections: 100
        superuser_reserved_connections: 5
        max_locks_per_transaction: 64
        max_prepared_transactions: 0
        huge_pages: try
        shared_buffers: 512MB
        work_mem: 128MB
        maintenance_work_mem: 256MB
        effective_cache_size: 4GB
        checkpoint_timeout: 15min
        checkpoint_completion_target: 0.9
        min_wal_size: 2GB
        max_wal_size: 4GB
        wal_buffers: 32MB
        default_statistics_target: 1000
        seq_page_cost: 1
        random_page_cost: 4
        effective_io_concurrency: 2
        synchronous_commit: on
        autovacuum: on
        autovacuum_max_workers: 5
        autovacuum_vacuum_scale_factor: 0.01
        autovacuum_analyze_scale_factor: 0.02
        autovacuum_vacuum_cost_limit: 200
        autovacuum_vacuum_cost_delay: 20
        autovacuum_naptime: 1s
        max_files_per_process: 4096
        archive_mode: on
        archive_timeout: 1800s
        archive_command: cd .
        wal_level: replica
        wal_keep_segments: 130
        max_wal_senders: 10
        max_replication_slots: 10
        hot_standby: on
        wal_log_hints: on
        shared_preload_libraries: pg_stat_statements,auto_explain
        pg_stat_statements.max: 10000
        pg_stat_statements.track: all
        pg_stat_statements.save: off
        auto_explain.log_min_duration: 10s
        auto_explain.log_analyze: true
        auto_explain.log_buffers: true
        auto_explain.log_timing: false
        auto_explain.log_triggers: true
        auto_explain.log_verbose: true
        auto_explain.log_nested_statements: true
        track_io_timing: on
        log_lock_waits: on
        log_temp_files: 0
        track_activities: on
        track_counts: on
        track_functions: all
        log_checkpoints: on
        logging_collector: on
        log_truncate_on_rotation: on
        log_rotation_age: 1d
        log_rotation_size: 0
        log_line_prefix: ‘%t [%p-%l] %r %q%u@%d ‘
        log_filename: ‘postgresql-%a.log‘
        log_directory: /var/log/postgresql
 
 #      recovery_conf:
#        restore_command: cp ../wal_archive/%f %p
 
  # some desired options for ‘initdb‘
  initdb:  # Note: It needs to be a list (some options need values, others are switches)
  - encoding: UTF8
  - locale: en_US.UTF-8
  - data-checksums
 
  pg_hba:  # Add following lines to pg_hba.conf after running ‘initdb‘
  - host replication replicator 127.0.0.1/32 md5
  - host all all 0.0.0.0/0 md5

 # Additional script to be launched after initial cluster creation (will be passed the connection URL as parameter)
# post_init: /usr/local/bin/setup_cluster.sh
 
  # Some additional users which needs to be created after initializing new cluster
#  users:
#    admin:
#      password: admin-pass
#      options:
#        - createrole
#        - createdb

  
postgresql:
  listen: 192.168.216.132,127.0.0.1:5432
  connect_address: 192.168.216.132:5432
  use_unix_socket: true
  data_dir: /var/lib/pgsql/11/data
  bin_dir: /usr/pgsql-11/bin
  config_dir: /var/lib/pgsql/11/data
  pgpass: /var/lib/pgsql/.pgpass
  authentication:
    replication:
      username: replicator
      password: replicator-pass
    superuser:
      username: postgres
      password: postgres-pass
#    rewind:  # Has no effect on postgres 10 and lower
#      username: rewind_user
#      password: rewind_password
  parameters:
    unix_socket_directories: /var/run/postgresql
    stats_temp_directory: /var/lib/pgsql_stats_tmp
 
#  callbacks:
#    on_start:
#    on_stop:
#    on_restart:
#    on_reload:
#    on_role_change:
 
  create_replica_methods:
#   - pgbackrest
#   - wal_e
   - basebackup
# pgbackrest:
#   command: /usr/bin/pgbackrest --stanza=<Stanza_Name> --delta restore
#   keep_data: True
#   no_params: True
#  wal_e
#    command: patroni_wale_restore
#    no_master: 1
#    envdir: /etc/wal_e/envdir
#    use_iam: 1
  basebackup:
    max-rate: ‘100M‘
 
 #watchdog:
#  mode: automatic # Allowed values: off, automatic, required
#  device: /dev/watchdog
#  safety_margin: 5
 
tags:
    nofailover: false
    noloadbalance: false
    clonefrom: false
    nosync: false
# specify a node to replicate from. This can be used to implement a cascading replication.
#    replicatefrom: (node name)

node3:patroni配置文件如下

[root@localhost tmp]# cat /etc/patroni/patroni.yml 

scope: postgres-cluster
name: pgnode03
namespace: /service/

restapi:
  listen: 192.168.216.134:8008
  connect_address: 192.168.216.134:8008
#  certfile: /etc/ssl/certs/ssl-cert-snakeoil.pem
#  keyfile: /etc/ssl/private/ssl-cert-snakeoil.key
#  authentication:
#    username: username
#    password: password

etcd:
  hosts: 192.168.216.130:2379,192.168.216.132:2379,192.168.216.134:2379

bootstrap:
  # this section will be written into Etcd:/<namespace>/<scope>/config after initializing new cluster
  # and all other cluster members will use it as a `global configuration`
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576
    master_start_timeout: 300
    synchronous_mode: false
    synchronous_mode_strict: false
    #standby_cluster:
      #host: 127.0.0.1
      #port: 1111
      #primary_slot_name: patroni 
    postgresql:
      use_pg_rewind: true
      use_slots: true
      parameters:
        max_connections: 100
        superuser_reserved_connections: 5
        max_locks_per_transaction: 64
        max_prepared_transactions: 0
        huge_pages: try
        shared_buffers: 512MB
        work_mem: 128MB
        maintenance_work_mem: 256MB
        effective_cache_size: 4GB
        checkpoint_timeout: 15min
        checkpoint_completion_target: 0.9
        min_wal_size: 2GB
        max_wal_size: 4GB
        wal_buffers: 32MB
        default_statistics_target: 1000
        seq_page_cost: 1
        random_page_cost: 4
        effective_io_concurrency: 2
        synchronous_commit: on
        autovacuum: on
        autovacuum_max_workers: 5
        autovacuum_vacuum_scale_factor: 0.01
        autovacuum_analyze_scale_factor: 0.02
        autovacuum_vacuum_cost_limit: 200
        autovacuum_vacuum_cost_delay: 20
        autovacuum_naptime: 1s
        max_files_per_process: 4096
        archive_mode: on
        archive_timeout: 1800s
        archive_command: cd .
        wal_level: replica
        wal_keep_segments: 130
        max_wal_senders: 10
        max_replication_slots: 10
        hot_standby: on
        wal_log_hints: on
        shared_preload_libraries: pg_stat_statements,auto_explain
        pg_stat_statements.max: 10000
        pg_stat_statements.track: all
        pg_stat_statements.save: off
        auto_explain.log_min_duration: 10s
        auto_explain.log_analyze: true
        auto_explain.log_buffers: true
        auto_explain.log_timing: false
        auto_explain.log_triggers: true
        auto_explain.log_verbose: true
        auto_explain.log_nested_statements: true
        track_io_timing: on
        log_lock_waits: on
        log_temp_files: 0
        track_activities: on
        track_counts: on
        track_functions: all
        log_checkpoints: on
        logging_collector: on
        log_truncate_on_rotation: on
        log_rotation_age: 1d
        log_rotation_size: 0
        log_line_prefix: ‘%t [%p-%l] %r %q%u@%d ‘
        log_filename: ‘postgresql-%a.log‘
        log_directory: /var/log/postgresql
 
 #      recovery_conf:
#        restore_command: cp ../wal_archive/%f %p
 
  # some desired options for ‘initdb‘
  initdb:  # Note: It needs to be a list (some options need values, others are switches)
  - encoding: UTF8
  - locale: en_US.UTF-8
  - data-checksums
 
  pg_hba:  # Add following lines to pg_hba.conf after running ‘initdb‘
  - host replication replicator 127.0.0.1/32 md5
  - host all all 0.0.0.0/0 md5

 # Additional script to be launched after initial cluster creation (will be passed the connection URL as parameter)
# post_init: /usr/local/bin/setup_cluster.sh
 
  # Some additional users which needs to be created after initializing new cluster
#  users:
#    admin:
#      password: admin-pass
#      options:
#        - createrole
#        - createdb

  
postgresql:
  listen: 192.168.216.134,127.0.0.1:5432
  connect_address: 192.168.216.134:5432
  use_unix_socket: true
  data_dir: /var/lib/pgsql/11/data
  bin_dir: /usr/pgsql-11/bin
  config_dir: /var/lib/pgsql/11/data
  pgpass: /var/lib/pgsql/.pgpass
  authentication:
    replication:
      username: replicator
      password: replicator-pass
    superuser:
      username: postgres
      password: postgres-pass
#    rewind:  # Has no effect on postgres 10 and lower
#      username: rewind_user
#      password: rewind_password
  parameters:
    unix_socket_directories: /var/run/postgresql
    stats_temp_directory: /var/lib/pgsql_stats_tmp
 
#  callbacks:
#    on_start:
#    on_stop:
#    on_restart:
#    on_reload:
#    on_role_change:
 
  create_replica_methods:
#   - pgbackrest
#   - wal_e
   - basebackup
# pgbackrest:
#   command: /usr/bin/pgbackrest --stanza=<Stanza_Name> --delta restore
#   keep_data: True
#   no_params: True
#  wal_e
#    command: patroni_wale_restore
#    no_master: 1
#    envdir: /etc/wal_e/envdir
#    use_iam: 1
  basebackup:
    max-rate: ‘100M‘
 
 #watchdog:
#  mode: automatic # Allowed values: off, automatic, required
#  device: /dev/watchdog
#  safety_margin: 5
 
tags:
    nofailover: false
    noloadbalance: false
    clonefrom: false
    nosync: false
# specify a node to replicate from. This can be used to implement a cascading replication.
#    replicatefrom: (node name)

  五、分别在3个node节点上创建/etc/systemd/system/patroni.service来通过systemctl管理patroni服务

1、创建软连接

ln -s /usr/bin/patronictl /usr/local/bin/patronictl
ln -s /usr/bin/patroni /usr/local/bin/patroni

2、在Node1上创建/etc/systemd/system/patroni.service

cat /etc/systemd/system/patroni.service
[Unit]
Description=Runners to orchestrate a high-availability PostgreSQL - patroni
After=syslog.target network.target
 
[Service]
Type=simple
 
User=postgres
Group=postgres

# Read in configuration file if it exists, otherwise proceed
EnvironmentFile=-/etc/patroni_env.conf

WorkingDirectory=~
 
# Where to send early-startup messages from the server
# This is normally controlled by the global default set by systemd
# StandardOutput=syslog

# Pre-commands to start watchdog device
# Uncomment if watchdog is part of your patroni setup
#ExecStartPre=-/usr/bin/sudo /sbin/modprobe softdog
#ExecStartPre=-/usr/bin/sudo /bin/chown postgres /dev/watchdog

# Start the patroni process
ExecStart=/usr/local/bin/patroni /etc/patroni/patroni.yml

# Send HUP to reload from patroni.yml
ExecReload=/bin/kill -s HUP $MAINPID
 
# only kill the patroni process, not it‘s children, so it will gracefully stop postgres
KillMode=process
 
# Give a reasonable amount of time for the server to start up/shut down
TimeoutSec=60
 
# Do not restart the service if it crashes, we want to manually inspect database on failure
Restart=no
 
[Install]
WantedBy=multi-user.target

3、在Node2上创建/etc/systemd/system/patroni.service

cat /etc/systemd/system/patroni.service
[Unit]
Description=Runners to orchestrate a high-availability PostgreSQL - patroni
After=syslog.target network.target
 
[Service]
Type=simple
 
User=postgres
Group=postgres

# Read in configuration file if it exists, otherwise proceed
EnvironmentFile=-/etc/patroni_env.conf

WorkingDirectory=~
 
# Where to send early-startup messages from the server
# This is normally controlled by the global default set by systemd
# StandardOutput=syslog

# Pre-commands to start watchdog device
# Uncomment if watchdog is part of your patroni setup
#ExecStartPre=-/usr/bin/sudo /sbin/modprobe softdog
#ExecStartPre=-/usr/bin/sudo /bin/chown postgres /dev/watchdog

# Start the patroni process
ExecStart=/usr/local/bin/patroni /etc/patroni/patroni.yml

# Send HUP to reload from patroni.yml
ExecReload=/bin/kill -s HUP $MAINPID
 
# only kill the patroni process, not it‘s children, so it will gracefully stop postgres
KillMode=process
 
# Give a reasonable amount of time for the server to start up/shut down
TimeoutSec=60
 
# Do not restart the service if it crashes, we want to manually inspect database on failure
Restart=no
 
[Install]
WantedBy=multi-user.target

 4、node3上操作同上,直接复制即可  

 

以上是关于一如何通过patroni如何管理3个postgres节点的主要内容,如果未能解决你的问题,请参考以下文章

基于patroni+etcd打造可自动故障转移的PostgreSQL集群

如何从命令行重新索引 Postgres 9.1.3

Patroni

Docker容器之Patroni篇

Docker容器之Patroni篇

如何将带有秘密管理器的谷歌应用引擎连接到 Postgres?