Mysql读写分离—5.7 gtid 主从 + ProxySql 配置及简单测试
Posted 翔之天空
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Mysql读写分离—5.7 gtid 主从 + ProxySql 配置及简单测试相关的知识，希望对你有一定的参考价值。
参考：官方wiki： https://github.com/sysown/proxysql/wiki ProxySQL 配置详解及读写分离(+GTID)等功能说明 (完整篇) https://www.cnblogs.com/kevingrace/p/10329714.html 实验目的：用proxysql实现读写分离。主库负责写入，从库负责写入(要设置read_only=1参数)。 因为只是demo 暂时proxysql安装在主库的服务器上。 实验前提：已有一主一从 两台mysql服务(IP如下)，已开启了GTID，从库已设置read_only=1参数。 环境说明： proxysql的ip：192.168.56.80 主库的ip：192.168.56.80 server_id:222 从库的ip：192.168.56.61 server_id:111 一：proxy架构 proxysql是比较灵活的中间件，可实现读写分离、在线实时调整参数等。 --1、proxysql的架构图：
--2、proxysql三层概念：
runtime： 内存运行，代表的是ProxySQL当前生效的配置，必须从memory层load进来才能生效。
memory：
通常情况下只在memory层修改内存配置，但不影响runtime和disk层。 修改后可以load到runtime层 使其现在运行的环境生效，没问题后可以 save到disk层持久化保存
disk：持久化的存储配置参数 到 SQLite3数据库( $(DATADIR)/proxysql.db)。
/etc/proxysql.cnf 参数配置文件只在第一次启动时候加载，之后的启动都从proxysql.db加载配置数据
--3、安装proxysql
proxysql下载地址：
https://github.com/sysown/proxysql/releases
安装包：proxysql-2.0.10-1-centos7.x86_64.rpm
--安装proxysql
--安装proxysql ， 缺失包 详见附1
[root@hostmysql80 mysql_setup]# rpm -ivh proxysql-2.0.10-1-centos7.x86_64.rpm
--查看版本
[root@hostmysql80 ~]# proxysql --version
ProxySQL version 2.0.10-27-g5b31997, codename Truls
--启动proxysql，(首次启动时 读取/etc/proxysql.cnf 参数配置文件 默认为6032和6033。6032端口是ProxySQL的管理端口，6033是ProxySQL对外提供服务的端口)
[root@hostmysql80 mysql]# systemctl start proxysql.service
--查看当前proxysql的端口为6032和6033
[root@hostmysql80 ~]# netstat -tunlp |grep proxy
tcp 0 0 0.0.0.0:6032 0.0.0.0:* LISTEN 1019/proxysql
tcp 0 0 0.0.0.0:6033 0.0.0.0:* LISTEN 1019/proxysql
--查看当前proxysql的线程数
[root@hostmysql80 ~]# ss -lntup|grep proxy
tcp LISTEN 0 128 *:6032 *:* users:(("proxysql",pid=1019,fd=26))
tcp LISTEN 0 128 *:6033 *:* users:(("proxysql",pid=1019,fd=24))
tcp LISTEN 0 128 *:6033 *:* users:(("proxysql",pid=1019,fd=23))
tcp LISTEN 0 128 *:6033 *:* users:(("proxysql",pid=1019,fd=22))
tcp LISTEN 0 128 *:6033 *:* users:(("proxysql",pid=1019,fd=21))
--disk层会生成保存持久化配置数据的db
[root@hostmysql80 proxysql]# ll /var/lib/proxysql/
total 520
-rw-rw----. 1 proxysql proxysql 1054 Mar 19 14:56 proxysql-ca.pem
-rw-rw----. 1 proxysql proxysql 1062 Mar 19 14:56 proxysql-cert.pem
-rw-------. 1 proxysql proxysql 196608 Mar 19 14:56 proxysql.db
-rw-rw----. 1 proxysql proxysql 1675 Mar 19 14:56 proxysql-key.pem
-rw-------. 1 proxysql proxysql 7942 Mar 20 09:15 proxysql.log
-rw-r--r--. 1 proxysql proxysql 5 Mar 20 09:15 proxysql.pid
-rw-------. 1 proxysql proxysql 311296 Mar 20 16:45 proxysql_stats.db
--查看proxysql状态
[root@hostmysql80 mysql]# systemctl status proxysql.service
● proxysql.service - High Performance Advanced Proxy for MySQL
Loaded: loaded (/etc/systemd/system/proxysql.service; enabled; vendor preset: disabled)
Active: active (running) since Fri 2020-03-20 09:15:08 CST; 6h ago
Process: 951 ExecStart=/usr/bin/proxysql -c /etc/proxysql.cnf (code=exited, status=0/SUCCESS)
Main PID: 1017 (proxysql)
Tasks: 20
CGroup: /system.slice/proxysql.service
├─1017 /usr/bin/proxysql -c /etc/proxysql.cnf
└─1018 /usr/bin/proxysql -c /etc/proxysql.cnf
Mar 20 09:15:07 hostmysql80 systemd[1]: Starting High Performance Advanced Proxy for MySQL...
Mar 20 09:15:07 hostmysql80 proxysql[951]: 2020-03-20 09:15:07 [INFO] Using config file /etc/proxysql.cnf
Mar 20 09:15:07 hostmysql80 proxysql[951]: 2020-03-20 09:15:07 [INFO] Using OpenSSL version: OpenSSL 1.1.0h 27 Mar 2018
Mar 20 09:15:07 hostmysql80 proxysql[951]: 2020-03-20 09:15:07 [INFO] SSL keys/certificates found in datadir (/var/lib/proxysql): loading them.
Mar 20 09:15:08 hostmysql80 proxysql[951]: Process 1067 died: No such process; trying to remove PID file. (/var/lib/proxysql/proxysql.pid)
Mar 20 09:15:08 hostmysql80 systemd[1]: Started High Performance Advanced Proxy for MySQL.
--进入proxysql ，用6032管理端口进入
[root@hostmysql80 mysql]# mysql -u admin -padmin -h 127.0.0.1 -P6032 --prompt='Admin> '
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \\g.
Your MySQL connection id is 1
Server version: 5.5.30 (ProxySQL Admin Module)
Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\\h' for help. Type '\\c' to clear the current input statement.--查看proxysql的库
Admin> show databases;
+-----+---------------+-------------------------------------+
| seq | name | file |
+-----+---------------+-------------------------------------+
| 0 | main | |
| 2 | disk | /var/lib/proxysql/proxysql.db |
| 3 | stats | |
| 4 | monitor | |
| 5 | stats_history | /var/lib/proxysql/proxysql_stats.db |
+-----+---------------+-------------------------------------+
5 rows in set (0.03 sec)
附 每个库的含义；
--main 内存参数配置库，表里存放后端db实例、用户验证、路由规则等信息。表名以 runtime_开头的表示proxysql当前运行的配置内容runtime层，不能通过dml语句修改，只能修改对应的不以 runtime_ 开头的（在内存）里的表(memory层)，然后LOAD使其runtime层(runtime_开头的表)生效， SAVE使其存到disk层(disk库的表)硬盘以供下次重启加载。
--disk库 是持久化保存到硬盘参数配置库，存储在sqliteDB数据文件中。
--stats库 是proxysql运行时抓取的统计信息。
--monitor库 监控模块，记录mysql数据库的监控信息。
--查看main库 (disk库除了没有runtime_开头的表之外，其他的都和main库的表一致。)
--具体的核心配置表(以下红色标出)的字段说明 可参考如下 附2
Admin> show tables from main;
+----------------------------------------------------+
| tables |
+----------------------------------------------------+
| global_variables | //the list of global variables which the proxy is configured to use, and which can be tweaked during runtime 全局变量表， 可以在线实时调整
| mysql_aws_aurora_hostgroups |
| mysql_collations | //the list of MySQL collations available for the proxy to work with 字符集和校验规则
| mysql_firewall_whitelist_rules |
| mysql_firewall_whitelist_sqli_fingerprints |
| mysql_firewall_whitelist_users |
| mysql_galera_hostgroups |
| mysql_group_replication_hostgroups |
| mysql_query_rules | //the list of query rules which are evaluated when routing traffic to the various backend servers. 定义查询路由规则
| mysql_query_rules_fast_routing |
| mysql_replication_hostgroups | //defines replication hostgroups for use with traditional master / slave ASYNC or SEMI-SYNC replication 读写分组信息
| mysql_servers | //the list of backend servers which ProxySQL connects to 设置proxysql读写等配置的表
| mysql_users | //the list of users and their credentials which connect to ProxySQL 配置连接数据库的账号和监控账号
| proxysql_servers |
| restapi_routes |
| runtime_checksums_values |
| runtime_global_variables |
| runtime_mysql_aws_aurora_hostgroups |
| runtime_mysql_firewall_whitelist_rules |
| runtime_mysql_firewall_whitelist_sqli_fingerprints |
| runtime_mysql_firewall_whitelist_users |
| runtime_mysql_galera_hostgroups |
| runtime_mysql_group_replication_hostgroups |
| runtime_mysql_query_rules |
| runtime_mysql_query_rules_fast_routing |
| runtime_mysql_replication_hostgroups |
| runtime_mysql_servers |
| runtime_mysql_users |
| runtime_proxysql_servers |
| runtime_restapi_routes |
| runtime_scheduler |
| scheduler |
+----------------------------------------------------+
32 rows in set (0.00 sec)
--查看stats库
Admin> show tables from stats;
+--------------------------------------+
| tables |
+--------------------------------------+
| global_variables |
| stats_memory_metrics |
| stats_mysql_commands_counters |
| stats_mysql_connection_pool |
| stats_mysql_connection_pool_reset |
| stats_mysql_errors |
| stats_mysql_errors_reset |
| stats_mysql_free_connections |
| stats_mysql_global |
| stats_mysql_gtid_executed |
| stats_mysql_prepared_statements_info |
| stats_mysql_processlist |
| stats_mysql_query_digest |
| stats_mysql_query_digest_reset |
| stats_mysql_query_rules |
| stats_mysql_users |
| stats_proxysql_servers_checksums |
| stats_proxysql_servers_metrics |
| stats_proxysql_servers_status |
+--------------------------------------+
19 rows in set (0.01 sec)
--查看monitor库
Admin> show tables from monitor;
+--------------------------------------+
| tables |
+--------------------------------------+
| mysql_server_aws_aurora_check_status |
| mysql_server_aws_aurora_failovers |
| mysql_server_aws_aurora_log |
| mysql_server_connect_log |
| mysql_server_galera_log |
| mysql_server_group_replication_log |
| mysql_server_ping_log |
| mysql_server_read_only_log |
| mysql_server_replication_lag_log |
+--------------------------------------+
9 rows in set (0.01 sec)--proxysql有三层配置，runtime、memory、disk。 每一层都是独立存在的，如果需要修改proxysql的参数配置，方式如下:
首先在memory层修改配置参数，不会影响现有运行的runtime层环境 以及 持久化保存参数的disk层环境。当memory修改完毕后，可以load到runtime 让现有运行环境runtime层生效，没问题后可以save到disk层让修改后的参数配置持久化。
一般有load from 和load to， save to和save from 两种方式加载参数配置。但一般习惯使用 load to 和save to 方式(即从memory加载到runtime用load to，memory加载到disk用save to)
--mysql_users表的加载
LOAD MYSQL USERS TO RUNTIME
SAVE MYSQL USERS TO DISK
--mysql_servers表的加载
LOAD MYSQL SERVERS TO RUNTIME
SAVE MYSQL SERVERS TO DISK
--mysql_query_rules表的加载
LOAD MYSQL QUERY RULES TO RUNTIME
SAVE MYSQL QUERY RULES TO DISK
--global_variables表的加载
LOAD MYSQL VARIABLES TO RUNTIME
SAVE MYSQL VARIABLES TO DISK
--global_variables表中 admin- 开头参数配置 的加载
LOAD ADMIN VARIABLES TO RUNTIME
SAVE ADMIN VARIABLES TO DISK--7.1查看要配置的核心配置表，查看应该是无数据的。
[root@hostmysql80 mysql]# mysql -u admin -padmin -h 127.0.0.1 -P6032 --prompt='Admin> '
--切换到main
Admin> use main;
Admin> SELECT * FROM mysql_servers; //配置mysql服务的信息
Empty set (0.01 sec)
Admin> SELECT * from mysql_replication_hostgroups; //配置写组和读组的信息
Empty set (0.01 sec)
Admin> SELECT * from mysql_users; //配置通过proxysql连接mysql服务的用户账号的信息
Empty set (0.01 sec)
Admin> SELECT * from mysql_query_rules; //配置路由规则
Empty set (0.01 sec)
--7.2在mysql_servers表中新增mysql的主库和从库信息。具体的mysql_servers表的字段说明 可参考如下 附2
--192.168.56.80为主库ip，192.168.56.61为从库ip。hostgroup_id是分组id，这里设置1为写组，2为读组(这里暂定都设置1组，后面ProxySQL需要通过每个节点的read_only值来自动调整它们是属于读组还是写组，比如从库192.168.56.61 会自动的把hostgroup_id 设置为2 详见7.6)。
Admin> INSERT INTO mysql_servers(hostgroup_id,hostname,port) VALUES (1,'192.168.56.80',3306);
Admin> INSERT INTO mysql_servers(hostgroup_id,hostname,port) VALUES (1,'192.168.56.61',3306);
--查看mysql_servers
Admin> select * from mysql_servers;
+--------------+---------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| hostgroup_id | hostname | port | gtid_port | status | weight | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | comment |
+--------------+---------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| 1 | 192.168.56.80 | 3306 | 0 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | |
| 1 | 192.168.56.61 | 3306 | 0 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | |
+--------------+---------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
2 rows in set (0.00 sec)
--加载到runtime和disk
Admin> load mysql servers to runtime;
Admin> save mysql servers to disk;
--7.3后端添加monitor监控。会监控主从的信息，写入到监控表中。还会把read_only为1的库自动设置为读组
--登录主库192.168.56.80添加监控的用户monitor，设置密码、权限等。
[root@hostmysql80 ~]# mysql -uroot -p
mysql> create user monitor@'192.168.56.%' identified by 'Monitor123$';
mysql> grant replication client on *.* to monitor@'192.168.56.%';
mysql> flush privileges;
--返回到proxysql 设置连接mysql数据库的监控monitor用户和密码
Admin> set mysql-monitor_username='monitor';
Admin> set mysql-monitor_password='Monitor123$';
Admin> select * from global_variables where variable_name like '%monitor_username%' or variable_name like '%monitor_password%';
+------------------------+----------------+
| variable_name | variable_value |
+------------------------+----------------+
| mysql-monitor_username | monitor |
| mysql-monitor_password | Monitor123$ |
+------------------------+----------------+
2 rows in set (0.00 sec)
--加载到runtime和disk
Admin> load mysql variables to runtime;
Admin> save mysql variables to disk;
--查看monitor监控表的信息
Admin> SHOW TABLES FROM monitor;
+--------------------------------------+
| tables |
+--------------------------------------+
| mysql_server_aws_aurora_check_status |
| mysql_server_aws_aurora_failovers |
| mysql_server_aws_aurora_log |
| mysql_server_connect_log |
| mysql_server_galera_log |
| mysql_server_group_replication_log |
| mysql_server_ping_log |
| mysql_server_read_only_log |
| mysql_server_replication_lag_log |
+--------------------------------------+
9 rows in set (0.01 sec)
--判断连接是否正常的监控表
Admin> SELECT * FROM monitor.mysql_server_connect_log ORDER BY time_start_us DESC LIMIT 10;
+---------------+------+------------------+-------------------------+---------------+
| hostname | port | time_start_us | connect_success_time_us | connect_error |
+---------------+------+------------------+-------------------------+---------------+
| 192.168.56.80 | 3306 | 1584948975209848 | 2310 | NULL |
| 192.168.56.61 | 3306 | 1584948974072648 | 4395 | NULL |
| 192.168.56.80 | 3306 | 1584948914769017 | 1650 | NULL |
| 192.168.56.61 | 3306 | 1584948914071812 | 2711 | NULL |
| 192.168.56.80 | 3306 | 1584948854877383 | 1566 | NULL |
| 192.168.56.61 | 3306 | 1584948854070703 | 1196 | NULL |
| 192.168.56.61 | 3306 | 1584948795047377 | 1380 | NULL |
| 192.168.56.80 | 3306 | 1584948794070877 | 1757 | NULL |
| 192.168.56.61 | 3306 | 1584948734867748 | 4011 | NULL |
| 192.168.56.80 | 3306 | 1584948734070699 | 1633 | NULL |
+---------------+------+------------------+-------------------------+---------------+
10 rows in set (0.00 sec)
--ping心跳信息的监控表
Admin> SELECT * FROM monitor.mysql_server_ping_log ORDER BY time_start_us DESC LIMIT 10;
+---------------+------+------------------+----------------------+------------+
| hostname | port | time_start_us | ping_success_time_us | ping_error |
+---------------+------+------------------+----------------------+------------+
| 192.168.56.80 | 3306 | 1584949004468120 | 133 | NULL |
| 192.168.56.61 | 3306 | 1584949004365508 | 1479 | NULL |
| 192.168.56.61 | 3306 | 1584948994535963 | 789 | NULL |
| 192.168.56.80 | 3306 | 1584948994364743 | 123 | NULL |
| 192.168.56.80 | 3306 | 1584948984501357 | 256 | NULL |
| 192.168.56.61 | 3306 | 1584948984364424 | 1403 | NULL |
| 192.168.56.80 | 3306 | 1584948974506591 | 554 | NULL |
| 192.168.56.61 | 3306 | 1584948974364011 | 1329 | NULL |
| 192.168.56.80 | 3306 | 1584948964493164 | 225 | NULL |
| 192.168.56.61 | 3306 | 1584948964363715 | 1497 | NULL |
+---------------+------+------------------+----------------------+------------+
10 rows in set (0.00 sec)
--7.4 设置复制组信息的表，设定写组为1，读组为2
Admin> insert into mysql_replication_hostgroups (writer_hostgroup ,reader_hostgroup ,comment ) values (1,2,'proxysql_demo');
Query OK, 1 row affected (0.00 sec)
* If they have read_only=0 , they will be moved to hostgroup 1
* If they have read_only=1 , they will be moved to hostgroup 2
--加载到runtime和disk
Admin> load mysql servers to runtime;
Admin> save mysql servers to disk;
--再次查看mysql_servers表，因为192.168.56.61的从库read_only设置为1，所以被monitor设置为了读组2。这样主库192.168.56.80就为只写的组，192.168.56.61从库只读，实现了库的读写分离配置。
Admin> select * from mysql_servers;
+--------------+---------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| hostgroup_id | hostname | port | gtid_port | status | weight | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | comment |
+--------------+---------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| 1 | 192.168.56.80 | 3306 | 0 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | |
| 2 | 192.168.56.61 | 3306 | 0 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | |
+--------------+---------------+------+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
2 rows in set (0.00 sec)
--查看mysql库设置了read_only参数的状态表
Admin> SELECT * FROM monitor.mysql_server_read_only_log ORDER BY time_start_us DESC LIMIT 10;
+---------------+------+------------------+-----------------+-----------+-------+
| hostname | port | time_start_us | success_time_us | read_only | error |
+---------------+------+------------------+-----------------+-----------+-------+
| 192.168.56.61 | 3306 | 1584949238400396 | 1294 | 1 | NULL |
| 192.168.56.80 | 3306 | 1584949238378363 | 555 | 0 | NULL |
| 192.168.56.61 | 3306 | 1584949236907160 | 1734 | 1 | NULL |
| 192.168.56.80 | 3306 | 1584949236877600 | 320 | 0 | NULL |
| 192.168.56.80 | 3306 | 1584949235401126 | 183 | 0 | NULL |
| 192.168.56.61 | 3306 | 1584949235377586 | 820 | 1 | NULL |
| 192.168.56.80 | 3306 | 1584949233899518 | 401 | 0 | NULL |
| 192.168.56.61 | 3306 | 1584949233877797 | 1786 | 1 | NULL |
| 192.168.56.80 | 3306 | 1584949232407828 | 739 | 0 | NULL |
| 192.168.56.61 | 3306 | 1584949232377432 | 1716 | 1 | NULL |
+---------------+------+------------------+-----------------+-----------+-------+
10 rows in set (0.00 sec)
--7.5 添加通过proxysql来连接mysql的用户 mysql_users 表
Admin> select * from mysql_users;
Empty set (0.00 sec)
--登录主库192.168.56.80添加通过proxysql来连接mysql的用户，设置密码、权限等。 proxy_user用户可以读写，read_user只读
[root@hostmysql80 ~]# mysql -uroot -p
mysql> create user proxy_user@'192.168.56.%' identified by 'proxy_user123$';
mysql> grant all on *.* to proxy_user@'192.168.56.%';
mysql> create user read_user@'192.168.56.%' identified by 'read_user123$';
mysql> grant select on *.* to read_user@'192.168.56.%';
mysql> flush privileges;
--返回到proxysql 设置连接mysql数据库的读写proxy_user和只读read_user用户和密码。default_hostgroup为默认的组，proxy_user设置1为写组 read_user设置2为读组。具体的mysql_users表的字段说明 可参考如下 附2
Admin> insert into mysql_users(username,password,default_hostgroup) values('proxy_user','proxy_user123$',1);
Admin> insert into mysql_users(username,password,default_hostgroup) values('read_user','read_user123$',2);
Admin> select * from mysql_users;
+------------+----------------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+---------+
| username | password | active | use_ssl | default_hostgroup | default_schema | schema_locked | transaction_persistent | fast_forward | backend | frontend | max_connections | comment |
+------------+----------------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+---------+
| proxy_user | proxy_user123$ | 1 | 0 | 1 | NULL | 0 | 1 | 0 | 1 | 1 | 10000 | |
| read_user | read_user123$ | 1 | 0 | 2 | NULL | 0 | 1 | 0 | 1 | 1 | 10000 | |
+------------+----------------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+---------+
2 rows in set (0.00 sec)
--加载到runtime和disk
Admin> load mysql users to runtime;
Admin> save mysql users to disk;
--主库server_id：222，从库server_id：111
--通过uproxy_user用户登录mysql服务器，因为该用户是写组1，所以会自动连接主库(server_id：222)。
[root@hostmysql80 ~]# mysql -uproxy_user -pproxy_user123$ -h192.168.56.80 -P6033
mysql> show variables like 'server_id';
+----------------+-------+
| Variable_name | Value |
+----------------+-------+
| server_id | 222 |
+----------------+-------+
1 rows in set (0.14 sec)
--通过read_user用户登录mysql服务器，因为该用户是读组2，所以会自动连接从库(server_id：111)。
[root@hostmysql61 ~]# mysql -uread_user -pread_user123$ -h192.168.56.80 -P6033
mysql> show variables like 'server_id';
+----------------+-------+
| Variable_name | Value |
+----------------+-------+
| server_id | 111 |
+----------------+-------+
1 rows in set (0.02 sec)
--7.6配置路由规则
--新增一个简单的路由规则，把select开头的sql语句 放到从库(server_id：111)上面运行(也就是读组2)。
Admin> insert into mysql_query_rules(rule_id,active,match_digest,destination_hostgroup,apply) VALUES (1,1,'^SELECT',2,1);
Query OK, 1 row affected (0.00 sec)
--加载到runtime和disk
Admin> load mysql query rules to runtime;
Admin> save mysql query rules to disk;
--通过uproxy_user用户登录mysql服务器，因为该用户是写组1，所以会自动连接主库(server_id：222)。
[root@hostmysql80 ~]# mysql -uproxy_user -pproxy_user123$ -h192.168.56.80 -P6033
mysql> show variables like 'server_id';
+----------------+-------+
| Variable_name | Value |
+----------------+-------+
| server_id | 222 |
+----------------+-------+
1 rows in set (0.14 sec)
--执行建表操作
mysql> create table flydb.t_proxysql_rule_test (a int);
Query OK, 0 rows affected (0.05 sec)
--执行插入数据操作
mysql> insert into flydb.t_proxysql_rule_test values(1);
Query OK, 1 row affected (0.01 sec)
--执行select查询操作，因为select开头 所以应该走的从库(server_id：111)上面运行(也就是读组2)
mysql> select * from flydb.t_proxysql_rule_test;
+------+
| a |
+------+
| 1 |
+------+
1 row in set (0.00 sec)
--可以通过stats库下的stats_mysql_query_digest表查看具体sql运行的信息
--可以查看到红色标记的三个sql都是通过proxy_user用户连接到mysql服务的，执行建表和插入操作都是走的写组1(主库操作)，select开头的查询sql走的读组2(从库操作)
Admin> SELECT hostgroup,schemaname,username,client_address,digest_text,count_star FROM stats.stats_mysql_query_digest ORDER BY sum_time DESC limit 10;
+-----------+--------------------+------------+----------------+--------------------------------------------------+------------+
| hostgroup | schemaname | username | client_address | digest_text | count_star |
+-----------+--------------------+------------+----------------+--------------------------------------------------+------------+
| 1 | information_schema | proxy_user | | show variables like ? | 2 |
| 2 | information_schema | read_user | | show variables like ? | 1 |
| 1 | information_schema | proxy_user | | create table flydb.t_proxysql_rule_test (a int) | 1 |
| 1 | information_schema | proxy_user | | insert into flydb.t_proxysql_rule_test values(?) | 1 |
| 2 | information_schema | proxy_user | | select * from flydb.t_proxysql_rule_test | 1 |
| 2 | information_schema | read_user | | select @@version_comment limit ? | 1 |
| 1 | information_schema | proxy_user | | select @@version_comment limit ? | 2 |
+-----------+--------------------+------------+----------------+--------------------------------------------------+------------+
8 rows in set (0.00 sec)
--7.7查看stats统计信息的表
Admin> SHOW TABLES FROM stats;
+--------------------------------------+
| tables |
+--------------------------------------+
| global_variables |
| stats_memory_metrics |
| stats_mysql_commands_counters |
| stats_mysql_connection_pool |
| stats_mysql_connection_pool_reset |
| stats_mysql_errors |
| stats_mysql_errors_reset |
| stats_mysql_free_connections |
| stats_mysql_global |
| stats_mysql_gtid_executed |
| stats_mysql_prepared_statements_info |
| stats_mysql_processlist |
| stats_mysql_query_digest |
| stats_mysql_query_digest_reset |
| stats_mysql_query_rules |
| stats_mysql_users |
| stats_proxysql_servers_checksums |
| stats_proxysql_servers_metrics |
| stats_proxysql_servers_status |
+--------------------------------------+
19 rows in set (0.03 sec)
--查看统计mysql数据库连接池信息的表
Admin> SELECT * FROM stats.stats_mysql_connection_pool;
+-----------+---------------+----------+--------+----------+----------+--------+---------+-------------+---------+-------------------+-----------------+-----------------+------------+
| hostgroup | srv_host | srv_port | status | ConnUsed | ConnFree | ConnOK | ConnERR | MaxConnUsed | Queries | Queries_GTID_sync | Bytes_data_sent | Bytes_data_recv | Latency_us |
+-----------+---------------+----------+--------+----------+----------+--------+---------+-------------+---------+-------------------+-----------------+-----------------+------------+
| 1 | 192.168.56.80 | 3306 | ONLINE | 0 | 1 | 1 | 0 | 1 | 29 | 0 | 951 | 543 | 398 |
| 2 | 192.168.56.61 | 3306 | ONLINE | 0 | 1 | 1 | 0 | 1 | 2 | 0 | 66 | 80 | 1490 |
+-----------+---------------+----------+--------+----------+----------+--------+---------+-------------+---------+-------------------+-----------------+-----------------+------------+
2 rows in set (0.06 sec)
--查看统计运行sql次数的表
Admin> SELECT * FROM stats_mysql_commands_counters WHERE Total_cnt;
+--------------+---------------+-----------+-----------+-----------+---------+---------+----------+----------+-----------+-----------+--------+--------+---------+----------+
| Command | Total_Time_us | Total_cnt | cnt_100us | cnt_500us | cnt_1ms | cnt_5ms | cnt_10ms | cnt_50ms | cnt_100ms | cnt_500ms | cnt_1s | cnt_5s | cnt_10s | cnt_INFs |
+--------------+---------------+-----------+-----------+-----------+---------+---------+----------+----------+-----------+-----------+--------+--------+---------+----------+
| CREATE_TABLE | 62530 | 2 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 |
| INSERT | 9794 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| SELECT | 4229 | 4 | 3 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| SHOW | 192809 | 3 | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 1 | 0 | 0 | 0 | 0 |
+--------------+---------------+-----------+-----------+-----------+---------+---------+----------+----------+-----------+-----------+--------+--------+---------+----------+
4 rows in set (0.01 sec)
--查看统计运行具体sql的表
Admin> SELECT * FROM stats_mysql_query_digest ORDER BY sum_time DESC limit 5;
+-----------+--------------------+------------+----------------+--------------------+--------------------------------------------------+------------+------------+------------+----------+----------+----------+-------------------+---------------+
| hostgroup | schemaname | username | client_address | digest | digest_text | count_star | first_seen | last_seen | sum_time | min_time | max_time | sum_rows_affected | sum_rows_sent |
+-----------+--------------------+------------+----------------+--------------------+--------------------------------------------------+------------+------------+------------+----------+----------+----------+-------------------+---------------+
| 1 | information_schema | proxy_user | | 0xB06310E983BD5E0B | show variables like ? | 2 | 1585017073 | 1585017535 | 161593 | 36212 | 125381 | 0 | 2 |
| 1 | information_schema | proxy_user | | 0xC8D7E725163C0840 | create table flydb.t_proxysql_rule_test (a int) | 1 | 1585017569 | 1585017569 | 50755 | 50755 | 50755 | 0 | 0 |
| 2 | information_schema | read_user | | 0xB06310E983BD5E0B | show variables like ? | 1 | 1585017083 | 1585017083 | 31216 | 31216 | 31216 | 0 | 1 |
| 1 | information_schema | proxy_user | | 0x71C66C96E0C12B1D | create table t_proxysql_rule_test (a int) | 1 | 1585017562 | 1585017562 | 11775 | 11775 | 11775 | 0 | 0 |
| 1 | information_schema | proxy_user | | 0x0DEF4CDD8F99F1E6 | insert into flydb.t_proxysql_rule_test values(?) | 1 | 1585017580 | 1585017580 | 9794 | 9794 | 9794 | 1 | 0 |
+-----------+--------------------+------------+----------------+--------------------+--------------------------------------------------+------------+------------+------------+----------+----------+----------+-------------------+---------------+
5 rows in set (0.01 sec)
--看top sql的信息。
Admin> SELECT hostgroup hg, sum_time, count_star, digest_text FROM stats_mysql_query_digest ORDER BY sum_time DESC limit 10;
+----+----------+------------+------------------------------------------+
| hg | sum_time | count_star | digest_text |
+----+----------+------------+------------------------------------------+
| 1 | 134873 | 1 | show variables like ? |
| 1 | 61296 | 1 | create table t_proxysql_test(a int ) |
| 1 | 24725 | 1 | SELECT * FROM gtid_executed WHERE ?=? |
| 1 | 18057 | 1 | show databases |
| 1 | 17840 | 1 | SELECT DATABASE() |
| 1 | 17791 | 1 | insert into t_proxysql_test values (?) |
| 2 | 13127 | 2 | show variables like ? |
| 1 | 10006 | 1 | SELECT * FROM t3 WHERE ?=? |
| 1 | 9314 | 1 | SELECT * FROM t_auto WHERE ?=? |
| 1 | 5310 | 1 | SELECT * FROM test_limit_table WHERE ?=? |
+----+----------+------------+------------------------------------------+
10 rows in set (0.00 sec)--拷贝一份disk层的数据 做测试
[root@hostmysql80 proxysql]# cp /var/lib/proxysql/proxysql.db /mysql_setup/proxysql_test.db
[root@hostmysql80 proxysql]# cd /mysql_setup/
--进入sqlite库
[root@hostmysql80 mysql_setup]# sqlite3
SQLite version 3.7.17 2013-05-20 00:56:22
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> .databases
seq name file
--- --------------- ----------------------------------------------------------
0 main
--加载测试库proxysql_test.db 到sqlite库中
sqlite> ATTACH DATABASE 'proxysql_test.db' as 'test';
sqlite> .databases
seq name file
--- --------------- ----------------------------------------------------------
0 main
2 test /mysql_setup/proxysql_test.db
--设置显示字段
sqlite> .header on
--设置列对其
sqlite> .mode column
--设置列的宽度(这里只设置了前三列)
sqlite> .width 20, 20, 10
--select可以查询到disk层的数据
sqlite> select hostgroup_id,hostname,port from mysql_servers;
hostgroup_id hostname port
-------------------- -------------------- ----------
1 192.168.56.80 3306
2 192.168.56.61 3306 [root@hostmysql80 mysql_setup]# rpm -ivh proxysql-2.0.10-1-centos7.x86_64.rpm
error: Failed dependencies:
perl(DBD::mysql) is needed by proxysql-2.0.10-1.x86_64
perl(DBI) is needed by proxysql-2.0.10-1.x86_64
[root@hostmysql80 mysql_setup]# yum install -y perl-DBI.x86_64
[root@hostmysql80 mysql_setup]# yum install -y perl-DBD-mysql.el7.x86_64mysql_servers
Table mysql_servers defines all the MySQL servers: Admin> SHOW CREATE TABLE mysql_servers\\G *************************** 1. row *************************** table: mysql_servers Create Table: CREATE TABLE mysql_servers ( hostgroup_id INT CHECK (hostgroup_id>=0) NOT NULL DEFAULT 0, hostname VARCHAR NOT NULL, port INT NOT NULL DEFAULT 3306, gtid_port INT CHECK (gtid_port <> port) NOT NULL DEFAULT 0, status VARCHAR CHECK (UPPER(status) IN ('ONLINE','SHUNNED','OFFLINE_SOFT', 'OFFLINE_HARD')) NOT NULL DEFAULT 'ONLINE', weight INT CHECK (weight >= 0 AND weight <=10000000) NOT NULL DEFAULT 1, compression INT CHECK (compression >=0 AND compression <= 102400) NOT NULL DEFAULT 0, max_connections INT CHECK (max_connections >=0) NOT NULL DEFAULT 1000, max_replication_lag INT CHECK (max_replication_lag >= 0 AND max_replication_lag <= 126144000) NOT NULL DEFAULT 0, use_ssl INT CHECK (use_ssl IN(0,1)) NOT NULL DEFAULT 0, max_latency_ms INT UNSIGNED CHECK (max_latency_ms>=0) NOT NULL DEFAULT 0, comment VARCHAR NOT NULL DEFAULT '', PRIMARY KEY (hostgroup_id, hostname, port) ) 1 row in set (0.00 sec) The fields have the following semantics:- hostgroup_id: the hostgroup in which this mysqld instance is included. Notice that the same instance can be part of more than one hostgroup
- hostname, port: the TCP endpoint at which the mysqld instance can be contacted
- gtid_port: the backend server port where ProxySQL Binlog Reader listens on for GTID tracking
-
status:
- ONLINE - backend server is fully operational
- SHUNNED - backend sever is temporarily taken out of use because of either too many connection errors in a time that was too short, or replication lag exceeded the allowed threshold
- OFFLINE_SOFT - when a server is put into OFFLINE_SOFT mode, new incoming connections aren't accepted anymore, while the existing connections are kept until they became inactive. In other words, connections are kept in use until the current transaction is completed. This allows to gracefully detach a backend
- OFFLINE_HARD - when a server is put into OFFLINE_HARD mode, the existing connections are dropped, while new incoming connections aren't accepted either. This is equivalent to deleting the server from a hostgroup, or temporarily taking it out of the hostgroup for maintenance work
- weight - the bigger the weight of a server relative to other weights, the higher the probability of the server to be chosen from a hostgroup
- compression - if the value is greater than 0, new connections to that server will use compression
- max_connections - the maximum number of SHOW CREATE TABLE mysql_replication_hostgroups\\Gconnections ProxySQL will open to this backend server. Even though this server will have the highest weight, no new connections will be opened to it once this limit is hit. Please ensure that the backend is configured with a correct value of max_connections to avoid that ProxySQL will try to go beyond that limit
- max_replication_lag - if greater than 0, ProxySQL will regularly monitor replication lag and if it goes beyond such threshold it will temporary shun the host until replication catch ups
- use_ssl - if set to 1, connections to the backend will use SSL
- max_latency_ms - ping time is regularly monitored. If a host has a ping time greater than max_latency_ms it is excluded from the connection pool (although the server stays ONLINE)
- comment - text field that can be used for any purposed defined by the user. Could be a description of what the host stores, a reminder of when the host was added or disabled, or a JSON processed by some checker script.
mysql_replication_hostgroups
Table mysql_replication_hostgroups defines replication hostgroups for use with traditional master / slave ASYNC or SEMI-SYNC replication. In case Group Replication / InnoDB Cluster or Galera / Percona XtraDB Cluster is used for replication the mysql_group_replication_hostgroups or mysql_galera_hostgroups (available in version 2.x) should be used instead. Admin> *************************** 1. row *************************** table: mysql_replication_hostgroups Create Table: CREATE TABLE mysql_replication_hostgroups ( writer_hostgroup INT CHECK (writer_hostgroup>=0) NOT NULL PRIMARY KEY, reader_hostgroup INT NOT NULL CHECK (reader_hostgroup<>writer_hostgroup AND reader_hostgroup>0), check_type VARCHAR CHECK (LOWER(check_type) IN ('read_only','innodb_read_only','super_read_only')) NOT NULL DEFAULT 'read_only', comment VARCHAR, UNIQUE (reader_hostgroup)) 1 row in set (0.00 sec) Each row in mysql_replication_hostgroups represent a pair of writer_hostgroup and reader_hostgroup . ProxySQL will monitor the value of read_only for all the servers in specified hostgroups, and based on the value of read_only will assign the server to the writer or reader hostgroups. The field comment can be used to store any arbitrary data. The fields have the following semantics:- writer_hostgroup - the hostgroup that all traffic will be sent to by default, nodes that have read_only=0 in MySQL will be assigned to this hostgroup.
- reader_hostgroup - the hostgroup that read traffic should be sent to, query rules or a separate read only user should be defined to route traffic to this hostgroup, nodes that have read_only=1 will be assigned to this hostgroup.
- check_type - the MySQL variable checked when executing a Read Only check, read_only by default (alternatively super_read_only can be used as well). For AWS Aurora innodb_read_only should be used.
- comment - text field that can be used for any purposed defined by the user. Could be a description of what the cluster stores, a reminder of when the hostgroup was added or disabled, or a JSON processed by some checker script.
mysql_group_replication_hostgroups
Table mysql_group_replication_hostgroups defines hostgroups for use with Oracle Group Replication / InnoDB Cluster Admin> show create table mysql_group_replication_hostgroups\\G *************************** 1. row *************************** table: mysql_group_replication_hostgroups Create Table: CREATE TABLE mysql_group_replication_hostgroups ( writer_hostgroup INT CHECK (writer_hostgroup>=0) NOT NULL PRIMARY KEY, backup_writer_hostgroup INT CHECK (backup_writer_hostgroup>=0 AND backup_writer_hostgroup<>writer_hostgroup) NOT NULL, reader_hostgroup INT NOT NULL CHECK (reader_hostgroup<>writer_hostgroup AND backup_writer_hostgroup<>reader_hostgroup AND reader_hostgroup>0), offline_hostgroup INT NOT NULL CHECK (offline_hostgroup<>writer_hostgroup AND offline_hostgroup<>reader_hostgroup AND backup_writer_hostgroup<>offline_hostgroup AND offline_hostgroup>=0), active INT CHECK (active IN (0,1)) NOT NULL DEFAULT 1, max_writers INT NOT NULL CHECK (max_writers >= 0) DEFAULT 1, writer_is_also_reader INT CHECK (writer_is_also_reader IN (0,1)) NOT NULL DEFAULT 0, max_transactions_behind INT CHECK (max_transactions_behind>=0) NOT NULL DEFAULT 0, comment VARCHAR, UNIQUE (reader_hostgroup), UNIQUE (offline_hostgroup), UNIQUE (backup_writer_hostgroup)) 1 row in set (0.00 sec) The fields have the following semantics:- writer_hostgroup - the hostgroup that all traffic will be sent to by default, nodes that have read_only=0 in MySQL will be assigned to this hostgroup.
- backup_writer_hostgroup - if the cluster has multiple nodes with read_only=0 and max_writers, ProxySQL will put the additional nodes (in excess of max_writes) in the backup_writer_hostgroup.
- reader_hostgroup - the hostgroup that read traffic should be sent to, query rules or a separate read only user should be defined to route traffic to this hostgroup, nodes that have read_only=1 will be assigned to this hostgroup.
- offline_hostgroup - when ProxySQL's monitoring determines a node is OFFLINE it will be put into the offline_hostgroup.
- active - when enabled, ProxySQL monitors the hostgroups and moves nodes between the appropriate hostgroups.
- max_writers' - this value determines the maximum number of nodes that should be allowed in the writer_hostgroup, nodes in excess of this value will be put into the backup_writer_hostgroup`
- writer_is_also_reader - determines if a node should be added to the reader_hostgroup as well as the writer_hostgroup after it is promoted.
- max_transactions_behind - determines the maximum number of transactions behind the writers that ProxySQL should allow before shunning the node to prevent stale reads (this is determined by querying the transactions_behind field of the sys.gr_member_routing_candidate_status table in MySQL).
- comment - text field that can be used for any purposed defined by the user. Could be a description of what the cluster stores, a reminder of when the hostgroup was added or disabled, or a JSON processed by some checker script.
mysql_galera_hostgroups
Table mysql_galera_hostgroups (available in ProxySQL 2.x and higher) defines hostgroups for use with Galera Cluster / Percona XtraDB Cluster. Admin> show create table mysql_galera_hostgroups\\G *************************** 1. row *************************** table: mysql_galera_hostgroups Create Table: CREATE TABLE mysql_galera_hostgroups ( writer_hostgroup INT CHECK (writer_hostgroup>=0) NOT NULL PRIMARY KEY, backup_writer_hostgroup INT CHECK (backup_writer_hostgroup>=0 AND backup_writer_hostgroup<>writer_hostgroup) NOT NULL, reader_hostgroup INT NOT NULL CHECK (reader_hostgroup<>writer_hostgroup AND backup_writer_hostgroup<>reader_hostgroup AND reader_hostgroup>0), offline_hostgroup INT NOT NULL CHECK (offline_hostgroup<>writer_hostgroup AND offline_hostgroup<>reader_hostgroup AND backup_writer_hostgroup<>offline_hostgroup AND offline_hostgroup>=0), active INT CHECK (active IN (0,1)) NOT NULL DEFAULT 1, max_writers INT NOT NULL CHECK (max_writers >= 0) DEFAULT 1, writer_is_also_reader INT CHECK (writer_is_also_reader IN (0,1)) NOT NULL DEFAULT 0, max_transactions_behind INT CHECK (max_transactions_behind>=0) NOT NULL DEFAULT 0, comment VARCHAR, UNIQUE (reader_hostgroup), UNIQUE (offline_hostgroup), UNIQUE (backup_writer_hostgroup)) 1 row in set (0.00 sec) The fields have the following semantics:- writer_hostgroup - the hostgroup that all traffic will be sent to by default, nodes that have read_only=0 in MySQL will be assigned to this hostgroup.
- backup_writer_hostgroup - if the cluster has multiple nodes with read_only=0 and max_writers, ProxySQL will put the additional nodes (in excess of max_writes) in the backup_writer_hostgroup.
- reader_hostgroup - the hostgroup that read traffic should be sent to, query rules or a separate read only user should be defined to route traffic to this hostgroup, nodes that have read_only=1 will be assigned to this hostgroup.
- offline_hostgroup - when ProxySQL's monitoring determines a host is OFFLINE it will be put into the offline_hostgroup
- active - when enabled, ProxySQL monitors the hostgroups and moves servers between the appropriate hostgroups.
- max_writers' - this value determines the maximum number of nodes that should be allowed in the writer_hostgroup, nodes in excess of this value will be put into the backup_writer_hostgroup`
- writer_is_also_reader - determines if a node should be added to the reader_hostgroup as well as the writer_hostgroup after it is promoted.
- max_transactions_behind - determines the maximum number of writesets behind the cluster that ProxySQL should allow before shunning the node to prevent stale reads (this is determined by querying the wsrep_local_recv_queue Galera variable).
- comment - text field that can be used for any purposed defined by the user. Could be a description of what the cluster stores, a reminder of when the hostgroup was added or disabled, or a JSON processed by some checker script.
mysql_users
Table mysql_users defines MySQL users, used to connect to backends. Admin> SHOW CREATE TABLE mysql_users\\G *************************** 1. row *************************** table: mysql_users Create Table: CREATE TABLE mysql_users ( username VARCHAR NOT NULL, password VARCHAR, active INT CHECK (active IN (0,1)) NOT NULL DEFAULT 1, use_ssl INT CHECK (use_ssl IN (0,1)) NOT NULL DEFAULT 0, default_hostgroup INT NOT NULL DEFAULT 0, default_schema VARCHAR, schema_locked INT CHECK (schema_locked IN (0,1)) NOT NULL DEFAULT 0, transaction_persistent INT CHECK (transaction_persistent IN (0,1)) NOT NULL DEFAULT 0, fast_forward INT CHECK (fast_forward IN (0,1)) NOT NULL DEFAULT 0, backend INT CHECK (backend IN (0,1)) NOT NULL DEFAULT 1, frontend INT CHECK (frontend IN (0,1)) NOT NULL DEFAULT 1, max_connections INT CHECK (max_connections >=0) NOT NULL DEFAULT 10000, comment VARCHAR NOT NULL DEFAULT '', PRIMARY KEY (username, backend), UNIQUE (username, frontend)) 1 row in set (0.00 sec) The fields have the following semantics:- username, password - credentials for connecting to the mysqld or ProxySQL instance. See also Password management
- active - the users with active = 0 will be tracked in the database, but will be never loaded in the in-memory data structures
- default_hostgroup - if there is no matching rule for the queries send by this users, the traffic is generates is sent to the specified hostgroup
- default_schema - the schema to which the connection should change by default
- schema_locked - not supported yet (TODO: check)
- transaction_persistent - if this is set for the user with which the MySQL client is connecting to ProxySQL (thus a "frontend" user - see below), transactions started within a hostgroup will remain within that hostgroup regardless of any other rules
- fast_forward - if set it bypass the query processing layer (rewriting, caching) and pass through the query directly as is to the backend server.
- frontend - if set to 1, this (username, password) pair is used for authenticating to the ProxySQL instance
- backend - if set to 1, this (username, password) pair is used for authenticating to the mysqld servers against any hostgroup
- max_connections - defines the maximum number of allowable frontend connections for a specific user.
-
comment - text
以上是关于Mysql读写分离—5.7 gtid 主从 + ProxySql 配置及简单测试的主要内容，如果未能解决你的问题，请参考以下文章
Mysql读写分离—5.7 gtid 主从 + ProxySql 配置及简单测试
MHA+ProxySQL，mysql的高可用集群+读写分离GTID主从复制
consul1.6实现Mysql-Gtid主从读写分离-03
maxscale配合MHA搭建读写分离的高可用架构（基于GTID replication主从架构，mysql5.6）