初探DM迁移MySQL到TIDB
Posted IT那活儿
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了初探DM迁移MySQL到TIDB相关的知识，希望对你有一定的参考价值。
DM部署方式有DM-Ansible、Binary和TIUP方式，TIUP为最新的部署方式，更加便捷。
以下步骤为使用DM-Ansible方式部署过程。
1. 安装依赖包
▼▼▼
yum -y install epel-release git curl sshpass
wget https://bootstrap.pypa.io/pip/2.7/get-pip.py
python get-pip.py
wget https://download.pingcap.org/dm-ansible-v2.0.0.tar.gz
[tidb@luyy01 ~]$ tar -zxvf dm-ansible-v2.0.0.tar.gz
[tidb@luyy01 ~]$ mv dm-ansible-v2.0.0 dm-ansible
[tidb@luyy01 ~]$sudo pip install -r ./requirements.txt
2. 在中控机上配置 SSH 互信和 sudo 规则
▼▼▼
cd /home/tidb/dm-ansible
vi hosts.ini
[servers]
192.168.199.62
192.168.199.63
192.168.199.64
[all:vars]
username = tidb
[tidb@luyy01 dm-ansible]$ ansible-playbook -i hosts.ini create_users.yml -u root -k
SSH password:
3. 下载 DM 及监控组件安装包至中控机
4. 编辑 inventory.ini 配置文件
▼▼▼
## DM modules
[dm_master_servers]
dm_master ansible_host=192.168.199.61 dm_master_port=18261
[dm_worker_servers]
dm-worker1 ansible_host=192.168.199.62 dm_worker_port=18262 source_id="mysql-replica-01" mysql_host=192.168.199.61 mysql_user=admin mysql_password='YUZP1pa88fWbIzudw3bDTEBF6ZUTnw==' mysql_port=3306
[dm_portal_servers]
dm_portal ansible_host=192.168.199.61
## Monitoring modules
[prometheus_servers]
prometheus ansible_host=192.168.199.63
[grafana_servers]
grafana ansible_host=192.168.199.63
[alertmanager_servers]
alertmanager ansible_host=192.168.199.63
## Global variables
[all:vars]
cluster_name = test-cluster
ansible_user = tidb
dm_version = v2.0.0
deploy_dir = /home/tidb/deploy
grafana_admin_user = "admin"
grafana_admin_password = "admin"
5. 使用 dmctl 加密上游 MySQL 用户密码
▼▼▼
cd /home/tidb/dm-ansible/resources/bin &&
./dmctl -encrypt 'abc!@#123
6.修改内核参数并部署 DM 集群组件和监控组件
7. 启动DM集群
实例 |
主机 |
用户名 |
密码 |
端口 |
上游 MySQL |
192.***.***.** |
admin |
admin |
3306 |
下游 TiDB |
192.***.***.** |
root |
4000 |
1. dm-master 进程配置
▼▼▼
cd /home/tidb/deploy/conf
vi dm-master.toml
enable-gtid = false
[[deploy]]
source-id = "mysql-replica-01"
dm-worker = "192.168.199.62:18262"
2. 配置任务
登录网页http://192.***.***.**:8280/
可手动创建task文件
▼▼▼
name: luyayao
task-mode: all
is-sharding: false
clean-dump-file: true
ignore-checking-items: [schema_of_shard_tables]
target-database:
host: 192.168.199.63
port: 4000
user: root
password: KyMi25sGljNJPVL8OV3JrEA=
mysql-instances:
- source-id: mysql-replica-01
meta:
binlog-name: mysql-bin.0000001
binlog-pos: 4
filter-rules: []
route-rules:
- mysql-replica-01.route_rules.1
- mysql-replica-01.route_rules.2
black-white-list: mysql-replica-01.bw_list.1
mydumper-config-name: mysql-replica-01.dump
routes:
mysql-replica-01.route_rules.1:
schema-pattern: employees
table-pattern: ""
target-schema: employees
target-table: ""
mysql-replica-01.route_rules.2:
schema-pattern: employees
table-pattern: "*"
target-schema: employees
filters: {}
black-white-list:
mysql-replica-01.bw_list.1:
do-tables:
- db-name: employees
tbl-name: "*"
do-dbs: []
ignore-tables: []
ignore-dbs: []
mydumpers:
mysql-replica-01.dump:
mydumper-path: bin/mydumper
threads: 4
chunk-filesize: 64
skip-tz-utc: true
extra-args: "--no-locks"
3. 进入交互窗口
▼▼▼
./dmctl -master-addr 192.168.199.61:18261
operate-source create /home/tidb/deploy/conf/dm-master.yaml
▼▼▼
cat dm-master.yaml
server-id: 101
source-id: mysql-replica-01
flavor: mysql
enable-relay: false
enable-gtid: false
from:
host: 192.168.199.61
user: admin
password: YUZP1pa88fWbIzudw3bDTEBF6ZUTnw==
port: 3306
4. 开始任务
▼▼▼
start-task -s mysql-replica-01 /home/tidb/deploy/conf/luyayao.yaml
» query-status #查询任务运行状态
5. 日志查看
ps -ef|grep dm 可查看进程中dm-master 和 dm-work中的日志
通过查看日志和登录TIDB数据库发现数据已从DM完整迁移了过来。
更多精彩干货分享
点击下方名片关注
IT那活儿