利用yugong将mysql的数据迁移到mycat

Posted 2022-01-28 w329636271

一.下载jar包

jar包地址

https://download.csdn.net/download/w329636271/21121870

命令行执行:

java -jar yugong-shaded.jar -c conf\\\\5_mysql-mysql-full-sync.properties -y conf\\\\5_mysql-mysql-full-trans.yaml

2.对应配置文件修改

5_mysql-mysql-full-sync.properties如下:

改配置文件,主要配置源数据库(mysql)的信息,以及目标数据库(mycat)的信息,以及需要迁移的表名(yugong.table.white),主要迁移两个表fenpian_0,fenpian

#源数据库
yugong.database.source.url=jdbc:mysql://localhost:3307/db2_0?characterEncoding=utf8&useSSL=false&serverTimezone=Asia/Shanghai&allowPublicKeyRetrieval=true
yugong.database.source.username=root
yugong.database.source.password=123456
yugong.database.source.type=MYSQL
yugong.database.source.encode=UTF-8
yugong.database.source.poolSize=200
#目标数据库
yugong.database.target.url=jdbc:mysql://localhost:8066/mysql?characterEncoding=utf8&useSSL=false&serverTimezone=Asia/Shanghai&allowPublicKeyRetrieval=true
yugong.database.target.username=root
yugong.database.target.password=123456
yugong.database.target.type=MYSQL
yugong.database.target.encode=UTF-8
yugong.database.target.poolSize=200


#yugong.table.batchApply=true  # TODO
yugong.table.batchApply=true
yugong.table.onceCrawNum=1000
yugong.table.tpsLimit=0
# 忽略源表pk检查的表，多表用英文逗号分隔
#yugong.table.ignorePkInspection=fenpian
# 分别列出需要指定的主键或联合主键
#yugong.table.ignorePkInspection.ShopOrderDetail=OrderId,ProductId
#yugong.table.ignorePkInspection.ShopOrderDetailMulti=OrderId,ProductId,MultiProductId

# use connection default schema
yugong.table.ignoreSchema=false
# skip Applier Load Db failed data
yugong.table.skipApplierException=false
#需要迁移的数据库表
yugong.table.white=fenpian_0,fenpian
yugong.table.black=
# tables use multi-thread enable or disable
# TODO test concurrent
yugong.table.concurrent.enable=true
# tables use multi-thread size
yugong.table.concurrent.size=6
# retry times
yugong.table.retry.times=3
# retry interval or sleep time (ms)
yugong.table.retry.interval=1000
# MARK/FULL/INC/ALL(REC+FULL+INC)/CHECK/CLEAR
yugong.table.mode=FULL


# yugong extractor
yugong.extractor.dump=false
yugong.extractor.concurrent.enable=true
yugong.extractor.concurrent.global=false
yugong.extractor.concurrent.size=60
yugong.extractor.noupdate.sleep=1000
yugong.extractor.noupdate.thresold=0
yugong.extractor.once=false

# yugong applier
yugong.applier.concurrent.enable=true
yugong.applier.concurrent.global=false
yugong.applier.concurrent.size=20
yugong.applier.dump=false
# stats
yugong.stat.print.interval=5
yugong.progress.print.interval=1
# alarm email
yugong.alarm.email.host=
yugong.alarm.email.username=
yugong.alarm.email.password=
yugong.alarm.email.stmp.port=
yugong.alarm.email.ssl.support=
yugong.alarm.receiver=

#yugong.cdc.time.start=2020-08-

 5_mysql-mysql-full-trans.yaml

配置shema_to为需要存储的数据库为目标库的db2.

表fenpian_0迁移到目标库的db2的fenpian2中,并且忽略字段fee

表fenpian默认配置直接迁移到db2的fenpian中.

注:column_alias可以进行字段名称的修改

databases:
  source:
    schema:
  target:
    schema:
table:
  a: todo
  b: todo
extractor:
  a: todo
  b: todo
appiler:
  a: todo
  b: todo
translators:
  table:
    '*':
      - class: com.taobao.yugong.translator.NameTableMetaTranslator
        properties:
          column_case_format_from: UPPER_CAMEL
          column_case_format_to: LOWER_UNDERSCORE
          table_case_format_from: UPPER_CAMEL
          table_case_format_to: LOWER_UNDERSCORE
    ProductProperty: []
  record:
    '|BEFORE|':
      - class: com.taobao.yugong.translator.NameStyleDataTranslator
        properties:
          column_case_format_from: UPPER_CAMEL
          column_case_format_to: LOWER_UNDERSCORE
          table_case_format_from: UPPER_CAMEL
          table_case_format_to: LOWER_UNDERSCORE
          schema_to: db2
    fenpian_0:
      - class: com.taobao.yugong.translator.NameStyleDataTranslator
        properties:
          table_to: fenpian2
      - class: com.taobao.yugong.translator.ColumnFixDataTranslator
        properties:
#          column_alias:
#            headimgurl:
#              - head_url
#            joindate:
#              - create_time
          exclude_columns:
#            - timestamp
            - fee

git地址: https://github.com/zwyqz/mycat-yugong