MyCat水平分库

Posted 架构师学习路线

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了MyCat水平分库相关的知识,希望对你有一定的参考价值。


一.什么是水平分库

将一张表水平切分到多个库中
MyCat水平分库

1.1分片原则
  • 1.需要分片的表是少数的

  • 2.能不切分尽量不要切分

  • 3.日志表可以采取归档方式

  • 4.选择合适的切分规则和分片建,确保数据分片均匀,否则依然存在性能瓶颈

  • 5.尽量避免跨分片join操作,保证关联操作表在同一分片

1.2分片后如何处理查询
  • 1.根据简单分片规则,对分片键进行路由到正确的后端物理数据库

  • 2.如果不是使用分片键的话,将会遍历后端数据库,极大消耗性能

二.水平切分步骤

2.1根据业务状态确定要进行水平切分的表

读写频繁,访问量非常大的表才需要切分,一般是订单表

如何选择分片键

  • 1.尽可能的比较均匀分布数据到各个节点上,自增长主键并不是很好的选择,因为不会用于查询

  • 2.业务字段是最频繁的或者最重要的查询条件

2.2分析业务模型选择分片键及分片算法

一般分片键选择的是频繁作为查询的字段,关键是能保证分片后的数据分布均匀,常用简单取模算法

  • 对订单相关表进行水平切分

  • 不仅仅是订单表,经常与订单表关联查询的表也需要一并分片,避免跨分片查询,大表不适合作为全局表

  • 订单号,可以保证分片均匀,但是实际业务很少根据订单号来查询

  • 下单人ID,业务查询更多,更适合,但是分片并不均匀,但不严重,值得考虑

  • 采用简单取模分片算法,可以保证数据尽可能均匀

2.3适用mycat部署分片集群
  • 1.使用schema.xml配置逻辑库及逻辑表

<!-- <table name="order_master" primaryKey="order_id" dataNode="dn_orderdb" /> -->
<table name="order_master" primaryKey="order_id" dataNode="dn_orderdb01,dn_orderdb02,dn_orderdb03,dn_orderdb04" rule="order_master"/>
<!-- 只需要增加表所在的物理节点schema名称和指定规则rule -->

<dataNode name="dn_orderdb01" dataHost="mysql10143" database="orderdb01" />
<dataNode name="dn_orderdb02" dataHost="mysql10143" database="orderdb02" />
<dataNode name="dn_orderdb03" dataHost="mysql10144" database="orderdb03" />
<dataNode name="dn_orderdb04" dataHost="mysql10144" database="orderdb04" />
  • 2.使用rule.xml配置分片表的分片规则

    <tableRule name="order_master">
<rule>
<columns>customer_id</columns>
<algorithm>mod-long</algorithm>
</rule>
</tableRule>
<function name="mod-long" class="org.opencloudb.route.function.PartitionByMod">
<!-- 分片算法,唯一;JAVA类路径,1.6是"io.mycat.route.function.PartitionByMod"-->
<property name="count">4</property>
</function>
  • 3.验证

# 现在逻辑库上进行查看,此时是没有数据的
# 需要提前建库建表,否则报错表不存在
app_imooc@172.16.10.142 00:26: [imooc_db]> select * from order_master;
Empty set (0.08 sec)

# 在逻辑库imooc_db上插入5条数据,正常是orderdb02 2条数据(节点索引顺序为1),其他各1条数据
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70001,90001,'yzw1',1,1,1,'address1',1,20);
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70002,90002,'yzw2',2,2,2,'address1',1,20);
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70003,90003,'yzw3',3,3,3,'address1',1,20);
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70004,90004,'yzw4',4,4,4,'address1',1,20);
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70005,90005,'yzw5',5,5,5,'address1',1,20);

# truncate table报错
app_imooc@172.16.10.142 10:51: [imooc_db]> truncate table order_master;
ERROR 1105 (HY000): DROP command denied to user 'bm_mycat'@'172.16.10.142' for table 'order_master'

# 逻辑库查看插入结果
app_imooc@172.16.10.142 10:50: [imooc_db]> select order_id,order_sn,customer_id from order_master;
+----------+----------+-------------+
| order_id | order_sn | customer_id |
+----------+----------+-------------+
| 80005 | 70002 | 90002 |
| 80005 | 70003 | 90003 |
| 80005 | 70004 | 90004 |
| 80006 | 70001 | 90001 |
| 80007 | 70005 | 90005 |
+----------+----------+-------------+
5 rows in set (0.00 sec)

# 在物理库查看是否有这4条数据,验证成功
root@localhost 10:52: [orderdb01]> select order_id,order_sn,customer_id from order_master;
+----------+----------+-------------+
| order_id | order_sn | customer_id |
+----------+----------+-------------+
| 80005 | 70004 | 90004 |
+----------+----------+-------------+
1 row in set (0.00 sec)

root@localhost 10:52: [orderdb02]> select order_id,order_sn,customer_id from order_master;
+----------+----------+-------------+
| order_id | order_sn | customer_id |
+----------+----------+-------------+
| 80006 | 70001 | 90001 |
| 80007 | 70005 | 90005 |
+----------+----------+-------------+
2 rows in set (0.00 sec)

root@localhost 10:05: [orderdb03]> select order_id,order_sn,customer_id from order_master;

+----------+----------+-------------+
| order_id | order_sn | customer_id |
+----------+----------+-------------+
| 80005 | 70002 | 90002 |
+----------+----------+-------------+
1 row in set (0.00 sec)

root@localhost 10:52: [orderdb04]> select order_id,order_sn,customer_id from order_master;
+----------+----------+-------------+
| order_id | order_sn | customer_id |
+----------+----------+-------------+
| 80005 | 70003 | 90003 |
+----------+----------+-------------+
1 row in set (0.00 sec)
  • 4.使用server.xml配置访问权限

#配置用户登录
2.4测试分片集群,采用应用端双写方式进行
2.5业务及数据迁移

三.全局自增ID

  • 分片表中的自增ID在逻辑表中有重复

  • 第三方给ID或者使用mycat自增ID

3.1全局自增ID方法
  • 1.本地文件方式:适用服务器本地磁盘的方式

  • 2.数据库方式:适用数据库存储的方式

  • 3.本地时间戳方式:适用时间戳

  • 4.分布式zookeeper生成ID

3.2本地文件全局ID
  • 优点:本地加载,读取速度快,配置简单

  • 缺点:集群部署无法使用,不同的mycat无法保证id唯一,使mycat变成了有状态的中间件

配置方法
  • 1.server.xml增加属性

<property name="sequnceHandlerType>0</property>
  • 2.sequence_conf.properties配置

#default global sequence
GLOBAL.HISIDS=
GLOBAL.MINID=10001
GLOBAL.MAXID=20000
GLOBAL.CURID=10000

# self define sequence
ORDER_MASTER.HISIDS=
ORDER_MASTER.MINID=1001
ORDER_MASTER.MAXID=2000
ORDER_MASTER.CURID=1000


# 以上配置文件中,自定义表名必须大写书写HISIDS:表示使用过的历史分段(一般无特殊需要可不配置)
# MINID :最小ID 值
# MAXID :表示最大ID 值
# CURID :表示当前ID 值。
# 当 sequence_conf.properties的配置名字与 表名一致的时候sql可以不包含ID字段(此处表名为id_local_file),逻辑表需要增加属性autoIncrement="true"
  • 3.插入数据

# mycat restart
# delete from order_master;
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70001,90001,'yzw1',1,1,1,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70002,90002,'yzw2',2,2,2,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70003,90003,'yzw3',3,3,3,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70004,90004,'yzw4',4,4,4,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70005,90005,'yzw5',5,5,5,'address1',1,20);
# 此时逻辑表中的order_id唯一
app_imooc@172.16.10.142 11:35: [imooc_db]> select order_id,order_sn,customer_id from order_master;
+----------+----------+-------------+
| order_id | order_sn | customer_id |
+----------+----------+-------------+
| 10004 | 70004 | 90004 |
| 10003 | 70003 | 90003 |
| 10001 | 70001 | 90001 |
| 10005 | 70005 | 90005 |
| 10002 | 70002 | 90002 |
+----------+----------+-------------+
5 rows in set (0.01 sec)

insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70001,90001,'yzw1',1,1,1,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70002,90002,'yzw2',2,2,2,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70003,90003,'yzw3',3,3,3,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70004,90004,'yzw4',4,4,4,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70005,90005,'yzw5',5,5,5,'address1',1,20);

# 此时逻辑表中的order_id依然是唯一,但是ID值使用的是自定义表自增属性的值
app_imooc@172.16.10.142 11:48: [imooc_db]> select order_id,order_sn,customer_id from order_master;
+----------+----------+-------------+
| order_id | order_sn | customer_id |
+----------+----------+-------------+
| 1009 | 70003 | 90003 |
| 1008 | 70002 | 90002 |
| 1007 | 70001 | 90001 |
| 1011 | 70005 | 90005 |
| 1010 | 70004 | 90004 |
+----------+----------+-------------+
5 rows in set (0.00 sec)
# 不指定自增字段插入
# 逻辑表必须增加自增属性,否则不指定自增字段插入则使用分片表本身自增id
#<table name="order_master" primaryKey="order_id" autoIncrement="true" dataNode="dn_orderdb01,dn_orderdb02,dn_orderdb03,dn_orderdb04" rule="order_master"/>
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70001,90001,'yzw1',1,1,1,'address1',1,20);
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70002,90002,'yzw2',2,2,2,'address1',1,20);
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70003,90003,'yzw3',3,3,3,'address1',1,20);
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70004,90004,'yzw4',4,4,4,'address1',1,20);
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70005,90005,'yzw5',5,5,5,'address1',1,20);

app_imooc@172.16.10.142 11:58: [imooc_db]> select order_id,order_sn,customer_id from order_master;
+----------+----------+-------------+
| order_id | order_sn | customer_id |
+----------+----------+-------------+
| 1001 | 70001 | 90001 |
| 1005 | 70005 | 90005 |
| 1003 | 70003 | 90003 |
| 1004 | 70004 | 90004 |
| 1002 | 70002 | 90002 |
+----------+----------+-------------+
5 rows in set (0.01 sec)
3.3数据库方式
  • 优点:mycat重启后,sequence值不会被初始化,因为从数据库中取,每次取完按自增步长增加

  • 缺点:sequence数据库主从切换后,存在mycat适用缓存序列号,可能存在ID重复风险,可以手动增加步长避免

配置方法
  • 1.server.xml增加属性

<property name="sequnceHandlerType">1</property>
  • 2.sequence_db_conf.properties

#sequence stored in datanode
GLOBAL=dn_test
ORDER_MASTER=dn_test
datanode必须要mycat能访问的数据库,并不一定需要在分片表的数据库中
  • 3.schema.xml增加主机节点和数据节点,让mycat能访问上面配置的数据节点

  • 4.在对应节点的数据库中增加函数和表,用户必须授予表、函数、存储过程的访问权限

# 需要全局ID的表明需要写入表中,大写
INSERT INTO mycat_sequence values('ORDER_MASTER', 1, 100);
root@master 11:48: [test]> select * from mycat_sequence;
+--------------+---------------+-----------+
| NAME | current_value | increment |
+--------------+---------------+-----------+
| GLOBAL | 100000 | 100 |
| ORDER_MASTER | 1 | 100 |
+--------------+---------------+-----------+
2 rows in set (0.00 sec)
  • 5.schema.xml增加配置,适用全局自增IDautoIncrement="true"

<table name="order_master" primaryKey="order_id" autoIncrement="true" dataNode="dn_orderdb01,dn_orderdb02,dn_orderdb03,dn_orderdb04" rule="order_master"/>
  • 6.在逻辑库中插入数据

# 适用ID表里面的全局ID插入
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70001,90001,'yzw1',1,1,1,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70002,90002,'yzw2',2,2,2,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70003,90003,'yzw3',3,3,3,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70004,90004,'yzw4',4,4,4,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70005,90005,'yzw5',5,5,5,'address1',1,20);
# 使用ID表里面的对应表ID插入
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70001,90001,'yzw1',1,1,1,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70002,90002,'yzw2',2,2,2,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70003,90003,'yzw3',3,3,3,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70004,90004,'yzw4',4,4,4,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70005,90005,'yzw5',5,5,5,'address1',1,20);
# 不指定id列,效果同上
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70001,90001,'yzw1',1,1,1,'address1',1,20);
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70002,90002,'yzw2',2,2,2,'address1',1,20);
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70003,90003,'yzw3',3,3,3,'address1',1,20);
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70004,90004,'yzw4',4,4,4,'address1',1,20);
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70005,90005,'yzw5',5,5,5,'address1',1,20);
3.4本地时间戳方式

ID= 64 位二进制 (42(毫秒)+5(机器 ID)+5(业务编码)+12(重复累加)

  • 优点:与mycat无关,跟时间有关系

  • 缺点:字段最大长度必须大于等于18,int无法满足

配置方法
  • 1.server.xml增加属性

<property name="sequnceHandlerType">2</property>
  • 2.sequence_time_conf.properties

#sequence depend on TIME
WORKID=01
DATAACENTERID=01

# 0-31任意整数
  • 3.在对应节点的数据库中增加函数和表,用户必须授予表、函数、存储过程的访问权限

# 需要全局ID的表明需要写入表中,大写
INSERT INTO mycat_sequence values('ORDER_MASTER', 1, 100);
root@master 11:48: [test]> select * from mycat_sequence;
+--------------+---------------+-----------+
| NAME | current_value | increment |
+--------------+---------------+-----------+
| GLOBAL | 100000 | 100 |
| ORDER_MASTER | 1 | 100 |
+--------------+---------------+-----------+
2 rows in set (0.00 sec)
  • 5.schema.xml增加配置,适用全局自增IDautoIncrement="true"

<table name="order_master" primaryKey="order_id" autoIncrement="true" dataNode="dn_orderdb01,dn_orderdb02,dn_orderdb03,dn_orderdb04" rule="order_master"/>
  • 6.在逻辑库中插入数据

# int无法满足
ERROR 1264 (22003): Out of range value for column 'order_id' at row

app_imooc@172.16.10.142 12:32: [imooc_db]> select next value for MYCATSEQ_ORDER_MASTER;
+---------------------+
| 1025962766269288448 |
+---------------------+
| 1025962766269288448 |
+---------------------+
1 row in set (0.00 sec)

app_imooc@172.16.10.142 12:32: [imooc_db]> select next value for MYCATSEQ_GLOBAL;
+---------------------+
| 1025962827724230656 |
+---------------------+
| 1025962827724230656 |
+---------------------+
1 row in set (0.00 sec)
1
# 适用ID表里面的全局ID插入
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70001,90001,'yzw1',1,1,1,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70002,90002,'yzw2',2,2,2,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70003,90003,'yzw3',3,3,3,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70004,90004,'yzw4',4,4,4,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_GLOBAL,70005,90005,'yzw5',5,5,5,'address1',1,20);
# 使用ID表里面的对应表ID插入
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70001,90001,'yzw1',1,1,1,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70002,90002,'yzw2',2,2,2,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70003,90003,'yzw3',3,3,3,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70004,90004,'yzw4',4,4,4,'address1',1,20);
insert into order_master(order_id,order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(next value for MYCATSEQ_ORDER_MASTER,70005,90005,'yzw5',5,5,5,'address1',1,20);
# 不指定id列,效果同上
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70001,90001,'yzw1',1,1,1,'address1',1,20);
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70002,90002,'yzw2',2,2,2,'address1',1,20);
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70003,90003,'yzw3',3,3,3,'address1',1,20);
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70004,90004,'yzw4',4,4,4,'address1',1,20);
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70005,90005,'yzw5',5,5,5,'address1',1,20);

select order_id,order_sn,customer_id from order_master;
3.5分布式zookeeper生成ID 未测试,1.5没有默认配置文件

基于ZK 与本地配置的分布式ID 生成器(可以通过ZK 获取集群(机房)唯一InstanceID,也可以通过配置文件配置InstanceID)ID 结构:long 64 位,ID 最大可占63 位
current time millis(微秒时间戳38 位,可以使用17 年)
instanceId(实例ID,可以通过ZK 或者配置文件获取,5 位,也就是十进制0-31)
threadId(线程ID,9 位)
increment(自增,6 位)
一共63 位,可以承受单机房单机器单线程1000*(2^6)=640000 的并发。

  • 优点:无悲观锁,无强竞争,吞吐量更高

  • 缺点:对zookeeper集群的要求增加。

配置方法
  • 1.server.xml增加属性

<property name="sequnceHandlerType">3</property>
  • 2.sequence_distributed_conf.properties

INSTANCEID=01
CLUSTERID=01
  • 3.在对应节点的数据库中增加函数和表,用户必须授予表、函数、存储过程的访问权限

# 需要全局ID的表明需要写入表中,大写
INSERT INTO mycat_sequence values('ORDER_MASTER', 1, 100);
root@master 11:48: [test]> select * from mycat_sequence;
+--------------+---------------+-----------+
| NAME | current_value | increment |
+--------------+---------------+-----------+
| GLOBAL | 100000 | 100 |
| ORDER_MASTER | 1 | 100 |
+--------------+---------------+-----------+
2 rows in set (0.00 sec)
  • 5.schema.xml增加配置,适用全局自增IDautoIncrement="true"

<table name="order_master" primaryKey="order_id" autoIncrement="true" dataNode="dn_orderdb01,dn_orderdb02,dn_orderdb03,dn_orderdb04" rule="order_master"/>
  • 6.在逻辑库中插入数据

四.ER关系

  • 跨分片查询存在跨节点问题

app_imooc@172.16.10.142 13:55:  [imooc_db]> select * from order_master m join order_detail o on m.order_id=o.order_id;
ERROR 1064 (HY000): invalid route in sql, multi tables found but datanode has no intersection sql:select * from order_master m join order_detail o on m.order_id=o.order_id
  • 根据ER关系,也需要对order_detail这张表进行分片

4.1配置步骤
  • 1.在原先order_master所在4个分配数据库(orderdb01/02/03/04)中建立order_detail表结构

CREATE TABLE `order_detail` (
`order_detail_id` int(10) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增主键ID,订单详情表ID',
`order_id` int(10) unsigned NOT NULL COMMENT '订单表ID',
`product_id` int(10) unsigned NOT NULL COMMENT '订单商品ID',
`product_name` varchar(50) NOT NULL COMMENT '商品名称',
`product_cnt` int(11) NOT NULL DEFAULT '1' COMMENT '购买商品数量',
`product_price` decimal(8,2) NOT NULL COMMENT '购买商品单价',
`average_cost` decimal(8,2) NOT NULL DEFAULT '0.00' COMMENT '平均成本价格',
`weight` float DEFAULT NULL COMMENT '商品重量',
`fee_money` decimal(8,2) NOT NULL DEFAULT '0.00' COMMENT '优惠分摊金额',
`w_id` int(10) unsigned NOT NULL COMMENT '仓库ID',
`modified_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '最后修改时间',
PRIMARY KEY (`order_detail_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='订单详情表';
  • 2.schema.xml将全局分片表进行ER分片,在父表中增加子表信息

<table name="order_master" primaryKey="order_id" autoIncrement="true" dataNode="dn_orderdb01,dn_orderdb02,dn_orderdb03,dn_orderdb04" rule="order_master">
<childTable name="order_detail" primaryKey="order_detail_id" joinKey="order_id" parentKey="order_id" autoIncrement="true" />
<!-- 指定关联子表的表名、主键,及父表关联key、父表的主键,同时也需要指定全局ID -->
</table>
  • 3.配置表order_detail全局自增ID(使用数据库方式)

# server.xml

# sequence_db_conf.properties增加order_detail自增ID
#sequence stored in datanode
GLOBAL=dn_test
ORDER_MASTER=dn_test
ORDER_DETAIL=dn_test
# 数据库中增加分片表信息
insert into mycat_sequence values('ORDER_DETAIL',1,1);
  • 4.插入数据

# 先插入order_master
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70001,90001,'yzw1',1,1,1,'address1',1,20);
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70002,90002,'yzw2',2,2,2,'address1',1,20);
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70003,90003,'yzw3',3,3,3,'address1',1,20);
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70004,90004,'yzw4',4,4,4,'address1',1,20);
insert into order_master(order_sn,customer_id,shipping_user,province,city,district,address,payment_method,order_money) values(70005,90005,'yzw5',5,5,5,'address1',1,20);
app_imooc@172.16.10.142 14:46: [imooc_db]> select order_sn,customer_id,order_id from order_master;
+----------+-------------+----------+
| order_sn | customer_id | order_id |
+----------+-------------+----------+
| 70005 | 90005 | 5 |
| 70001 | 90001 | 6 |
| 70005 | 90005 | 10 |
| 70004 | 90004 | 4 |
| 70004 | 90004 | 9 |
| 70003 | 90003 | 3 |
| 70003 | 90003 | 8 |
| 70002 | 90002 | 2 |
| 70002 | 90002 | 7 |
+----------+-------------+----------+
9 rows in set (0.00 sec)

# 根据order_id插入order_detail
insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,2,12,'商品2',1,22.2,12,1,10,2);
insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,3,13,'商品3',1,22.2,12,1,10,2);
insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,4,14,'商品4',1,22.2,12,1,10,2);
insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,5,15,'商品5',1,22.2,12,1,10,2);
insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,6,16,'商品6',1,22.2,12,1,10,2);
insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,7,17,'商品7',1,22.2,12,1,10,2);
insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,8,18,'商品8',1,22.2,12,1,10,2);
insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,9,19,'商品9',1,22.2,12,1,10,2);
insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,10,20,'商品10',1,22.2,12,1,10,2);
insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,5,21,'商品11',1,22.2,12,1,10,2);
insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,6,12,'商品2',1,22.2,12,1,10,2);
insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,8,12,'商品2',1,22.2,12,1,10,2);
insert into order_detail(order_detail_id,order_id,product_id,product_name,product_cnt,product_price,average_cost,weight,fee_money,w_id) values(next value for MYCATSEQ_ORDER_DETAIL,8,12,'商品2',1,22.2,12,1,10,2);
  • 5.此时再进行分片关联查询验证数据

select m.order_id,o.order_detail_id,o.product_id,o.product_name from order_master m join order_detail o on m.order_id=o.order_id;
+----------+-----------------+------------+--------------+
| order_id | order_detail_id | product_id | product_name |
+----------+-----------------+------------+--------------+
| 3 | 4 | 13 | 商品3 |
| 8 | 9 | 18 | 商品8 |
| 8 | 14 | 12 | 商品2 |
| 8 | 15 | 12 | 商品2 |
| 2 | 3 | 12 | 商品2 |
| 7 | 8 | 17 | 商品7 |
| 4 | 5 | 14 | 商品4 |
| 9 | 10 | 19 | 商品9 |
| 5 | 6 | 15 | 商品5 |
| 6 | 7 | 16 | 商品6 |
| 10 | 11 | 20 | 商品10 |
| 5 | 12 | 21 | 商品11 |
| 6 | 13 | 12 | 商品2 |
+----------+-----------------+------------+--------------+
13 rows in set (0.00 sec)
  • 6.在其中一个分片上可以验证,order_id一致的表落在同一个分片

root@localhost 15:05:  [orderdb02]> select order_sn,customer_id,order_id,mod(customer_id,4) from order_master;
+----------+-------------+----------+
| order_sn | customer_id | order_id |
+----------+-------------+----------+
| 70005 | 90005 | 5 |
| 70001 | 90001 | 6 |
| 70005 | 90005 | 10 |
+----------+-------------+----------+
3 rows in set (0.00 sec)

root@localhost 15:06: [orderdb02]> select order_detail_id,product_id,order_id from order_detail;
+-----------------+------------+----------+
| order_detail_id | product_id | order_id |
+-----------------+------------+----------+
| 6 | 15 | 5 |
| 7 | 16 | 6 |
| 11 | 20 | 10 |
| 12 | 21 | 5 |
| 13 | 12 | 6 |
+-----------------+------------+----------+
5 rows in set (0.00 sec)



原文:https://www.cnblogs.com/jenvid/p/10180461.html

MyCat水平分库
MyCat水平分库
想要获取学习实战、高并发、架构 、笔试面试资料
请扫码咨询+薇薇微信

以上是关于MyCat水平分库的主要内容,如果未能解决你的问题,请参考以下文章

MyCat水平分库

mycat 垂直分库与水平分表使用详解

Docker安装Mycat和Mysql进行水平分库分表实战图文教学

《SQL与数据库基础》21. 分库分表

Mycat+分库分表

mycat:水平分库之按月分库