通过binlog统计DML操作的次数

Posted 2021-02-21

解析分析线上30分中26G的binlog，排查阿里云RDS主库插入ROWS和监控数据不匹配问题。
解析binlog汇总到一起：

for i in ls /mnt/bi/;do
/opt/software/mysql-8.0.19-linux-glibc2.12-x86_64/bin/mysqlbinlog -v --base64-output=decode-rows $i >> all_decoded.sql
done

对解析的binlog进行INSERT筛选按执行次数排序统计：

[root@backup bi_decode]# grep -i ‘INSERT INTO datacenter‘ all_decoded.sql|sort -n|uniq -c|sort -n -k 1 -r
4924224 ### INSERT INTO datacenter.fr_mid_go_up_log
3053223 ### INSERT INTO datacenter.fr_mid_shop_station_go_up
2376044 ### INSERT INTO datacenter.mid_small_big_station_heart
878781 ### INSERT INTO datacenter.fr_station_battery_monitor_his_20200429
611570 ### INSERT INTO datacenter.fr_mid_station_borrow_count
603260 ### INSERT INTO datacenter.fr_mid_station_profit
478563 ### INSERT INTO datacenter.fr_station_lost_detail
184708 ### INSERT INTO datacenter.fr_station_seller_monitor_v2
106951 ### INSERT INTO datacenter.empty_battery_station_detail
61646 ### INSERT INTO datacenter.fr_station_seller_monitor_his
58213 ### INSERT INTO datacenter.fr_station_region_monitor_v2
30290 ### INSERT INTO datacenter.fr_mid_orgr2user
22269 ### INSERT INTO datacenter.fr_mid_seller_trade_info
19425 ### INSERT INTO datacenter.fr_station_region_monitor_his
17814 ### INSERT INTO datacenter.fr_mid_ycb_manager_organization_rela_all_03