提取mysql binlog日志内容

Posted lijingkuan

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了提取mysql binlog日志内容相关的知识，希望对你有一定的参考价值。

整理自网络

（mysqlbinlog命令 -v 显示每个字段插入的值， -v -v显示每个字段的注释，包括数据类型）


MySQL binlog日志记录了MySQL数据库从启用日志以来所有对当前数据库的变更。

binlog日志属于二进制文件，实现基于时间点的恢复或者不完全恢复时必须要用到二进制日志。

下面给出了提取binlog日志的例子：

1.查看mysql数据库版本

mysql> show variables like 'version';
+---------------+------------+
| Variable_name | Value      |
+---------------+------------+
| version       | 5.6.28-log |
+---------------+------------+
1 row in set (0.00 sec)

2.查看binlog格式 

mysql> show variables like 'binlog_format';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| binlog_format | ROW   |
+---------------+-------+
1 row in set (0.00 sec)

3.查看当前binlog文件，binlog列表最后一行。 

mysql> show binary logs;
+------------------+-----------+
| Log_name         | File_size |
+------------------+-----------+
| mysql-bin.000001 |     65426 |
| mysql-bin.000002 |   1176885 |
| mysql-bin.000003 |      2064 |
| mysql-bin.000004 |       120 |
| mysql-bin.000005 |      2848 |
| mysql-bin.000006 |       975 |
| mysql-bin.000007 |       631 |
| mysql-bin.000008 |       191 |
+------------------+-----------+
8 rows in set (0.00 sec)


mysql> show master status\\G
*************************** 1. row ***************************
             File: mysql-bin.000008
         Position: 191
     Binlog_Do_DB: 
 Binlog_Ignore_DB: 
Executed_Gtid_Set: 5933d78c-a22b-11e5-af00-0800273fac50:1-13
1 row in set (0.00 sec)


4.创建表，插入数据。 

create table binlogtest(id serial,name varchar(20));
    insert into binlogtest values (1,'lijingkuan');
    insert into binlogtest values (2,'test')

5.在数据库中查询mysql binlog中的events；

show binlog events from 191？应该可以，但运行出错，原因未知。


mysql> show binlog events in 'mysql-bin.000008';
+------------------+-----+----------------+-----------+-------------+--------------------------------------------------------------------+
| Log_name         | Pos | Event_type     | Server_id | End_log_pos | Info                                                               |
+------------------+-----+----------------+-----------+-------------+--------------------------------------------------------------------+
| mysql-bin.000008 |   4 | Format_desc    |       123 |         120 | Server ver: 5.6.28-log, Binlog ver: 4                              |
| mysql-bin.000008 | 120 | Previous_gtids |       123 |         191 | 5933d78c-a22b-11e5-af00-0800273fac50:1-13                          |
| mysql-bin.000008 | 191 | Gtid           |       123 |         239 | SET @@SESSION.GTID_NEXT= '5933d78c-a22b-11e5-af00-0800273fac50:14' |
| mysql-bin.000008 | 239 | Query          |       123 |         368 | use `lucien`; create table binlogtest(id serial,name varchar(20))  |
| mysql-bin.000008 | 368 | Gtid           |       123 |         416 | SET @@SESSION.GTID_NEXT= '5933d78c-a22b-11e5-af00-0800273fac50:15' |
| mysql-bin.000008 | 416 | Query          |       123 |         490 | BEGIN                                                              |
| mysql-bin.000008 | 490 | Rows_query     |       123 |         560 | # insert into binlogtest values (1,'lijingkuan')                   |
| mysql-bin.000008 | 560 | Table_map      |       123 |         618 | table_id: 71 (lucien.binlogtest)                                   |
| mysql-bin.000008 | 618 | Write_rows     |       123 |         673 | table_id: 71 flags: STMT_END_F                                     |
| mysql-bin.000008 | 673 | Xid            |       123 |         704 | COMMIT /* xid=31 */                                                |
| mysql-bin.000008 | 704 | Gtid           |       123 |         752 | SET @@SESSION.GTID_NEXT= '5933d78c-a22b-11e5-af00-0800273fac50:16' |
| mysql-bin.000008 | 752 | Query          |       123 |         826 | BEGIN                                                              |
| mysql-bin.000008 | 826 | Rows_query     |       123 |         890 | # insert into binlogtest values (2,'test')                         |
| mysql-bin.000008 | 890 | Table_map      |       123 |         948 | table_id: 71 (lucien.binlogtest)                                   |
| mysql-bin.000008 | 948 | Write_rows     |       123 |         997 | table_id: 71 flags: STMT_END_F                                     |
| mysql-bin.000008 | 997 | Xid            |       123 |        1028 | COMMIT /* xid=42 */                                                |
+------------------+-----+----------------+-----------+-------------+--------------------------------------------------------------------+

从上可以看出，一个事务在binlog中产生多行记录。

事务1：创建表

191-----239 获取GTID

239-----368 创建表语句


事务2：插入第一条数据

368-----416 获取GTID

416-----490 事务开始

490-----560 事务执行的SQL语句

560-----618 

618-----673

673-----704 事务提交（包含事务ID信息）


事务2：插入第二条数据

同事务1.


6.用mysqlbinlog查看binlog日志中相应条目 

[root@mysql01 binlogs]# mysqlbinlog -v -v mysql-bin.000008 
.......................................................
.......................................................

# at 191
#151222 14:55:13 server id 123  end_log_pos 239 CRC32 0x208cd83a        GTID [commit=yes]
SET @@SESSION.GTID_NEXT= '5933d78c-a22b-11e5-af00-0800273fac50:14'/*!*/;
# at 239
#151222 14:55:13 server id 123  end_log_pos 368 CRC32 0x57661241        Query   thread_id=4     exec_time=1     error_code=0
use `lucien`/*!*/;
SET TIMESTAMP=1450767313/*!*/;
SET @@session.pseudo_thread_id=4/*!*/;
SET @@session.foreign_key_checks=1, @@session.sql_auto_is_null=0, @@session.unique_checks=1, @@session.autocommit=1/*!*/;
SET @@session.sql_mode=1073741824/*!*/;
SET @@session.auto_increment_increment=1, @@session.auto_increment_offset=1/*!*/;
/*!\\C utf8 *//*!*/;
SET @@session.character_set_client=33,@@session.collation_connection=33,@@session.collation_server=8/*!*/;
SET @@session.lc_time_names=0/*!*/;
SET @@session.collation_database=DEFAULT/*!*/;
create table binlogtest(id serial,name varchar(20))
/*!*/;
# at 368
#151222 14:58:36 server id 123  end_log_pos 416 CRC32 0x0cc43ac4        GTID [commit=yes]
SET @@SESSION.GTID_NEXT= '5933d78c-a22b-11e5-af00-0800273fac50:15'/*!*/;
# at 416
#151222 14:58:36 server id 123  end_log_pos 490 CRC32 0xa5038708        Query   thread_id=5     exec_time=0     error_code=0
SET TIMESTAMP=1450767516/*!*/;
BEGIN
/*!*/;
# at 490
#151222 14:58:36 server id 123  end_log_pos 560 CRC32 0xc79548d7        Rows_query
# insert into binlogtest values (1,'lijingkuan')
# at 560
#151222 14:58:36 server id 123  end_log_pos 618 CRC32 0xbf120b46        Table_map: `lucien`.`binlogtest` mapped to number 71
# at 618
#151222 14:58:36 server id 123  end_log_pos 673 CRC32 0x2ce894d0        Write_rows: table id 71 flags: STMT_END_F

BINLOG '
nPR4Vh17AAAARgAAADACAACAAC5pbnNlcnQgaW50byBiaW5sb2d0ZXN0IHZhbHVlcyAoMSwnbGlq
aW5na3Vhbicp10iVxw==
nPR4VhN7AAAAOgAAAGoCAAAAAEcAAAAAAAEABmx1Y2llbgAKYmlubG9ndGVzdAACCA8CPAACRgsS
vw==
nPR4Vh57AAAANwAAAKECAAAAAEcAAAAAAAEAAgAC//wBAAAAAAAAAApsaWppbmdrdWFu0JToLA==
'/*!*/;
### INSERT INTO `lucien`.`binlogtest`
### SET
###   @1=1 /* LONGINT meta=0 nullable=0 is_null=0 */
###   @2='lijingkuan' /* VARSTRING(60) meta=60 nullable=1 is_null=0 */
# at 673
#151222 14:58:36 server id 123  end_log_pos 704 CRC32 0xc3dcf0b3        Xid = 31
COMMIT/*!*/;
# at 704
#151222 15:05:49 server id 123  end_log_pos 752 CRC32 0x231697ad        GTID [commit=yes]
SET @@SESSION.GTID_NEXT= '5933d78c-a22b-11e5-af00-0800273fac50:16'/*!*/;
# at 752
#151222 15:05:49 server id 123  end_log_pos 826 CRC32 0xd45eac59        Query   thread_id=6     exec_time=0     error_code=0
SET TIMESTAMP=1450767949/*!*/;
BEGIN
/*!*/;
# at 826
#151222 15:05:49 server id 123  end_log_pos 890 CRC32 0x02dc15d0        Rows_query
# insert into binlogtest values (2,'test')
# at 890
#151222 15:05:49 server id 123  end_log_pos 948 CRC32 0x2d629ce6        Table_map: `lucien`.`binlogtest` mapped to number 71
# at 948
#151222 15:05:49 server id 123  end_log_pos 997 CRC32 0xdf63c14e        Write_rows: table id 71 flags: STMT_END_F

BINLOG '
TfZ4Vh17AAAAQAAAAHoDAACAAChpbnNlcnQgaW50byBiaW5sb2d0ZXN0IHZhbHVlcyAoMiwndGVz
dCcp0BXcAg==
TfZ4VhN7AAAAOgAAALQDAAAAAEcAAAAAAAEABmx1Y2llbgAKYmlubG9ndGVzdAACCA8CPAAC5pxi
LQ==
TfZ4Vh57AAAAMQAAAOUDAAAAAEcAAAAAAAEAAgAC//wCAAAAAAAAAAR0ZXN0TsFj3w==
'/*!*/;
### INSERT INTO `lucien`.`binlogtest`
### SET
###   @1=2 /* LONGINT meta=0 nullable=0 is_null=0 */
###   @2='test' /* VARSTRING(60) meta=60 nullable=1 is_null=0 */
# at 997
#151222 15:05:49 server id 123  end_log_pos 1028 CRC32 0x6c55e57a       Xid = 42
COMMIT/*!*/;
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog *//*!*/;
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;

7.提取某类SQL语句，如insert 


[root@mysql01 binlogs]# mysqlbinlog  mysql-bin.000008 | grep insert
/*!40019 SET @@session.max_insert_delayed_threads=0*/;
[root@mysql01 binlogs]# mysqlbinlog -v  mysql-bin.000008 | grep insert
/*!40019 SET @@session.max_insert_delayed_threads=0*/;
[root@mysql01 binlogs]# mysqlbinlog -v -v mysql-bin.000008 | grep insert
/*!40019 SET @@session.max_insert_delayed_threads=0*/;
# insert into binlogtest values (1,'lijingkuan')
# insert into binlogtest values (2,'test')


只有mysqlbinlog -v -v 才能把使用row格式的binlog中的SQL提取出来？


8.演示mysqlbinlog方式提取binlog

提取指定的binlog日志
# mysqlbinlog /dbdata/binlogs/mysqlbin.000001 
# mysqlbinlog  /dbdata/binlogs/mysqlbin.000001 |grep insert           ------row 模式无法获得sql语句，能看到二进制的binlog，无法阅读</span>
/*!40019 SET @@session.max_insert_delayed_threads=0*/;
insert into tb values(2,'jack')
 
提取指定position位置的binlog日志
# mysqlbinlog --start-position="120" --stop-position="332"  /dbdata/binlogs/mysqlbin.000001 

提取指定位置的多个binlog日志文件
# mysqlbinlog --start-position="120" --stop-position="332"  /dbdata/binlogs/mysqlbin.000001 /dbdata/binlogs/mysqlbin.000002  |more 
 
提取指定数据库binlog并转换字符集到UTF8
# mysqlbinlog --database=lucien --set-charset=utf8 /dbdata/binlogs/mysqlbin.000001 /dbdata/binlogs/mysqlbin.000002 >test.sql

远程提取使用row格式的binlog日志并输出到本地文件
# mysqlbinlog -uroot -p123456 -h192.168.56.124 --read-from-remote-server -v -v mysqlbin.000008 >row.sql


9.mysql binlog日志分析

以事务2为例：

（1）获得事务GTID

SET @@SESSION.GTID_NEXT= '5933d78c-a22b-11e5-af00-0800273fac50:15'/*!*/;

（2）事务开始时间

SET TIMESTAMP=1450767516/*!*/;
   BEGIN

（3）sqlevent起点

# at 490 ：为事件的起点，是以490字节开始。

（4）sqlevent发生的时间点（一个事务中可能有多个sqlevent？）

#151222 14:58:36 事件发生的时间

（5）sqlevent结束时间点，花费时间，错误代码，事务ID，事务线程

end_log_pos 490 CRC32 0xa5038708        Query   thread_id=5     exec_time=0     error_code=0

Xid = 31


以上是关于提取mysql binlog日志内容的主要内容，如果未能解决你的问题，请参考以下文章

Mysql中错误日志binlog日志查询日志慢查询日志简单介绍

使用mysqlbinlog提取二进制日志

MySQL Binlog的介绍

Mysql日志---binlog

Mysql日志---binlog

mysql做了主从，删除binlog日志