利用.frm.ibd恢复数据
Posted 醒嘞
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了利用.frm.ibd恢复数据相关的知识，希望对你有一定的参考价值。
我们知道启用innodb_file_per_table选项后，单个表(InnoDB引擎)的数据和索引放入单独的文件中(.ibd)，建表语句保存在.frm文件中
本文假设192.168.85.132,3306实例无法启动(磁盘还能访问)，需要将原实例下的replcrash.py_user表恢复到192.168.85.132,3308实例
一、.frm得到建表语句
mysqlfrm可以读取.frm文件并从该文件中找到表定义数据生成CREATE语句。在大多数情况下，生成的CREATE语句用于在另一个服务器上创建表或进行诊断等。
1.1、操作模式
默认模式，使用--basedir或者--server选项生成一个临时实例，该模式需要指定--port选项给临时实例使用，该端口不能与现有的实例冲突。在读取.frm文件后，临时实例将被关闭，所有的临时文件将被删除。
诊断模式，使用--diagnostic选项。byte-by-byte读取.frm文件，尽可能多的恢复信息。
诊断模式可能遇到的问题
• 不能解读character set/collation values，这会影响到多字节字符列大小
• 不能读取默认值
• 生成的create语句可能会有语法错误
当使用默认模式无法读取文件，或者服务器上没有安装MySQL实例时就使用诊断模式~~
1.2、mysqlfrm安装使用
1、安装 # mysql-utilities依赖mysql-connector-python >= 2.0.0 [root@ZST1 tools]# wget https://dev.mysql.com/get/Downloads/Connector-Python/mysql-connector-python-2.1.7-1.el6.x86_64.rpm [root@ZST1 tools]# wget https://dev.mysql.com/get/Downloads/MySQLGUITools/mysql-utilities-1.6.5-1.el6.noarch.rpm # 安装 [root@ZST1 tools]# yum localinstall mysql-connector-python-2.1.7-1.el6.x86_64.rpm -y [root@ZST1 tools]# yum localinstall mysql-utilities-1.6.5-1.el6.noarch.rpm -y 2、使用 # default mode # --basedir [root@ZST1 ~]# mysqlfrm --basedir=/usr/local/mysql/ /data/mysql/mysql3306/data/replcrash/py_user.frm --port=3333 --user=root # --server [root@ZST1 ~]# mysqlfrm --server=mydba:mysql5719@192.168.85.132:3306 /data/mysql/mysql3308/data/replcrash/py_user.frm --port=3333 --user=root 官方不推荐使用root用户Running spawned server # diagnostic mode [root@ZST1 ~]# mysqlfrm --diagnostic /data/mysql/mysql3308/data/replcrash/py_user.frm
故障实例OFFLINE状态，建议使用--basedir得到建表语句
1.3、mysqlfrm逻辑(default mode)
在命令行末尾添加 -vvv 可以查看执行详细步骤
[root@ZST1 ~]# mysqlfrm --basedir=/usr/local/mysql/ /data/mysql/mysql3306/data/replcrash/py_user.frm --port=3333 --user=root -vvv # Checking read access to .frm files # Creating a temporary datadir = /tmp/tmpktcZWc # Spawning server with --user=root. # Starting the spawned server on port 3333 ... # Cloning the MySQL server located at /usr/local/mysql. # Configuring new instance... # Locating mysql tools... # Location of files: # mysqld: /usr/local/mysql/bin/mysqld # mysqladmin: /usr/local/mysql/bin/mysqladmin # Setting up empty database and mysql tables... 2017-12-28T09:36:22.383542Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details). 2017-12-28T09:36:24.528222Z 0 [Warning] InnoDB: New log files created, LSN=45790 2017-12-28T09:36:24.936974Z 0 [Warning] InnoDB: Creating foreign key constraint system tables. 2017-12-28T09:36:25.019189Z 0 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 925a5f84-ebb2-11e7-a13e-000c29c1025c. 2017-12-28T09:36:25.024957Z 0 [Warning] Gtid table is not ready to be used. Table \'mysql.gtid_executed\' cannot be opened. 2017-12-28T09:36:25.025657Z 1 [Warning] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option. # Starting new instance of the server... # Startup command for new server: /usr/local/mysql/bin/mysqld --no-defaults --basedir=/usr/local/mysql --datadir=/tmp/tmpktcZWc --pid-file=/tmp/tmpktcZWc/clone.pid --port=3333 --server-id=101 --socket=/tmp/tmpktcZWc/mysql.sock --tmpdir=/tmp/tmpktcZWc --user=root # Testing connection to new instance... 2017-12-28T09:36:29.346999Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details). 2017-12-28T09:36:29.369718Z 0 [Note] --secure-file-priv is set to NULL. Operations related to importing and exporting data are disabled 2017-12-28T09:36:29.369837Z 0 [Note] /usr/local/mysql/bin/mysqld (mysqld 5.7.19) starting as process 9382 ... 2017-12-28T09:36:29.387216Z 0 [Note] InnoDB: PUNCH HOLE support available 2017-12-28T09:36:29.387287Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2017-12-28T09:36:29.387309Z 0 [Note] InnoDB: Uses event mutexes 2017-12-28T09:36:29.387334Z 0 [Note] InnoDB: GCC builtin __sync_synchronize() is used for memory barrier 2017-12-28T09:36:29.387353Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.3 2017-12-28T09:36:29.387372Z 0 [Note] InnoDB: Using Linux native AIO 2017-12-28T09:36:29.388729Z 0 [Note] InnoDB: Number of pools: 1 2017-12-28T09:36:29.389559Z 0 [Note] InnoDB: Using CPU crc32 instructions 2017-12-28T09:36:29.396790Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M 2017-12-28T09:36:29.413527Z 0 [Note] InnoDB: Completed initialization of buffer pool 2017-12-28T09:36:29.419969Z 0 [Note] InnoDB: page_cleaner coordinator priority: -20 2017-12-28T09:36:29.434005Z 0 [Note] InnoDB: Highest supported file format is Barracuda. 2017-12-28T09:36:29.503503Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables 2017-12-28T09:36:29.503705Z 0 [Note] InnoDB: Setting file \'./ibtmp1\' size to 12 MB. Physically writing the file full; Please wait ... 2017-12-28T09:36:30.110625Z 0 [Note] InnoDB: File \'./ibtmp1\' size is now 12 MB. 2017-12-28T09:36:30.113767Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active. 2017-12-28T09:36:30.113804Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active. 2017-12-28T09:36:30.114361Z 0 [Note] InnoDB: Waiting for purge to start # trying again... 2017-12-28T09:36:30.165192Z 0 [Note] InnoDB: 5.7.19 started; log sequence number 2539315 2017-12-28T09:36:30.165487Z 0 [Note] Plugin \'FEDERATED\' is disabled. 2017-12-28T09:36:30.175080Z 0 [Note] InnoDB: Loading buffer pool(s) from /tmp/tmpktcZWc/ib_buffer_pool 2017-12-28T09:36:30.176769Z 0 [Warning] Failed to set up SSL because of the following SSL library error: SSL context is not usable without certificate and private key 2017-12-28T09:36:30.176801Z 0 [Note] Server hostname (bind-address): \'*\'; port: 3333 2017-12-28T09:36:30.176922Z 0 [Note] IPv6 is available. 2017-12-28T09:36:30.176941Z 0 [Note] - \'::\' resolves to \'::\'; 2017-12-28T09:36:30.176987Z 0 [Note] Server socket created on IP: \'::\'. 2017-12-28T09:36:30.190355Z 0 [Note] InnoDB: Buffer pool(s) load completed at 171228 17:36:30 2017-12-28T09:36:30.211166Z 0 [Note] Event Scheduler: Loaded 0 events 2017-12-28T09:36:30.211365Z 0 [Note] /usr/local/mysql/bin/mysqld: ready for connections. Version: \'5.7.19\' socket: \'/tmp/tmpktcZWc/mysql.sock\' port: 3333 MySQL Community Server (GPL) 2017-12-28T09:36:30.211377Z 0 [Note] Executing \'SELECT * FROM INFORMATION_SCHEMA.TABLES;\' to get a list of tables using the deprecated partition engine. You may use the startup option \'--disable-partition-engine-check\' to skip this check. 2017-12-28T09:36:30.211381Z 0 [Note] Beginning of list of non-natively partitioned tables 2017-12-28T09:36:30.241025Z 0 [Note] End of list of non-natively partitioned tables # trying again... # Success! # Setting the root password... mysqladmin: [Warning] Using a password on the command line interface can be insecure. Warning: Since password will be sent to server in plain text, use ssl connection to ensure password safety. # Connection Information: # -uroot -proot --socket=/tmp/tmpktcZWc/mysql.sock #...done. 2017-12-28T09:36:31.297094Z 3 [Note] Aborted connection 3 to db: \'unconnected\' user: \'root\' host: \'localhost\' (Got an error reading communication packets) # Connecting to spawned server done. # Reading .frm files # # Reading the py_user.frm file. # Changing engine for .frm file /tmp/tmpktcZWc/replcrash_temp/py_user.frm: # Skipping to header at : 2 # General Data from .frm file: {\'IO_SIZE\': 4096, \'MYSQL_VERSION_ID\': 50719, \'avg_row_length\': 0, \'charset_low\': 0, \'create_options\': 9, \'db_create_pack\': 2, \'default_charset\': 33, \'default_part_eng\': 0, \'extra_size\': 33, \'frm_file_ver\': 5, \'frm_version\': 10, \'key_block_size\': 0, \'key_info_length\': 33, \'key_length\': 361, \'legacy_db_type\': \'INNODB\', \'length\': 12288, \'max_rows\': 0, \'min_rows\': 0, \'rec_length\': 138, \'row_type\': 0, \'table_charset\': 33, \'tmp_key_length\': 361} # Engine string: InnoDB # Server version in file: 5.7.19 # # CREATE statement for /data/mysql/mysql3306/data/replcrash/py_user.frm: # CREATE TABLE `replcrash`.`py_user` ( `uid` int(11) NOT NULL AUTO_INCREMENT, `name` varchar(32) DEFAULT NULL, `add_time` datetime DEFAULT NULL, `server_id` varchar(10) DEFAULT NULL, PRIMARY KEY (`uid`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 # Shutting down spawned server # Removing the temporary datadir 2017-12-28T09:36:31.471827Z 0 [Note] Giving 0 client threads a chance to die gracefully 2017-12-28T09:36:31.471874Z 0 [Note] Shutting down slave threads 2017-12-28T09:36:31.471891Z 0 [Note] Forcefully disconnecting 0 remaining clients 2017-12-28T09:36:31.471908Z 0 [Note] Event Scheduler: Purging the queue. 0 events 2017-12-28T09:36:31.473626Z 0 [Note] Binlog end 2017-12-28T09:36:31.473789Z 0 [Note] Shutting down plugin \'ngram\' 2017-12-28T09:36:31.473820Z 0 [Note] Shutting down plugin \'BLACKHOLE\' 2017-12-28T09:36:31.473839Z 0 [Note] Shutting down plugin \'ARCHIVE\' 2017-12-28T09:36:31.473853Z 0 [Note] Shutting down plugin \'partition\' 2017-12-28T09:36:31.473866Z 0 [Note] Shutting down plugin \'INNODB_SYS_VIRTUAL\' 2017-12-28T09:36:31.473880Z 0 [Note] Shutting down plugin \'INNODB_SYS_DATAFILES\' 2017-12-28T09:36:31.473893Z 0 [Note] Shutting down plugin \'INNODB_SYS_TABLESPACES\' 2017-12-28T09:36:31.473905Z 0 [Note] Shutting down plugin \'INNODB_SYS_FOREIGN_COLS\' 2017-12-28T09:36:31.473918Z 0 [Note] Shutting down plugin \'INNODB_SYS_FOREIGN\' 2017-12-28T09:36:31.473931Z 0 [Note] Shutting down plugin \'INNODB_SYS_FIELDS\' 2017-12-28T09:36:31.473943Z 0 [Note] Shutting down plugin \'INNODB_SYS_COLUMNS\' 2017-12-28T09:36:31.473956Z 0 [Note] Shutting down plugin \'INNODB_SYS_INDEXES\' 2017-12-28T09:36:31.473969Z 0 [Note] Shutting down plugin \'INNODB_SYS_TABLESTATS\' 2017-12-28T09:36:31.473981Z 0 [Note] Shutting down plugin \'INNODB_SYS_TABLES\' 2017-12-28T09:36:31.474010Z 0 [Note] Shutting down plugin \'INNODB_FT_INDEX_TABLE\' 2017-12-28T09:36:31.474024Z 0 [Note] Shutting down plugin \'INNODB_FT_INDEX_CACHE\' 2017-12-28T09:36:31.474036Z 0 [Note] Shutting down plugin \'INNODB_FT_CONFIG\' 2017-12-28T09:36:31.474156Z 0 [Note] Shutting down plugin \'INNODB_FT_BEING_DELETED\' 2017-12-28T09:36:31.474175Z 0 [Note] Shutting down plugin \'INNODB_FT_DELETED\' 2017-12-28T09:36:31.474188Z 0 [Note] Shutting down plugin \'INNODB_FT_DEFAULT_STOPWORD\' 2017-12-28T09:36:31.474214Z 0 [Note] Shutting down plugin \'INNODB_METRICS\' 2017-12-28T09:36:31.474227Z 0 [Note] Shutting down plugin \'INNODB_TEMP_TABLE_INFO\' 2017-12-28T09:36:31.474241Z 0 [Note] Shutting down plugin \'INNODB_BUFFER_POOL_STATS\' 2017-12-28T09:36:31.474256Z 0 [Note] Shutting down plugin \'INNODB_BUFFER_PAGE_LRU\' 2017-12-28T09:36:31.474269Z 0 [Note] Shutting down plugin \'INNODB_BUFFER_PAGE\' 2017-12-28T09:36:31.474281Z 0 [Note] Shutting down plugin \'INNODB_CMP_PER_INDEX_RESET\' 2017-12-28T09:36:31.474303Z 0 [Note] Shutting down plugin \'INNODB_CMP_PER_INDEX\' 2017-12-28T09:36:31.474317Z 0 [Note] Shutting down plugin \'INNODB_CMPMEM_RESET\' 2017-12-28T09:36:31.474330Z 0 [Note] Shutting down plugin \'INNODB_CMPMEM\' 2017-12-28T09:36:31.474343Z 0 [Note] Shutting down plugin \'INNODB_CMP_RESET\' 2017-12-28T09:36:31.474356Z 0 [Note] Shutting down plugin \'INNODB_CMP\' 2017-12-28T09:36:31.474369Z 0 [Note] Shutting down plugin \'INNODB_LOCK_WAITS\' 2017-12-28T09:36:31.474382Z 0 [Note] Shutting down plugin \'INNODB_LOCKS\' 2017-12-28T09:36:31.474395Z 0 [Note] Shutting down plugin \'INNODB_TRX\' 2017-12-28T09:36:31.474407Z 0 [Note] Shutting down plugin \'InnoDB\' 2017-12-28T09:36:31.474590Z 0 [Note] InnoDB: FTS optimize thread exiting. 2017-12-28T09:36:31.476410Z 0 [Note] InnoDB: Starting shutdown... 2017-12-28T09:36:31.577509Z 0 [Note] InnoDB: Dumping buffer pool(s) to /tmp/tmpktcZWc/ib_buffer_pool 2017-12-28T09:36:31.577612Z 0 [ERROR] InnoDB: Cannot open \'/tmp/tmpktcZWc/ib_buffer_pool.incomplete\' for writing: No such file or directory 2017-12-28T09:36:33.117642Z 0 [Note] InnoDB: Shutdown completed; log sequence number 2539334 2017-12-28T09:36:33.117712Z 0 [Note] Shutting down plugin \'MEMORY\' 2017-12-28T09:36:33.117727Z 0 [Note] Shutting down plugin \'PERFORMANCE_SCHEMA\' 2017-12-28T09:36:33.117761Z 0 [Note] Shutting down plugin \'MRG_MYISAM\' 2017-12-28T09:36:33.117767Z 0 [Note] Shutting down plugin \'MyISAM\' 2017-12-28T09:36:33.117783Z 0 [Note] Shutting down plugin \'CSV\' 2017-12-28T09:36:33.117790Z 0 [Note] Shutting down plugin \'sha256_password\' 2017-12-28T09:36:33.117794Z 0 [Note] Shutting down plugin \'mysql_native_password\' 2017-12-28T09:36:33.117966Z 0 [Note] Shutting down plugin \'binlog\' 2017-12-28T09:36:33.118884Z 0 [Note] Unable to delete pid file: No such file or directory 2017-12-28T09:36:33.118898Z 0 [Note] /usr/local/mysql/bin/mysqld: Shutdown complete [root@ZST1 ~]#
• 检查读取.frm文件的权限
• 创建临时datadir
• 克隆生成新实例
• 启动、连接新实例
• 读取.frm文件，生成create语句
• 关闭新实例，删除临时datadir
mysqlfrm的使用参考：5.13 mysqlfrm — File reader for .frm files、3.3.2 How do you recover the CREATE statement from a damaged or offline server?
二、恢复表数据
这里用到表空间传输，恢复环境使用mysqlfrm得到语句创建相同结构的数据表，然后discard表空间，拷贝故障实例下的ibd文件并修改权限，重新载入表空间，载入过程会有警告，但不影响数据的读取
# 将mysqlfrm得到建表语句拷贝到192.168.85.132,3308上执行 mydba@192.168.85.132,3308 [(none)]> use replcrash; mydba@192.168.85.132,3308 [replcrash]> drop table if exists py_user; # 执行mysqlfrm得到的create语句 mydba@192.168.85.132,3308 [replcrash]> CREATE TABLE `replcrash`.`py_user` ( -> `uid` int(11) NOT NULL AUTO_INCREMENT, -> `name` varchar(32) DEFAULT NULL, -> `add_time` datetime DEFAULT CURRENT_TIMESTAMP, -> `server_id` varchar(10) DEFAULT NULL, -> PRIMARY KEY (`uid`) -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8; Query OK, 0 rows affected (0.17 sec) # discard 表空间 mydba@192.168.85.132,3308 [replcrash]> alter table py_user discard tablespace; Query OK, 0 rows affected (0.86 sec) # 拷贝ibd文件，并修改权限 [root@ZST1 ~]# scp /data/mysql/mysql3306/data/replcrash/py_user.ibd 192.168.85.132:/data/mysql/mysql3308/data/replcrash/ [root@ZST1 ~]# chown -R mysql:mysql /data/mysql/mysql3308/data/replcrash/py_user.ibd # 载入表空间 mydba@192.168.85.132,3308 [replcrash]> alter table py_user import tablespace; Query OK, 0 rows affected, 1 warning (0.24 sec) mydba@192.168.Mysql 通过frm&ibd 恢复数据