Waiting for table metadata lock

Posted 2020-07-20 渔夫数据库笔记

  本篇博客准备分两部分详细的介绍一下mysql (基于mysql 5.6)中的metadata lock，

一. 官方文档中关于Metadata Locking描述(5.6)

8.11.4 Metadata Locking

MySQL uses metadata locking to manage concurrent access to database objects and to ensure data consistency. Metadata locking applies not just to tables, but also to schemas and stored programs (procedures, functions, triggers, and scheduled events).  <span style="color:#330000;">>>mysql使用metadata locking来保证数据库对象并发访问时的数据一致性。metadata locking不仅仅用来保护表，也用来保护数据库中的其他对象，比如说 schemas,procedures,functions,triggers,scheduled events。</span>

Metadata locking does involve some overhead, which increases as query volume increases. Metadata contention increases the more that multiple queries attempt to access the same objects.  <span style="color:#330000;">>>随着访问量的增加，metadata locking所导致的开销也会增加。越多的查询访问相同的对象，产生metadata lock争用的可能越大。</span><span style="color:#ff0000;">
</span>
Metadata locking is not a replacement for the table definition cache, and its mutexes and locks differ from the LOCK_open mutex. The following discussion provides some information about how metadata locking works.  <span style="color:#330000;">>>metedata locking 不能代替 table definition cache。下面讨论了metadata locking如何工作。</span>

To ensure transaction serializability, the server must not permit one session to perform a data definition language (DDL) statement on a table that is used in an uncompleted explicitly or implicitly started transaction in another session. The server achieves this by acquiring metadata locks on tables used within a transaction and deferring release of those locks until the transaction ends. A metadata lock on a table prevents changes to the table's structure. This locking approach has the implication that a table that is being used by a transaction within one session cannot be used in DDL statements by other sessions until the transaction ends.  <span style="color:#330000;">>>为了确保事物的串行(同一个对象上)，如果一个对象正在被一个未结束的事物使用(显示或者隐示)，那么就不能允许其他的会话对其进行ddl操作。mysql通过在事物过程中给对象加上metadata lock来实现对象的串行访问(metadata lock在事物结束的时候被释放)。如果一个表被加上了metadata lock，那么对该表的ddl操作会被阻塞(阻塞ddl，就像“元数据锁”这个名称锁说的那样)。</span>

This principle applies not only to transactional tables, but also to nontransactional tables. Suppose that a session begins a transaction that uses transactional table t and nontransactional table nt as follows: <span style="color:#ff0000;"> </span><span style="color:#330000;">>>上面说的metadata lock原则不仅针对事物表(如果innodb表)，对非事物表(如myisam表)一样有效。假设一个会话开启了如下事物(事物中使用到了事物表t和非事物表nt)</span>

START TRANSACTION;
SELECT * FROM t;
SELECT * FROM nt;
The server holds metadata locks on both t and nt until the transaction ends. If another session attempts a DDL or write lock operation on either table, it blocks until metadata lock release at transaction end. For example, a second session blocks if it attempts any of these operations:  <span style="color:#330000;">>>上面的例子中mysql会对事物表t和非事物表nt都加上metadata lock直到事物结束。如果上面的事物还没有结束时，在另一个会话中进行如下操作(对t和nt进行ddl)，会发现相关ddl操作都会被阻塞</span>

DROP TABLE t;
ALTER TABLE t ...;
DROP TABLE nt;
ALTER TABLE nt ...;
LOCK TABLE t ... WRITE;
If the server acquires metadata locks for a statement that is syntactically valid but fails during execution, it does not release the locks early. <span style="color:#ff0000;">Lock release is still deferred to the end of the transaction because the failed statement is written to the binary log and the locks protect log consistency</span>.>>如果一个语法正确的语句，在执行过程中失败，加在该表上的metadata lock并不会立即释放，而是等到事物结束的时候才会释放(后面我会举例)。上面表红部分不能理解是什么意思，执行失败的语句怎么会记录到binlog中？

In autocommit mode, each statement is in effect a complete transaction, so metadata locks acquired for the statement are held only to the end of the statement.  >>在autocommit模式下，每一个语句实际上就是一个完整的事物，所以metadata lock在语句开始的时候被加上，在语句结束的时候被释放

Metadata locks acquired during a PREPARE statement are released once the statement has been prepared, even if preparation occurs within a multiple-statement transaction.  >>prepare语句获得的metadata lock在sql已经被prepare后就会被释放，即使是在一个由多个sql组成的事物中。

Before MySQL 5.5, when a transaction acquired the equivalent of a metadata lock for a table used within a statement, it released the lock at the end of the statement. This approach had the disadvantage that if a DDL statement occurred for a table that was being used by another session in an active transaction, statements could be written to the binary log in the wrong order.  >>在mysql 5.5之前，会在sql执行的过程中对表加一个类似metadata lock的锁，当语句结束的时候失败该锁(现在是在整个事物过程中加metadata lock，但是在autocommit模式下，也是在语句执行过程中加锁，执行结束后释放，因为一个语句即一个事物)。这种方法有一个缺点，可能会导致binlog记录顺序跟实际执行顺序不一致。

二. mysql 中出现Waiting for table metadata lock的场景

   在show processlist时我们可能见到过某个语句处于Waiting for table metadata lock状态，现在我们来分析一下，在哪些情况下会出现Waiting for table metadata lock

场景1：

  当一个表在被一个未结束的事物使用(包括select)，另一个会话中对该表进行ddl操作时，因为无法获得独占的metadata lock该ddl操作会被阻塞，此时show processlist查看ddl操作处于Waiting for table metadata lock状态。这时我们希望知道是什么阻塞了我的ddl语句，通过下面三步，你就能找到是什么操作阻塞了你的ddl操作：

1)show processlist

  如果对ddl语句造成阻塞的语句正在执行中，你通过show processlist能够发现这样的语句(查看对ddl语句中所涉及表的操作)。

2)information_schema.innodb_trx

  查看innodb事物表information_schema.innodb_trx，如果在一个显示事物中(使用start transaction或者begin开启的事物)，对于相关表的操作已经完成(所以show processlist无法查看到该操作)，但是该事物没有被提交，此时在另一个会话中对该表进行ddl操作时，因为无法获得独占的metadata lock，该ddl操作会被阻塞。此时通过查看information_schema.innodb_trx表可以看到有活跃的事物

3)performance_schema.events_statements_current

  如果某个表的ddl操作因为无法获得独占的metadata lock而被阻塞，并且查看1)和2)中的方法都没有查到相关信息，那么还有一种可能是因为在一个显式的事务中，对表a进行了一个失败的操作（比如查询了一个不存在的字段），这时事务没有开始，但是失败语句获取到的锁依然有效。从performance_schema.events_statements_current表中可以查到失败的语句。events_statements_current表记录了数据库当前thread最近一次执行的sql。

场景2：

  我们知道metadata lock是用来保护数据结构的，如果表a正在被一个活跃的事物使用，另一个会话对表a的ddl操作会被阻塞。当表a的ddl操作因为无法获得独占的metadata lock(Waiting for table metadata lock)被阻塞时，后续对于表a的任何操作都会被阻塞，包括select操作(状态为Waiting for table metadata lock)

场景3：

  inplace(当然5.6开始多数的ddl操作都支持online_ddl)方式对表a执行的ddl ，在ddl语句执行过程中会阻塞对于表a的所有dml操作。在online_ddl过程中不是在整个语句执行过程对表a加独占的metadata lock，在After table(online_ddl具体过程见下图)，会获取metadata lock的独占锁，当进行到altering table(也是最耗时的一步)时，所有dml都是可以正常进行的。

下面是online_ddl的处理过程(profile)

+--------------------------------+----------+
| Status                         | Duration |
+--------------------------------+----------+
| starting                       | 0.000119 |
| checking permissions           | 0.000013 |
| checking permissions           | 0.000011 |
| init                           | 0.000007 |
| Opening tables                 | 0.007743 |
| setup                          | 0.000055 |
| creating table                 | 0.004871 |
| After create                   | 0.000152 |
| System lock                    | 0.000018 |
| preparing for alter table      | 0.009417 |
| altering table                 | 0.064343 |
| committing alter table to stor | 0.024200 |
| end                            | 0.000025 |
| query end                      | 0.002839 |
| closing tables                 | 0.000017 |
| freeing items                  | 0.000040 |
| cleaning up                    | 0.000025 |
+--------------------------------+----------+