MySQL 的 read_buffer_size 参数是如何影响写缓冲和写性能的？

Posted 2020-07-27 zengkefu

Each thread // that does a sequential scan for a MyISAM table// allocates a buffer of this size (in bytes) for each table it scans. 

If you do many sequential scans, you might want to increase this value, which defaults to 131072. The value of this variable should be a multiple of 4KB. If 

it is set to a value that is not a multiple of 4KB, its value will be rounded down to the nearest multiple of 4KB.

This option is also used in the following context for all storage engines:

For caching the indexes in a temporary file (not a temporary table), when sorting rows for ORDER BY.
用于缓存临时表索引---order by

For bulk insert into partitions.
缓存 批量插入到分区

For caching results of nested queries.
缓存 嵌套查询的结果集

read_buffer_size 设置以顺序扫描的方式扫描表数据的时候使用缓冲区的大小. 
每个线程进行顺序扫描的时候都会产生该buffer ,而且同一个Query中如果有多个表进行全表扫描,会产生多个该buffer.

 

 

How read_buffer_size Impacts Write Buffering and Write Performance

• June 23, 2010

In mysql, even though the name read_buffer_size implies that the variable controls only read buffering, but it actually does dual purpose by providing  sequential IO buffering for both reads and writes.

In case of write buffering, ; it groups the sequential writes until read_buffer_size (actually min(read_buffer_size, 8K)); and then does the physical write once the buffer is full. In most cases; this value is the initial value ofread_buffer_size when server actually started first time; as this is a dynamic global variable; even if you change the value dynamically at run time; it will not affect write buffering size (and in some cases of read buffering as well) as this is stored one-time in my_default_record_cache_size (might be a bug ?); and that variable is used in initializing IO cache buffers.

Two use cases where read_buffer_size is actually used for buffering writes within MySQL:

• SELECT INTO … OUTFILE ‘fileName‘
  • When writing to OUTFILE, the writes are buffered before writing to OUTFILE
• When filesort is used, during merge buffers and when merged results are written to a temporary file, then writes are buffered

Normally you will see performance boost due to IO buffering on slower disks or when you have IO saturation as ios are grouped together to a single write; but when you have good IO sub-system or when writes are almost NOOP (RAID controller caching), etc., then buffering has negative impact.

Here is some stats on how many physical writes are actually posted for a simple SELECT … INTO OUTFILE (file size 384936838 bytes) for variable read_buffer_size values (server needs to be restarted in-order to get the new value):

read_buffer_size	physical writes	exe time in secs
=0 (defaults to 8200)	23495	28.39
=131072 (default)	2937	27.44
=16777216	23	26.71
=33554432	12	26.00
=536870912	1	26.72

Total writes are calculated using simple patch that I wrote around mysys/my_write.c to get the real physical writes posted as a global status counter.

 

 

 

Shell

 

1 2 3 4 5 6 7 8 9

mysql> show global status like ‘Write_count‘; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | Write_count   | 23496 | +---------------+-------+ 1 row in set (0.00 sec)

As you can see, increase inread_buffer_size might save total physical writes and might help if you have lot of OUTFILEs or heavy file sorting to some extent; but again this will actually affect overall performance due to one of the known bug and the buffer is also allocated per query based; so be careful as allocation and initialization of big buffers are much costlier than real IO cost.

In either case; may be worth if the sequential read buffering is actually controlled by read_buffer_size and introduce new write_buffer_size that controls the write buffering instead of using the same for both or use a different variable like io_buffer_size.