详述Redis持久化方式

Posted 2022-12-13 程序猿老高

1、概述

Redis支持RDB(Redis DataBase)和AOF(Append Only File) 两种持久化机制，默认开启RDB机制，生成名为dump.rdb的数据文件，该文件存放在redis.conf配置文件内dir配置项指定的目录中，该配置项默认值为./，./表示当前目录，启动redis服务时执行redis-server /etc/redis.conf指令，所以./表示执行该指令时所在工作目录，如下示例：

a、示例

[root@bogon ~]# ps -ef | grep redis            
root      17392  17291  0 17:39 pts/0    00:00:00 grep --color=auto redis
[root@bogon ~]# pwd 查看当前工作目录
/root
[root@bogon ~]# redis-server /etc/redis.conf  该指令在/root目录下执行，所以dump.rdb文件存储在该目录。
17393:C 23 Aug 2020 17:39:52.380 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
17393:C 23 Aug 2020 17:39:52.380 # Redis version=5.0.5, bits=64, commit=00000000, modified=0, pid=17393, just started
17393:C 23 Aug 2020 17:39:52.380 # Configuration loaded
[root@bogon ~]# redis-cli -h 127.0.0.1 -p 6379              
127.0.0.1:6379> set name Jim
OK
127.0.0.1:6379> save
OK
127.0.0.1:6379> exit
[root@bogon ~]# ll
总用量 12
-rw-------. 1 root root 1698 5月   8 01:24 anaconda-ks.cfg
-rw-r--r--. 1 root root  156 8月  23 17:40 dump.rdb
-rw-r--r--. 1 root root 1746 5月   8 01:25 initial-setup-ks.cfg
drwxr-xr-x. 2 root root    6 5月   8 01:27 公共
drwxr-xr-x. 2 root root    6 5月   8 01:27 模板
drwxr-xr-x. 2 root root    6 5月   8 01:27 视频
drwxr-xr-x. 2 root root    6 5月   8 01:27 图片
drwxr-xr-x. 2 root root    6 5月   8 01:27 文档
drwxr-xr-x. 2 root root    6 5月   8 01:27 下载
drwxr-xr-x. 2 root root    6 5月   8 01:27 音乐
drwxr-xr-x. 2 root root    6 5月   8 01:27 桌面
[root@bogon ~]#

b、示例

[root@bogon ~]# redis-cli -h 127.0.0.1 shutdown 关闭Redis服务
[root@bogon ~]# ps -ef |grep redis
root      17703  17291  0 17:52 pts/0    00:00:00 grep --color=auto redis
[root@bogon ~]# cd /usr/local/bin 切换目录
[root@bogon bin]# pwd 查看当前工作目录
/usr/local/bin
[root@bogon bin]# redis-server /etc/redis.conf  /usr/local/bin目录下启动redis
17766:C 23 Aug 2020 17:58:01.736 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
17766:C 23 Aug 2020 17:58:01.736 # Redis version=5.0.5, bits=64, commit=00000000, modified=0, pid=17766, just started
17766:C 23 Aug 2020 17:58:01.736 # Configuration loaded
[root@bogon bin]# ps -ef |grep redis
root      17767      1  0 17:58 ?        00:00:00 redis-server 127.0.0.1:6379
root      17780  17291  0 17:58 pts/0    00:00:00 grep --color=auto redis
[root@bogon bin]# redis-cli -h 127.0.0.1 -p 6379
127.0.0.1:6379> set age 12
OK
127.0.0.1:6379> save
OK
127.0.0.1:6379> exit
[root@bogon bin]# ll
总用量 32740
-rw-r--r--. 1 root root     163 8月  23 17:58 dump.rdb
-rwxr-xr-x. 1 root root 4366656 8月  17 15:26 redis-benchmark
-rwxr-xr-x. 1 root root 8111904 8月  17 15:26 redis-check-aof
-rwxr-xr-x. 1 root root 8111904 8月  17 15:26 redis-check-rdb
-rwxr-xr-x. 1 root root 4806880 8月  17 15:26 redis-cli
lrwxrwxrwx. 1 root root      12 8月  17 15:26 redis-sentinel -> redis-server
-rwxr-xr-x. 1 root root 8111904 8月  17 15:26 redis-server
[root@bogon bin]#

2、RDB

a、原理

在指定的时间间隔内将内存中的数据集快照（Snapshot）写入到临时文件中，持久化过程结束后将使用该临时文件替换上次持久化好的文件，恢复数据时直接将快照文件内容读至内存。

b、恢复

由于dir配置项默认值为./，./表示相对路径，所以持久化后的rdb文件默认存储在启动Redis服务时所在的工作目录，所以只有在上次启动Redis服务时所在的工作目录内启动Redis服务才能完成数据的恢复，否则会因找不到rdb文件而恢复失败；如果dir配置成绝对目录，则在任意工作目录启动Redis服务都可以。

c、配置

################################ SNAPSHOTTING  ################################
#
# Save the DB on disk:
#
#   save <seconds> <changes>
#   Will save the DB if both the given number of seconds and the given number of write operations against the DB occurred.
#
#   In the example below the behaviour will be to save:
#   after 900 sec (15 min) if at least 1 key changed
#   after 300 sec (5 min) if at least 10 keys changed
#   after 60 sec if at least 10000 keys changed
#
#   Note: you can disable saving completely by commenting out all "save" lines.
#
#   It is also possible to remove all the previously configured save
#   points by adding a save directive with a single empty string argument like in the following example:
#
#   save "" #禁用RDB持久化机制，不设置任何“save 秒 写次数”指令也可禁用该持久化机制

save 900 1 #15分钟内至少执行1次写操作
save 300 10 #5分钟内至少执行10次写操作
save 60 10000 # 1分钟内至少执行1万次写操作
# 上面配置满足其中一个条件即触发RDB机制；
# 执行save、bgsave、shutdown或flushall指令则会立即触发该机制，执行flushall指令所产生的dump.rdb文件为空，没有实际实战意义，flushdb指令不能生成dump.rdb文件。
# save与bgsave指令都会通过调用rdbSave函数实现数据持久化，但它们调用方式有所不同：①、save指令执行后直接调用rdbSave函数开始数据持久化，持久化结束之前，Redis服务器主进程被阻塞，此时，不能处理客户端的任何请求；②、bgsave指令执行后，redis先fork一个子进程，子进程负责调用rdbSave函数，持久化结束之前，Redis 服务器仍然可以处理客户端的请求；

# By default Redis will stop accepting writes if RDB snapshots are enabled
# (at least one save point) and the latest background save failed.
# This will make the user aware (in a hard way) that data is not persisting
# on disk properly, otherwise chances are that no one will notice and some
# disaster will happen.
#
# If the background saving process will start working again Redis will
# automatically allow writes again.
#
# However if you have setup your proper monitoring of the Redis server
# and persistence, you may want to disable this feature so that Redis will
# continue to work as usual even if there are problems with disk,
# permissions, and so forth.
stop-writes-on-bgsave-error yes #bgsave命令出错时是否停止写操作

# Compress string objects using LZF when dump .rdb databases?
# For default that's set to 'yes' as it's almost always a win.
# If you want to save some CPU in the saving child set it to 'no' but
# the dataset will likely be bigger if you have compressible values or keys.
rdbcompression yes #是否进行压缩存储，默认为yes，即redis采用LZF算法压缩数据，但会消耗CPU资源

# Since version 5 of RDB a CRC64 checksum is placed at the end of the file.
# This makes the format more resistant to corruption but there is a performance
# hit to pay (around 10%) when saving and loading RDB files, so you can disable it
# for maximum performances.
#
# RDB files created with checksum disabled have a checksum of zero that will
# tell the loading code to skip the check.
rdbchecksum yes #是否开启rdb数据校验，数据持久化后可以让redis使用CRC64算法进行数据校验，但是这会增加大约10%的性能消耗，如果希望获取到最大的性能提升，可以关闭此功能。
# The filename where to dump the DB
dbfilename dump.rdb #文件名

# The working directory. 
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# The Append Only File will also be created inside this directory. #AOF持久化机制生成的文件也存储到这个目录
#
# Note that you must specify a directory here, not a file name.
dir /usr/local/bin #设置工作目录，默认./，由于每次启动Redis时的工作目录可能不尽相同，致使数据文件存储到不同目录中，不便于后期管理，为避免这一问题出现，建议统一将其存储目录。

3、AOF

a、原理

以日志形式将Redis执行过的每个写指令(读操作不记录) 追加到文件末尾，该文件存放在redis.conf配置文件内dir配置项指定的目录中，Redis启动之初自动根据该日志文件的内容将写指令从前到后执行一次以完成数据的恢复。由于AOF持久化机制保存的是每个执行过的写数据指令，因此该文件很容易被人读懂。

b、恢复

由于dir配置项默认值为./，./表示相对路径，所以持久化后的aof文件默认存储在启动Redis服务时所在的工作目录，所以只有在上次启动Redis服务时所在的工作目录内启动Redis服务才能完成数据的恢复，否则会因找不到aof文件而恢复失败；如果dir配置成绝对目录，则在任意工作目录启动Redis服务都可以。

c、配置

############################## APPEND ONLY MODE ###############################

# By default Redis asynchronously dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).
#
# The Append Only File is an alternative persistence mode that provides
# much better durability. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
# 上面英文告诉我们：若同时开启RDB和AOF两种持久化方式，redis重启时会优先通过AOF文件来恢复数据，因为在通常情况下AOF文件保存的数据集要比RDB文件保存的数据集要完整。
#
# Please check http://redis.io/topics/persistence for more information.

appendonly no #默认no，即不开启aof持久化机制

# The name of the append only file (default: "appendonly.aof")

appendfilename "appendonly.aof" #文件名

# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".

# appendfsync always #同步持久化，即每次所执行的写数据指令都会被立刻追加到文件末尾，性能较差但数据完整性比较好
appendfsync everysec #异步持久化，每秒记录已执行过的写数据指令，如果一秒内出现宕机，则可能出现数据丢失，致使AOF文件不完整，进而在再次重启Redis恢复数据时出现异常，此时可通过Redis-check-aof --fix指令修复该AOF文件，该指令会将不完整的错误指令删除，修复后再次重启Redis即可。
# appendfsync no #从不持久化

# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync none". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.

no-appendfsync-on-rewrite no #重写时是否可以运用Appendfsync，用默认no即可，保证数据安全性。

# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.

# AOF机制采用文件追加方式实现数据持久化，这种方式会使文件越来越大，为避免出现这种情况，Redis提供了重写机制，默认情况下当AOF文件大小是上次rewrite后文件大小的一倍且文件大于64M时触发重写机制，也可以通过bgrewriteaof指令手动触发该重写机制，该机制触发后Redis会启动AOF文件的内容压缩，只保留可恢复数据的最小指令集。重写aof文件的操作，并没有读取旧的aof文件，而是将整个内存中的数据库内容用命令的方式重写了一个新的aof文件（先写入临时文件最后再rename）。
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

# An AOF file may be found to be truncated at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can't happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted in the middle
# the server will still exit with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
aof-load-truncated yes

# When rewriting the AOF file, Redis is able to use an RDB preamble in the
# AOF file for faster rewrites and recoveries. When this option is turned
# on the rewritten AOF file is composed of two different stanzas:
#
#   [RDB file][AOF tail]
#
# When loading Redis recognizes that the AOF file starts with the "REDIS"
# string and loads the prefixed RDB file, and continues loading the AOF
# tail.
aof-use-rdb-preamble yes

4、区别

a、RDB持久化机制适合恢复大数据集，但在定时持久化之前一旦出现宕机将丢失此前没来得及持久化的所有数据，如下演示：

删掉/usr/local/bin目录dump.rdb文件—>连接Redis—>set指令添加一条数据—>手动“关机”Linux系统—>启动Linux系统—>连接Redis—>执行keys * 指令，发现并没有找到set指令添加的数据，/usr/local/bin目录也不存在dump.rdb文件。

注意：对于“set指令添加一条数据—>手动“关机”Linux系统”操作需要注意如下问题：set指令添加数据后，必须在15分钟内手动“关机”，否则将会触发RDB机制，生成dump.rdb文件

b、持久化相同数量的数据集时，AOF文件比RDB文件大，数据恢复的速度没有RDB快，但数据同步效率比RDB高，进而数据安全性比RDB高。