Redis配置文件详解和持久化实现（RDBAOF）
Posted Janzen_Q
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Redis配置文件详解和持久化实现（RDBAOF）相关的知识，希望对你有一定的参考价值。
Redis配置文件详解
bind 0.0.0.0 #监听地址，可用空格隔开多个IP 默认127.0.0.1 protected-mode yes #安全访问限制，启用状态下若停用bind IP和访问密码时，仅允许通过127.0.0.1:6379进行远程访问，访问时将出现告警提示，拒绝其他远程访问 默认 yes port 6379 #服务监听tcp端口，默认6379
Redis RDB持久化及AOF持久化
一、RDB 实现模式：
1、save 手动执行：save期间会阻塞其他请求执行
2、bgsave 手动执行：后台异步执行，不会阻塞其他请求
3、自动执行（默认）：根据配置文件中 save 规则自动触发RDB操作
二、RDB相关配置详解：
################################ SNAPSHOTTING ################################ # # Save the DB on disk: # # save <seconds> <changes> # # Will save the DB if both the given number of seconds and the given # number of write operations against the DB occurred. # # In the example below the behaviour will be to save: # after 900 sec (15 min) if at least 1 key changed # after 300 sec (5 min) if at least 10 keys changed # after 60 sec if at least 10000 keys changed # # Note: you can disable saving completely by commenting out all "save" lines. # # It is also possible to remove all the previously configured save # points by adding a save directive with a single empty string argument # like in the following example: # # save "" save 900 1 #在 900秒 内有大于等于 1 个key的值发生改变触发RDB备份 save 300 10 #在 300秒 内有大于等于 10 个key的值发生改变触发RDB备份
save 60 10000 #在 60秒 内有大于等于 1000 个key的值发生改变触发RDB备份
# By default Redis will stop accepting writes if RDB snapshots are enabled
# (at least one save point) and the latest background save failed.
# This will make the user aware (in a hard way) that data is not persisting
# on disk properly, otherwise chances are that no one will notice and some
# disaster will happen.
#
# If the background saving process will start working again Redis will
# automatically allow writes again.
#
# However if you have setup your proper monitoring of the Redis server
# and persistence, you may want to disable this feature so that Redis will
# continue to work as usual even if there are problems with disk,
# permissions, and so forth.
stop-writes-on-bgsave-error yes #当出现空间不足等原因导致RDB文件保存错误时，禁止redis写入，减少发生故障时的数据丢失量，会造成业务中断。不禁止写入，数据丢失风险增大，但当前不会造成业务中断 默认yes,生产环境建议修改为 no
# Compress string objects using LZF when dump .rdb databases?
# For default that\'s set to \'yes\' as it\'s almost always a win.
# If you want to save some CPU in the saving child set it to \'no\' but
# the dataset will likely be bigger if you have compressible values or keys.
rdbcompression yes #持久化生成RDB文件进行压缩，默认 yes
# Since version 5 of RDB a CRC64 checksum is placed at the end of the file.
# This makes the format more resistant to corruption but there is a performance
# hit to pay (around 10%) when saving and loading RDB files, so you can disable it
# for maximum performances.
#
# RDB files created with checksum disabled have a checksum of zero that will
# tell the loading code to skip the check.
rdbchecksum yes #对备份文件进行CRC64校验，默认yes
# The filename where to dump the DB
dbfilename dump.rdb #RDB文件名（无需携带路径）。触发RDB完成后服务自动生成，服务重启后未开启AOF模式情况下，会直接通过此文件加载数据，默认 dump.rdb
# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the \'dbfilename\' configuration directive.
#
# The Append Only File will also be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir ./ #备份文件存放目录 默认为 ./，注意目录权限问题注意：每次执行RDB时。新生成的rdb文件会覆盖原有文件，因此要注意对历史rdb文件进行定期备份
三、AOF相关配置详解
############################## APPEND ONLY MODE ###############################
# By default Redis asynchronously dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).
#
# The Append Only File is an alternative persistence mode that provides
# much better durability. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Please check http://redis.io/topics/persistence for more information.
appendonly no #启用AOF,优先级高于RDB，对于已存在数据的redis应手动rewriteaof生成aof重写文件后，再修改配置文件重启服务
# The name of the append only file (default: "appendonly.aof")
appendfilename "appendonly_6379.aof" #AOF文件名称，存放于 dir 参数配置的目录下，默认 appendonly.aof
# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don\'t fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that\'s usually the right compromise between
# speed and data safety. It\'s up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that\'s snapshotting),
# or on the contrary, use "always" that\'s very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".
# appendfsync always
appendfsync everysec #AOF同步策略,默认 everysec
# no 不通过执行fsync，由操作系统保证将数据同步至磁盘，Linux默认fsync策略为30s,最多会导致丢失30s数据
# everysec 每秒执行一次fsync操作，最多导致丢失1s的数据，对性能压力较小，安全性较高，生产环境建议值
# always 每次写入数据都执行fsync操作，对服务器性能负担大，性能最差，但安全性最高
# appendfsync no
# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it\'s possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync none". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.
no-appendfsync-on-rewrite no #执行bgrewriteaof期间，对新增append数据，暂缓使用AOF同步策略，默认 no
#当同时执行bgrewriteaof操作和主进程写入aof时，由于bgrewriteaof会占用大量磁盘操作，会导致主进程执行aof时出现阻塞情况。主要影响磁盘IO和请求阻塞时间
# no : 不暂缓写入，aof记录依旧立即同步至磁盘，不会发生数据丢失情况，但需承担请求阻塞风险
# yes : 暂缓写入aof记录（appendfsync no），临时将数据写入缓冲区内，由操作系统调度写入磁盘，如果此时发生redis宕机会造成缓冲区数据丢失，丢失数据量为操作系统的fsync策略（Linux默认30s），性能较好不会触发请求阻塞
# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.
#rewriteaof 可以保证AOF文件的精简及完整，启动时更快的加载数据
auto-aof-rewrite-percentage 100 #自动触发rewriteaof的增长百分比，设置为0时表示不触发自动rewrite aof, 默认 100
auto-aof-rewrite-min-size 64mb #自动触发rewriteaof的最小文件容量，默认 64mb
# An AOF file may be found to be truncated at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can\'t happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted in the middle
# the server will still exit with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
aof-load-truncated yes #允许加载由于某些情况（AOF写入过程中，进程被Kill 或 断电）导致末尾文件异常的AOF文件，默认 yes ,建议 yes
# When rewriting the AOF file, Redis is able to use an RDB preamble in the
# AOF file for faster rewrites and recoveries. When this option is turned
# on the rewritten AOF file is composed of two different stanzas:
#
# [RDB file][AOF tail]
#
# When loading Redis recognizes that the AOF file starts with the "REDIS"
# string and loads the prefixed RDB file, and continues loading the AOF
# tail.
aof-use-rdb-preamble yes #同时启用AOF和RDB持久化配置，启用后 AOF重写文件将同时包含AOF格式及RDB格式，RDB格式文件用于保存历史存量数据，AOF格式文件用于记录近期增量数据，能够更快速生成重写文件，并在出现问题时快速载入数据，默认 no
四、启用AOF策略的正确顺序
#手动查看redis aof配置是否启动
[root@Redis-Ubuntu-1804-p22:~]# redis-cli -a redis --no-auth-warning config get appendonly 1) "appendonly" 2) "no"
#启动aof策略，并查看当前进程及目录文件情况
[root@Redis-Ubuntu-1804-p22:~]# redis-cli -a redis --no-auth-warning config set appendonly yes;pstree | grep redis-server ; ll /app/redis/data/ OK |-2*[redis-server---3*[redis-server]] |-redis-server-+-redis-server | `-3*[redis-server] total 127072 drwxr-xr-x 2 redis redis 4096 Apr 15 03:46 ./ drwxr-xr-x 7 redis redis 4096 Apr 14 11:39 ../ -rw-r--r-- 1 redis redis 0 Apr 15 03:46 appendonly_6379.aof -rw-r--r-- 1 redis redis 128895906 Apr 15 03:12 dump_6379.rdb -rw-r--r-- 1 redis redis 93 Apr 14 20:04 dump_6380.rdb -rw-r--r-- 1 redis redis 93 Apr 15 02:49 dump_6381.rdb -rw-r--r-- 1 redis redis 1208320 Apr 15 03:46 temp-rewriteaof-4547.aof [root@Redis-Ubuntu-1804-p22:~]# pstree | grep redis-server ; ll /app/redis/data/ |-3*[redis-server---3*[redis-server]] total 251768 drwxr-xr-x 2 redis redis 4096 Apr 15 03:46 ./ drwxr-xr-x 7 redis redis 4096 Apr 14 11:39 ../ -rw-r--r-- 1 redis redis 128895906 Apr 15 03:46 appendonly_6379.aof -rw-r--r-- 1 redis redis 128895906 Apr 15 03:12 dump_6379.rdb -rw-r--r-- 1 redis redis 93 Apr 14 20:04 dump_6380.rdb -rw-r--r-- 1 redis redis 93 Apr 15 02:49 dump_6381.rdb
#修改配置文件内的设置 appendonly yes [root@Redis-Ubuntu-1804-p22:~]# sed -i \'s/^appendonly no/appendonly yes/\' /app/redis/etc/redis_6379.conf [root@Redis-Ubuntu-1804-p22:~]# cat /app/redis/etc/redis_6379.conf | grep "^appendonly" appendonly yes
#查看当前redis数据，进行重启验证对比 [root@Redis-Ubuntu-1804-p22:~]# redis-cli -a redis --no-auth-warning info Keyspace # Keyspace db0:keys=4531650,expires=0,avg_ttl=0 #重启redis,验证aof是否正常启用，并校验数据是否准确 [root@Redis-Ubuntu-1804-p22:~]# systemctl restart redis_6379.service [root@Redis-Ubuntu-1804-p22:~]# redis-cli -a redis --no-auth-warning info Keyspace # Keyspace db0:keys=4531650,expires=0,avg_ttl=0 [root@Redis-Ubuntu-1804-p22:~]# ll /app/redis/data total 251772 drwxr-xr-x 2 redis redis 4096 Apr 15 03:57 ./ drwxr-xr-x 7 redis redis 4096 Apr 14 11:39 ../ -rw-r--r-- 1 redis redis 128895906 Apr 15 03:46 appendonly_6379.aof -rw-r--r-- 1 redis redis 128895906 Apr 15 03:57 dump_6379.rdb -rw-r--r-- 1 redis redis 93 Apr 14 20:04 dump_6380.rdb -rw-r--r-- 1 redis redis 93 Apr 15 02:49 dump_6381.rdb [root@Redis-Ubuntu-1804-p22:~]# #日志 3849:signal-handler (1681502220) Received SIGTERM scheduling shutdown... 3849:M 15 Apr 2023 03:57:00.659 # User requested shutdown... 3849:M 15 Apr 2023 03:57:00.659 * Calling fsync() on the AOF file. 3849:M 15 Apr 2023 03:57:00.659 * Saving the final RDB snapshot before exiting. 3849:M 15 Apr 2023 03:57:13.133 * DB saved on disk 3849:M 15 Apr 2023 03:57:13.133 * Removing the pid file. 3849:M 15 Apr 2023 03:57:13.133 # Redis is now ready to exit, bye bye... 4846:C 15 Apr 2023 03:57:13.195 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 4846:C 15 Apr 2023 03:57:13.195 # Redis version=5.0.14, bits=64, commit=00000000, modified=0, pid=4846, just started 4846:C 15 Apr 2023 03:57:13.195 # Configuration loaded 4846:C 15 Apr 2023 03:57:13.195 * supervised by systemd, will signal readiness 4846:M 15 Apr 2023 03:57:13.202 * Running mode=standalone, port=6379. 4846:M 15 Apr 2023 03:57:13.203 # Server initialized 4846:M 15 Apr 2023 03:57:13.204 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command \'echo never > /sys/kernel/mm/transparent_hugepage/enabled\' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled. 4846:M 15 Apr 2023 03:57:13.204 * Reading RDB preamble from AOF file... 4846:M 15 Apr 2023 03:57:17.447 * Reading the remaining AOF tail... 4846:M 15 Apr 2023 03:57:17.447 * DB loaded from append only file: 4.243 seconds 4846:M 15 Apr 2023 03:57:17.448 * Ready to accept connections
五、手动执行bgwriteaof
# 执行 redis-cli -a redis --no-auth-warning bgrewriteaof 并查看进程及文件目录状态
[root@Redis-Ubuntu-1804-p22:~]# redis-cli -a redis --no-auth-warning bgrewriteaof ; pstree | grep redis-server ; ll -h /app/redis/data/ Background append only file rewriting started |-2*[redis-server---3*[redis-server]] |-redis-server-+-redis-server | `-3*[redis-server] total 247M drwxr-xr-x 2 redis redis 4.0K Apr 15 04:32 ./ drwxr-xr-x 7 redis redis 4.0K Apr 14 11:39 ../ -rw-r--r-- 1 redis redis 124M Apr 15 04:26 appendonly_6379.aof -rw-r--r-- 1 redis redis 123M Apr 15 04:29 dump_6379.rdb -rw-r--r-- 1 redis redis 93 Apr 14 20:04 dump_6380.rdb -rw-r--r-- 1 redis redis 93 Apr 15 02:49 dump_6381.rdb -rw-r--r-- 1 redis redis 312K Apr 15 04:32 temp-rewriteaof-4924.aof #对比看到appendonly_6379.aof 经过rewriteaof后文件变小，但不会小于RDB文件 [root@Redis-Ubuntu-1804-p22:~]# ll /app/redis/data -h total 245M drwxr-xr-x 2 redis redis 4.0K Apr 15 04:32 ./ drwxr-xr-x 7 redis redis 4.0K Apr 14 11:39 ../ -rw-r--r-- 1 redis redis 123M Apr 15 04:32 appendonly_6379.aof -rw-r--r-- 1 redis redis 123M Apr 15 04:29 dump_6379.rdb -rw-r--r-- 1 redis redis 93 Apr 14 20:04 dump_6380.rdb -rw-r--r-- 1 redis redis 93 Apr 15 02:49 dump_6381.rdb [root@Redis-Ubuntu-1804-p22:~]# #日志 4846:M 15 Apr 2023 04:32:05.605 * Background append only file rewriting started by pid 4924 4846:M 15 Apr 2023 04:32:23.890 * AOF rewrite child asks to stop sending diffs. 4924:C 15 Apr 2023 04:32:23.890 * Parent agreed to stop sending diffs. Finalizing AOF... 4924:C 15 Apr 2023 04:32:23.890 * Concatenating 0.00 MB of AOF diff received from parent. 4924:C 15 Apr 2023 04:32:23.891 * SYNC append only file rewrite performed 4924:C 15 Apr 2023 04:32:23.897 * AOF rewrite: 0 MB of memory used by copy-on-write 4846:M 15 Apr 2023 04:32:23.950 * Background AOF rewrite terminated with success 4846:M 15 Apr 2023 04:32:23.950 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB) 4846:M 15 Apr 2023 04:32:23.951 * Background AOF rewrite finished successfully
学习笔记——redis持久化之RDBAOF
2023-02-10
一、redis提供了2个不同形式的持久化方式
1、RDB（Redis DataBase）
2、AOF（Append Of File）
二、RDB的定义
RDB是在指定的时间间隔内将内存中的数据集快照写入磁盘，即Snapshot快照，它恢复时是将快照文件直接读到内存里。
三、备份执行的过程
redis会单独创建（fork）一个子进程来进行持久化，会将数据写入到一个临时文件中，待持久化过程都结束了，再用这个临时文件替换上次持久化好的文件。RDB方式要比AOF方式更加的高效。
RDB的缺点最后一次持久化后的数据可能丢失。
四、Fork
1、Fork的作用
它的作用是复制一个与当前进程一样的进程。是作为原进程的子进程。
2、说明
（1）在linux程序中，fork()会产生一个和父进程完全相同的子进程，但子进程在此后会被exe系统调用，处于效率考虑，Linux中引入了“写时复制技术”。
（2）一般情况父进程和子进程会共用同一段物理内存，只有进程空间的各段的内容要发生变化时，才会将父进程的内容复制一份给子进程。
五、redis中RDB快照触发的时机
1、在redis数据库中执行shutdown会触发RDB快照
2、在redis数据库中执行bgsave会触发RDB快照
3、在redis.conf配置文件中设置例如“save 60 10”会触发RDB快照
六、redis持久化之AOF
1、AOF（Append Only File）的含义
它是以日志的形式记录每个写操作（增量保存），将redis执行过的所有写指令记录下来（读操作不记录），只许9追加文件但不可以改写文件，redis启动之初会读取文件重新构建数据，简而言之，redis重启的话就根据日志文件的内容将写指令从前到后执行一次以完成数据的恢复工作。
2、AOF持久化流程
（1）客户端的请求写命令会被append追加到AOF缓冲区内
（2）AOF缓冲区根据AOF持久化策略将操作sync同步到磁盘的AOF文件中
（3）AOF文件大小超过重写策略或手动重写时，会对AOF文件rewrite重写，压缩AOF文件容量
（4）Redis服务重启时，会重新load加载AOF文件中的写操作，达到数据恢复的目的
3、说明
当AOF和RDB同时开启时，redis听AOF的。因为RDB最后一次缓存数据可能保存不完整。
以上是关于Redis配置文件详解和持久化实现（RDBAOF）的主要内容，如果未能解决你的问题，请参考以下文章