redis : Can't save in background: fork: Cannot allocate memory

Posted 2020-11-26 Hank.Yoon

redis : Can‘t save in background: fork: Cannot allocate memory

JAVA程序报错信息：

MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error

查看redis日志：

18793:S 02 Dec 10:02:02.069 # Can‘t save in background: fork: Cannot allocate memory
18793:S 02 Dec 10:02:08.088 # Can‘t save in background: fork: Cannot allocate memory
18793:S 02 Dec 10:02:14.006 # Can‘t save in background: fork: Cannot allocate memory
18793:S 02 Dec 10:02:20.021 # Can‘t save in background: fork: Cannot allocate memory
18793:S 02 Dec 10:02:26.038 # Can‘t save in background: fork: Cannot allocate memory
18793:S 02 Dec 10:02:32.054 # Can‘t save in background: fork: Cannot allocate memory
18793:S 02 Dec 10:02:38.067 # Can‘t save in background: fork: Cannot allocate memory
18793:S 02 Dec 10:02:44.086 # Can‘t save in background: fork: Cannot allocate memory
18793:S 02 Dec 10:02:50.002 # Can‘t save in background: fork: Cannot allocate memory
18793:S 02 Dec 10:02:56.017 # Can‘t save in background: fork: Cannot allocate memory
18793:S 02 Dec 10:03:02.037 # Can‘t save in background: fork: Cannot allocate memory
18793:S 02 Dec 10:03:08.056 # Can‘t save in background: fork: Cannot allocate memory
18793:S 02 Dec 10:03:14.073 # Can‘t save in background: fork: Cannot allocate memory
18793:S 02 Dec 10:03:20.091 # Can‘t save in background: fork: Cannot allocate memory
18793:S 02 Dec 10:03:26.007 # Can‘t save in background: fork: Cannot allocate memory

数据回写分同步和异步两种方式:
同步回写(SAVE), 主进程直接向磁盘回写数据. 在数据量大的情况下会导致系统假死很长时间
异步回写(BGSAVE), 主进程fork后, 复制自身并通过这个新的进程回写磁盘, 回写结束后新进程自行关闭

由于 BGSAVE 不需要主进程阻塞, 系统也不会假死, 一般会采用 BGSAVE 来实现数据回写.

redis在dump数据的时候会启动fork子进程，由于内存不够，导致无法持久化落盘

redis有个默认的选项：

stop-writes-on-bgsave-error yes

这个选项默认情况下，如果在RDB snapshots持久化过程中出现问题，设置该参数后，Redis是不允许用户进行任何更新操作。

 

不彻底的解决方式是，将这个选项改为false

stop-writes-on-bgsave-error false

但是这样只是当redis写硬盘快照出错时，可以让用户继续做更新操作，但是写硬盘仍然是失败的

 

彻底解决方案：直接修改内核参数 vm.overcommit_memory = 1
编辑文件 /etc/sysctl.conf 添加：
vm.overcommit_memory=1
执行sysctl -p使其生效

Linux内核会根据参数vm.overcommit_memory参数的设置决定是否放行。

vm.overcommit_memory = 1，直接放行
vm.overcommit_memory = 0：则比较 此次请求分配的虚拟内存大小和系统当前空闲的物理内存加上swap，决定是否放行。
vm.overcommit_memory = 2：则会比较进程所有已分配的虚拟内存加上此次请求分配的虚拟内存和系统当前的空闲物理内存加上swap，决定是否放行。