AOF Rewrite触发场景
函数rewriteAppendOnlyFileBackground实现了AOF Rewrite操作,并在下列三种场景中被调用:
- 执行bgrewriteaof命令触发,通过bgrewriteaofCommand函数来调用。
- 开启appendonly参数触发,通过startAppendOnly函数来调用。
- 满足AOF Rewrite条件触发,通过周期执行函数serverCron来调用。
bgrewriteaofCommand函数
当客户端执行bgrewriteaof命令时,会交由函数bgrewriteaofCommand进行处理:
void bgrewriteaofCommand(client *c) {
if (server.aof_child_pid != -1) {
addReplyError(c,"Background append only file rewriting already in progress");
} else if (server.rdb_child_pid != -1) {
server.aof_rewrite_scheduled = 1;
addReplyStatus(c,"Background append only file rewriting scheduled");
} else if (rewriteAppendOnlyFileBackground() == C_OK) {
addReplyStatus(c,"Background append only file rewriting started");
} else {
addReply(c,shared.err);
}
}
startAppendOnly函数
当客户端设置参数appendonly yes
来开启AOF日志,或则主从复制重新同步后开启AOF日志,都会调用startAppendOnly函数执行:
/* Called when the user switches from "appendonly no" to "appendonly yes"
* at runtime using the CONFIG command. */
int startAppendOnly(void) {
char cwd[MAXPATHLEN]; /* Current working dir path for error messages. */
int newfd;
newfd = open(server.aof_filename,O_WRONLY|O_APPEND|O_CREAT,0644);
serverAssert(server.aof_state == AOF_OFF);
if (newfd == -1) {
char *cwdp = getcwd(cwd,MAXPATHLEN);
serverLog(LL_WARNING,
"Redis needs to enable the AOF but can't open the "
"append only file %s (in server root dir %s): %s",
server.aof_filename,
cwdp ? cwdp : "unknown",
strerror(errno));
return C_ERR;
}
if (server.rdb_child_pid != -1) {
server.aof_rewrite_scheduled = 1;
serverLog(LL_WARNING,"AOF was enabled but there is already a child process saving an RDB file on disk. An AOF background was scheduled to start when possible.");
} else {
/* If there is a pending AOF rewrite, we need to switch it off and
* start a new one: the old one cannot be reused because it is not
* accumulating the AOF buffer. */
if (server.aof_child_pid != -1) {
serverLog(LL_WARNING,"AOF was enabled but there is already an AOF rewriting in background. Stopping background AOF and starting a rewrite now.");
killAppendOnlyChild();
}
if (rewriteAppendOnlyFileBackground() == C_ERR) {
close(newfd);
serverLog(LL_WARNING,"Redis needs to enable the AOF but can't trigger a background AOF rewrite operation. Check the above logs for more info about the error.");
return C_ERR;
}
}
/* We correctly switched on AOF, now wait for the rewrite to be complete
* in order to append data on disk. */
server.aof_state = AOF_WAIT_REWRITE;
server.aof_last_fsync = server.unixtime;
server.aof_fd = newfd;
return C_OK;
}
在Redis主从复制全量同步过程中,从节点加载解析RDB文件过程中,会临时性关闭AOF日志,等RDB文件加载结束后,无论加载成功或失败,都会通过restartAOFAfterSYNC函数来恢复临时关闭的AOF日志。
/* This function will try to re-enable the AOF file after the
* master-replica synchronization: if it fails after multiple attempts
* the replica cannot be considered reliable and exists with an
* error. */
void restartAOFAfterSYNC() {
unsigned int tries, max_tries = 10;
for (tries = 0; tries < max_tries; ++tries) {
if (startAppendOnly() == C_OK) break;
serverLog(LL_WARNING,
"Failed enabling the AOF after successful master synchronization! "
"Trying it again in one second.");
sleep(1);
}
if (tries == max_tries) {
serverLog(LL_WARNING,
"FATAL: this replica instance finished the synchronization with "
"its master, but the AOF can't be turned on. Exiting now.");
exit(1);
}
}
serverCron函数
Redis会周期检查AOF文件大小来判断是否需要进行AOF Rewrite操作:
/* This is our timer interrupt, called server.hz times per second.
* Here is where we do a number of things that need to be done asynchronously.
* For instance:
*
* - Triggering BGSAVE / AOF rewrite, and handling of terminated children.
*/
int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
/* Start a scheduled AOF rewrite if this was requested by the user while
* a BGSAVE was in progress. */
if (server.rdb_child_pid == -1 && server.aof_child_pid == -1 &&
server.aof_rewrite_scheduled)
{
rewriteAppendOnlyFileBackground();
}
/* Check if a background saving or AOF rewrite in progress terminated. */
if (server.rdb_child_pid != -1 || server.aof_child_pid != -1 ||
ldbPendingChildren())
{
/* */
} else {
/* Trigger an AOF rewrite if needed. */
if (server.aof_state == AOF_ON &&
server.rdb_child_pid == -1 &&
server.aof_child_pid == -1 &&
server.aof_rewrite_perc &&
server.aof_current_size > server.aof_rewrite_min_size)
{
long long base = server.aof_rewrite_base_size ?
server.aof_rewrite_base_size : 1;
long long growth = (server.aof_current_size*100/base) - 100;
if (growth >= server.aof_rewrite_perc) {
serverLog(LL_NOTICE,"Starting automatic rewriting of AOF on %lld%% growth",growth);
rewriteAppendOnlyFileBackground();
}
}
}
}
可以通过下面两个参数来控制AOF Rewrite操作:
-
参数auto-aof-rewrite-min-size,当AOF文件超过该阈值后才可能触发AOF重写,避免对较小的AOF文件进行重写,默认为64MB。
-
参数auto-aof-rewrite-percentage,当AOF文件增长比例超过该阈值后才会触发AOF重写,基准值是上一次AOF重写后的AOF文件大小,默认值为100,即"当前AOF文件大小"是"上一次AOF重写后文件大小"的2倍时触发AOF重写。
当AOF重写完成时,会设置AOF重写的基准值:
/* A background append only file rewriting (BGREWRITEAOF) terminated its work.
* Handle this. */
void backgroundRewriteDoneHandler(int exitcode, int bysignal) {
/* 更新AOF重写后AOF文件大小 */
aofUpdateCurrentSize();
/* 设置下一次AOF重写的基准值 */
server.aof_rewrite_base_size = server.aof_current_size;
server.aof_fsync_offset = server.aof_current_size;
}
AOF重写信息查看
可以通过客户端命令info persistence
来查看AOF相关信息:
# 执行命令
redis-cli -p 6566 info persistence
# 输出结果
# Persistence
loading:0
rdb_changes_since_last_save:174
rdb_bgsave_in_progress:0
rdb_last_save_time:1654524706
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:52
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:970752
# 是否开启AOF日志
aof_enabled:1
# 是否正在AOF重写
aof_rewrite_in_progress:0
# AOF重写调度设置
aof_rewrite_scheduled:0
# 最近一次AOF重写消耗时间
aof_last_rewrite_time_sec:52
# 当前AOF重写消耗时间,-1表示当前未处于重写状态
aof_current_rewrite_time_sec:-1
# 最近一次AOF重写状态
aof_last_bgrewrite_status:ok
# 最近一次AOF写日志状态
aof_last_write_status:ok
# 最后一次AOF重写过程中,分配给copy-on-write的内存大小(单元byte)
aof_last_cow_size:937984
# 当前AOF文件大小(单元byte)
aof_current_size:2460435827
# 上一次AOF重写后的文件大小(单元byte)
aof_base_size:2459418938
# 是否等待AOF重写
aof_pending_rewrite:0
aof_buffer_length:0
aof_rewrite_buffer_length:0
aof_pending_bio_fsync:0
aof_delayed_fsync:0
AOF重写并发控制
为保证AOF重写的准确性,需要保证同一时间只能有一个AOF Rewrite线程执行,为保证AOF重写性能和避免AOF重写消耗过多服务器资源,需要避免AOF重写和RDB备份同时进行,因此在开启AOF重写调度(server.aof_rewrite_scheduled = 1) 和创建AOF重写线程时,都会检查当前是否有RDB备份进程和AOF重写进程。