• Redis学习--从节点过期键清理策略


    复制从节点过期键清理

    在周期时间事件serverCron-->databasesCron函数中有如下代码:

    /* This function handles 'background' operations we are required to do
     * incrementally in Redis databases, such as active key expiring, resizing,
     * rehashing. */
    void databasesCron(void) {
        /* Expire keys by random sampling. Not required for slaves
         * as master will synthesize DELs for us. */
        if (server.active_expire_enabled) {
            if (server.masterhost == NULL) {
                activeExpireCycle(ACTIVE_EXPIRE_CYCLE_SLOW);
            } else {
                expireSlaveKeys();
            }
        }
    }
    

    参考资料

    https://help.aliyun.com/knowledge_detail/150185.html

    对于复制主节点,会使用activeExpireCycle(ACTIVE_EXPIRE_CYCLE_SLOW)来进行清理过期键,标准源码中该方法每次调用最多执行25ms。

    对于复制从节点,会使用expireSlaveKeys()来清理在读写模式下从库单独写入的过期键

    expireSlaveKeys函数

    标准模式下,复制从节点不能单独进行写操作,复制从节点的所有数据都从复制主节点上同步过来,当主节点主动删除key或对过期键进行清理时,主节点操作会同步到从节点上执行,保证主从数据一致。

    当复制从节点开启读写模式后,Redis使用全局变量slaveKeysWithExpire来记录直接写入到辅助从节点上带有过期事件的键:

    /* The dictionary where we remember key names and database ID of keys we may
     * want to expire from the slave. Since this function is not often used we
     * don't even care to initialize the database at startup. We'll do it once
     * the feature is used the first time, that is, when rememberSlaveKeyWithExpire()
     * is called.
     *
     * The dictionary has an SDS string representing the key as the hash table
     * key, while the value is a 64 bit unsigned integer with the bits corresponding
     * to the DB where the keys may exist set to 1. Currently the keys created
     * with a DB id > 63 are not expired, but a trivial fix is to set the bitmap
     * to the max 64 bit unsigned value when we know there is a key with a DB
     * ID greater than 63, and check all the configured DBs in such a case. */
    dict *slaveKeysWithExpire = NULL;
    
    /* Track keys that received an EXPIRE or similar command in the context
     * of a writable slave. */
    void rememberSlaveKeyWithExpire(redisDb *db, robj *key) {
        if (slaveKeysWithExpire == NULL) {
            static dictType dt = {
                dictSdsHash,                /* hash function */
                NULL,                       /* key dup */
                NULL,                       /* val dup */
                dictSdsKeyCompare,          /* key compare */
                dictSdsDestructor,          /* key destructor */
                NULL                        /* val destructor */
            };
            slaveKeysWithExpire = dictCreate(&dt,NULL);
        }
        if (db->id > 63) return;
    
        dictEntry *de = dictAddOrFind(slaveKeysWithExpire,key->ptr);
        /* If the entry was just created, set it to a copy of the SDS string
         * representing the key: we don't want to need to take those keys
         * in sync with the main DB. The keys will be removed by expireSlaveKeys()
         * as it scans to find keys to remove. */
        if (de->key == key->ptr) {
            de->key = sdsdup(key->ptr);
            dictSetUnsignedIntegerVal(de,0);
        }
    
        uint64_t dbids = dictGetUnsignedIntegerVal(de);
        dbids |= (uint64_t)1 << db->id;
        dictSetUnsignedIntegerVal(de,dbids);
    }
    

    由于全局变量slaveKeysWithExpire记录的过期键仅在复制从节点上存在,因此需要使用单独的expireSlaveKeys函数中基于全局变量slaveKeysWithExpire来进行过期数据清理。

    /* Check the set of keys created by the master with an expire set in order to
     * check if they should be evicted. */
    void expireSlaveKeys(void) {
        if (slaveKeysWithExpire == NULL ||
            dictSize(slaveKeysWithExpire) == 0) return;
    
        int cycles = 0, noexpire = 0;
        mstime_t start = mstime();
        while(1) {
            dictEntry *de = dictGetRandomKey(slaveKeysWithExpire);
            sds keyname = dictGetKey(de);
            uint64_t dbids = dictGetUnsignedIntegerVal(de);
            uint64_t new_dbids = 0;
    
            /* Check the key against every database corresponding to the
             * bits set in the value bitmap. */
            int dbid = 0;
            while(dbids && dbid < server.dbnum) {
                if ((dbids & 1) != 0) {
                    redisDb *db = server.db+dbid;
                    dictEntry *expire = dictFind(db->expires,keyname);
                    int expired = 0;
    
                    if (expire &&
                        activeExpireCycleTryExpire(server.db+dbid,expire,start))
                    {
                        expired = 1;
                    }
    
                    /* If the key was not expired in this DB, we need to set the
                     * corresponding bit in the new bitmap we set as value.
                     * At the end of the loop if the bitmap is zero, it means we
                     * no longer need to keep track of this key. */
                    if (expire && !expired) {
                        noexpire++;
                        new_dbids |= (uint64_t)1 << dbid;
                    }
                }
                dbid++;
                dbids >>= 1;
            }
    
            /* Set the new bitmap as value of the key, in the dictionary
             * of keys with an expire set directly in the writable slave. Otherwise
             * if the bitmap is zero, we no longer need to keep track of it. */
            if (new_dbids)
                dictSetUnsignedIntegerVal(de,new_dbids);
            else
                dictDelete(slaveKeysWithExpire,keyname);
    
            /* Stop conditions: found 3 keys we cna't expire in a row or
             * time limit was reached. */
            cycles++;
            if (noexpire > 3) break;
            if ((cycles % 64) == 0 && mstime()-start > 1) break;
            if (dictSize(slaveKeysWithExpire) == 0) break;
        }
    }
    
  • 相关阅读:
    Android Studio使用笔记
    Android Material Design之在RecyclerView中嵌套CardView实现
    RR 和RC 幻读问题
    mysql rr和rc区别
    7.2 Database Backup Methods 数据备份方法:
    7.1 Backup and Recovery Types 备份和恢复类型
    Chapter 7 Backup and Recovery 备份和恢复:
    mysqldump 一些参数体验
    (?m) 可以让.去匹配换行
    perl 正则前导字符
  • 原文地址:https://www.cnblogs.com/gaogao67/p/15131323.html
Copyright © 2020-2023  润新知