[原创]通过切面与分布式锁实现合并相同请求

在微服务里经常有并发相同的请求过来,当未命中缓存时,可能多条请求一起穿透缓存到DB,这就导致DB压力的增大

本文使用redis的分布式锁来合并相同的请求

当两个以上相同的请求来请求时, 通过竞争实现将相同的请求线性化.

假设缓存失效,也只有1个线程去访问DB,其他线程在等待和重试 来降低缓存穿透的风险.

代码如下

1.添加依赖

   <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-autoconfigure</artifactId>
            <version>1.5.2.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
            <version>1.5.2.RELEASE</version>
        </dependency>

<dependency>
            <groupId>org.aspectj</groupId>
            <artifactId>aspectjrt</artifactId>
            <version>1.6.11</version>
        </dependency>

        <dependency>
            <groupId>org.aspectj</groupId>
            <artifactId>aspectjweaver</artifactId>
            <version>1.6.11</version>
        </dependency>
        <!-- cglib -->
        <dependency>
            <groupId>cglib</groupId>
            <artifactId>cglib</artifactId>
            <version>2.1</version>
        </dependency>

        <dependency>
            <groupId>commons-configuration</groupId>
            <artifactId>commons-configuration</artifactId>
            <version>1.10</version>
        </dependency>

        <dependency>
            <groupId>redis.clients</groupId>
            <artifactId>jedis</artifactId>
            <version>2.9.0</version>
        </dependency>

2.添加注解

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

/**
 * 合并请求的注解
 * 仅支持单参数的接口方法
 */
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface MergeDuplicationRequestAttribute {
    /**
     * 分布式锁的key字符串 如 A:b:%s
     * @return
     */
    String redisLockKeyTemplate() default "";

    /**
     * 分布式锁的key组合项, 如id
     * 通过 String.format(redisLockKeyTemplate,getValues(object,fields)) 获取真实的分布式锁的key
     * @return
     */
    String[] redisLockKeyObjectFileds() default {};

    /**
     * 分布式锁的过期时间(毫秒)
     * @return
     */
    int expireMillseconds() default 1000;

    /**
     * 分布式锁的重试间隔(毫秒)
     * @return
     */
    int retryIntervalMillseconds() default 20;

    /**
     * 分布式锁的重试次数
     * @return
     */
    int retryTimes() default 3;
}

3.添加切面

import com.g2.order.server.annotation.MergeDuplicationRequestAttribute;
import com.g2.order.server.config.RedisLock;
import com.g2.order.server.utils.ObjectUtils;

import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.aspectj.lang.reflect.MethodSignature;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.EnableAspectJAutoProxy;
import org.springframework.core.annotation.Order;
import org.springframework.stereotype.Component;

import java.lang.reflect.Method;
import java.util.Arrays;

//开启AspectJ 自动代理模式,如果不填proxyTargetClass=true，默认为false，
@EnableAspectJAutoProxy(proxyTargetClass = true)
@Component
@Order(-1)
@Aspect
public class MergeDuplicationRequestAspect {
    private static Logger logger = LoggerFactory.getLogger(MergeDuplicationRequestAspect.class);

    @Autowired
    private RedisLock redisLock;

    @Pointcut("@annotation(com.g2.order.server.annotation.MergeDuplicationRequestAttribute)")
    public void mergeDuplicationRequest() {

    }

    @Around("mergeDuplicationRequest()")
    public Object handleControllerMethod(ProceedingJoinPoint proceedingJoinPoint) throws Throwable {
        //获取controller对应的方法.
        MethodSignature methodSignature = (MethodSignature) proceedingJoinPoint.getSignature();

        //获取方法
        Method method = methodSignature.getMethod();

        MergeDuplicationRequestAttribute annotation = method.getAnnotation(MergeDuplicationRequestAttribute.class);
        String key = annotation.redisLockKeyTemplate();
        String redisKey = key;
        Object[] args = proceedingJoinPoint.getArgs();
        if (args.length > 0) {
            Object param = args[0];
            String[] paramFields = annotation.redisLockKeyObjectFileds();
            if (paramFields.length > 0) {
                String propertiesValue = ObjectUtils.getPropertiesValue(param, Arrays.asList(paramFields));
                redisKey = String.format(key, propertiesValue);
            }
        }

        int retryIntervalMillseconds = annotation.retryIntervalMillseconds();
        int retryTimes = annotation.retryTimes();
        int expireMillseconds = annotation.expireMillseconds();
        if (!redisLock.lock(redisKey, expireMillseconds)) {
            for (int i = 1; i <= retryTimes; i++) {
                try {
                    logger.info("有相同的请求,第{}次休眠",i);
                    Thread.sleep(retryIntervalMillseconds);
                } catch (InterruptedException ex) {

                }

                if (redisLock.lock(redisKey, expireMillseconds)) {
                    break;
                }
            }
        }

        return proceedingJoinPoint.proceed();
    }
}

3.添加分布式锁代码(使用redis集群)

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.util.StringUtils;

import java.util.Arrays;
import java.util.Set;
import java.util.stream.Collectors;

import lombok.extern.slf4j.Slf4j;
import redis.clients.jedis.HostAndPort;
import redis.clients.jedis.JedisCluster;
import redis.clients.jedis.JedisPoolConfig;

/**
 * 自定义Redis配置类
 *
 *  
 * @date 2017/10/19
 */
@Slf4j
@Configuration
public class RedisConfig {

    /**
     * jedisCluster
     */
    @Bean
    @Autowired
    public JedisCluster jedisCluster(@Qualifier("jedis.pool.config") JedisPoolConfig config,
            @Value("${redis.host.address}") String hostAndPort,
            @Value("${redis.password}") String password) {
        log.info("开始初始化redis...");
        /**
         * 1 先检查redis集群是否已经配置
         */
        if (StringUtils.isEmpty(hostAndPort)) {
            throw new RuntimeException("Redis 集群初始化异常。请检查配置redis.host.address配置项");
        }

        /**
         * 2 根据配置构建hostAndPorts
         */
        Set<HostAndPort> hostAndPorts = Arrays.asList(hostAndPort.split(",")).stream().map(s -> {
            String[] split = s.split(":");
            return new HostAndPort(split[0], Integer.valueOf(split[1]));
        }).collect(Collectors.toSet());

        return new JedisCluster(hostAndPorts, 1000, 1000, 1, password, config);
    }

    @Bean(name = "jedis.pool.config")
    public JedisPoolConfig jedisPoolConfig(@Value("${jedis.pool.config.maxTotal}") int maxTotal,
            @Value("${jedis.pool.config.maxWaitMillis}") int maxWaitMillis,
            @Value("${jedis.pool.config.maxIdle}") int maxIdle) {
        JedisPoolConfig config = new JedisPoolConfig();
        config.setMaxTotal(maxTotal);
        config.setMaxIdle(maxIdle);
        config.setMaxWaitMillis(maxWaitMillis);
        return config;
    }
}

/**
 * 自定义Redis服务
 *
 * 
 * @date 2017/10/19
 */
public interface RedisLock {
    String OK_CODE = "OK";
    String OK_MULTI_CODE = "+OK";

    /**
     * 加锁
     *
     * @param lockKey 锁key
     * @param millseconds 过期时间(毫秒)
     * @return true:成功获取锁;false:没有获取到锁
     */
    boolean lock(final String lockKey, final int millseconds);

    /**
     * 解锁
     *
     * @param key 锁key
     * @return true:成功解锁;
     */
    boolean unlock(String key);

    default boolean isStatusOk(String status) {
        return (status != null) && (OK_CODE.equals(status) || OK_MULTI_CODE.equals(status));
    }
}

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import lombok.extern.slf4j.Slf4j;
import redis.clients.jedis.JedisCluster;

/**
 * 自定义Redis服务
 *
 * 
 * @date 2017/10/19
 */
@Service
@Slf4j
public class DefaultRedisLock implements RedisLock {

    @Autowired
    private JedisCluster redisService;

    @Override
    public boolean lock(String lockKey, int millseconds) {
        return isStatusOk(redisService.set(lockKey, "1", "NX", "PX", millseconds));
    }

    @Override
    public boolean unlock(String lockKey) {
        return redisService.del(lockKey) == 1;
    }
}

4.添加辅助类

import com.google.common.collect.Lists;
import java.beans.IntrospectionException;
import java.beans.PropertyDescriptor;
import java.lang.reflect.Array;
import java.lang.reflect.Field;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.List;
import java.util.Map;
import java.util.function.Function;
import java.util.stream.Collectors;

/**
 *  Object帮助类
 */
public class ObjectUtils {

    public static Object getValue(Object object, String propertyName)
            throws IntrospectionException, IllegalAccessException, InvocationTargetException {
        Class aClass = object.getClass();
        if (isBaseClassOrString(aClass)) {
            return object.toString();
        }

        if (isArrayOrList(aClass)) {
            return object.toString();
        }

        if (isMap(aClass)) {
            return ((Map) object).getOrDefault(propertyName, "null");
        }

        Field[] fields = aClass.getDeclaredFields();

        for (Field f : fields) {
            if (!f.getName().equals(propertyName)) {
                continue;
            }

            PropertyDescriptor descriptor = new PropertyDescriptor(f.getName(), aClass);
            Method readMethod = descriptor.getReadMethod();
            Object result = readMethod.invoke(object);
            return result;
        }

        return "null";
    }

    public static String getPropertiesValue(Object object, List<String> propertyNames)
            throws IntrospectionException, IllegalAccessException, InvocationTargetException {
        return getPropertiesValue(object, propertyNames, "_");
    }

    public static String getPropertiesValue(Object object, List<String> propertyNames, String delimiter)
            throws IntrospectionException, IllegalAccessException, InvocationTargetException {

        Function<List<Object>, String> joinFunction = list ->
                list.stream().map(p -> p == null ? "null" : p.toString()).collect(Collectors.joining(delimiter));

        return getPropertiesValue(object, propertyNames, joinFunction);
    }

    public static String getPropertiesValue(Object object, List<String> propertyNames, Function<List<Object>, String>
            joinFunction)
            throws IntrospectionException, IllegalAccessException, InvocationTargetException {
        List<Object> objects = Lists.newArrayList();
        for (String p : propertyNames) {
            Object result = getValue(object, p);
            objects.add(result);
        }

        return joinFunction.apply(objects);
    }

    private static boolean isBaseClassOrString(Class aClass) {
        return (aClass == String.class)
                || (aClass == Byte.class)
                || (aClass == Short.class)
                || (aClass == Integer.class)
                || (aClass == Double.class)
                || (aClass == Long.class)
                || (aClass == Boolean.class)
                || (aClass == Float.class)
                || (aClass == java.lang.Character.class);
    }

    private static boolean isArrayOrList(Object obj) {
        return obj instanceof Array
                || obj instanceof List;
    }

    private static boolean isMap(Object obj) {
        return obj instanceof Map;
    }
}

5.添加 启动代码及业务代码 和配置项

import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.web.servlet.ServletComponentScan;


/**
 * 程序入口
 */
@SpringBootApplication
public class App {
    public static void main(String[] args) {
        SpringApplication.run(App.class, args);
    }
}

import com.g2.order.server.annotation.MergeDuplicationRequestAttribute;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

import io.swagger.annotations.Api;

@Api(value = "H5Controller", description = "H5接口")
@RestController
@RequestMapping("/h5")
public class H5Controller {
    private static Logger logger = LoggerFactory.getLogger(H5Controller.class);

    @MergeDuplicationRequestAttribute(redisLockKeyTemplate = "A:%s", redisLockKeyObjectFileds = {"code"})
    @RequestMapping(value = "/{code}.jsonp",method = RequestMethod.GET)
    public Object testJsonp2(@PathVariable("code") String code) {
        try {
            Thread.sleep(1000);
        }catch (Exception ex){

        }
        return "234";
    }
}

server.port=88
redis.host.address=127.0.0.1:6389,127.0.0.1:6479,127.0.0.1:6579
redis.password=******
jedis.pool.config.maxTotal=100
jedis.pool.config.maxIdle=10
jedis.pool.config.maxWaitMillis=100000

6.测试结果如下:

当快速刷新两个浏览器 访问相同的地址 http://127.0.0.1:88/h5/123.jsonp 时,产生如下日志

2019-04-19 18:06:34.458  INFO 15080 --- [p-nio-88-exec-2] .s.a.MergeDuplicationRequestAspectConfig : 有相同的请求,第1次休眠
2019-04-19 18:06:34.479  INFO 15080 --- [p-nio-88-exec-2] .s.a.MergeDuplicationRequestAspectConfig : 有相同的请求,第2次休眠
2019-04-19 18:06:34.500  INFO 15080 --- [p-nio-88-exec-2] .s.a.MergeDuplicationRequestAspectConfig : 有相同的请求,第3次休眠
2019-04-19 18:06:56.061  INFO 15080 --- [p-nio-88-exec-5] .s.a.MergeDuplicationRequestAspectConfig : 有相同的请求,第1次休眠
2019-04-19 18:06:56.082  INFO 15080 --- [p-nio-88-exec-5] .s.a.MergeDuplicationRequestAspectConfig : 有相同的请求,第2次休眠
2019-04-19 18:06:56.103  INFO 15080 --- [p-nio-88-exec-5] .s.a.MergeDuplicationRequestAspectConfig : 有相同的请求,第3次休眠

7.改进方向

7.1分布式锁可能存在线程1释放了线程2的锁.(后期修改成 CAS模式来解决问题)

7.2在重试等待中,应当允许去访问另一个方法来获取缓存数据,如果成功立即中断返回.如果不成功才继续竞争锁

伪代码可能如下:

注解里增加 字段 getCacheFunction ="abc"

在切面里 调用

Object theInstance=proceedingJoinPoint.getTarget();

  Object cacheValueObj=MethodHelper.invoke(theInstance,getCacheFunction ,args)...