Lettuce是一个可伸缩线程安全的Redis客户端。多个线程可以共享同一个RedisConnection.本文是基于Lettuce5,主要介绍的知识点如下:
- Lettuce在Spring Boot中的配置
- Lettuce的同步,异步,响应式使用方式
- 事件的订阅
- 发布自定义事件
- 读写分离
- 读写分离策略实现源码
- 客户端分片实现
@Configuration
public class LettuceConfig {
/**
* 配置客户端资源
* @return
*/
@Bean(destroyMethod = "shutdown")
ClientResources clientResources() {
return DefaultClientResources.builder().ioThreadPoolSize(8).computationThreadPoolSize(10).build();
}
/**
* 配置Socket选项
* keepAlive=true
* tcpNoDelay=true
* connectionTimeout=5秒
* @return
*/
@Bean
SocketOptions socketOptions(){
return SocketOptions.builder().keepAlive(true).tcpNoDelay(true).connectTimeout(Duration.ofSeconds(5)).build();
}
/**
* 配置客户端选项
* @return
*/
@Bean
ClientOptions clientOptions(SocketOptions socketOptions) {
return ClientOptions.builder().socketOptions(socketOptions).build();
}
/**
* 创建RedisClient
* @param clientResources 客户端资源
* @param clientOptions 客户端选项
* @return
*/
@Bean(destroyMethod = "shutdown")
RedisClient redisClient(ClientResources clientResources, ClientOptions clientOptions) {
RedisURI uri = RedisURI.builder().withSentinel("xx.xx.xx.xx", 26009).withPassword("abcd1234").withSentinelMasterId("xxx").build();
RedisClient client = RedisClient.create(clientResources, uri);
client.setOptions(clientOptions);
return client;
}
/**
* 创建连接
* @param redisClient
* @return
*/
@Bean(destroyMethod = "close")
StatefulRedisConnection<String, String> connection(RedisClient redisClient) {
return redisClient.connect();
}
}
基本使用
public Mono<ServerResponse> hello(ServerRequest request) throws Exception {
//响应式使用
Mono<String> resp = redisConnection.reactive().get("gxt_new");
//同步使用
redisConnection.sync().get("test");
redisConnection.async().get("test").get(5, TimeUnit.SECONDS);
return ServerResponse.ok().body(resp, String.class);
}
客户端订阅事件
客户端使用事件总线传输运行期间产生的事件;EventBus可以从客户端资源进行配置和获取，并用于客户端和自定义事件。
如下事件可以被客户端发送:
- 连接事件
- 测量事件
- 集群拓扑事件
client.getResources().eventBus().get().subscribe(e -> {
System.out.println("client 订阅事件： " + e);
});
client 订阅事件： ConnectionActivatedEvent [/xx:49910 -> /xx:6008] client 订阅事件： ConnectionActivatedEvent [/xx:49911 -> /xx:6018] client 订阅事件： ConnectedEvent [/xx:49912 -> /xx:6018]
发布事件
发布使用也是通过使用eventBus进行发布事件,Event接口只是一个标签接口
eventBus.publish(new Event() {
@Override
public String toString() {
return "自定义事件";
}
});
订阅者就可以订阅到这个自定义事件了
client 订阅事件： 自定义事件
读写分离
@Bean(destroyMethod = "close")
StatefulRedisMasterSlaveConnection<String, String> statefulRedisMasterSlaveConnection(RedisClient redisClient, RedisURI redisURI) {
StatefulRedisMasterSlaveConnection connection = MasterSlave.connect(redisClient, new Utf8StringCodec(), redisURI);
connection.setReadFrom(ReadFrom.NEAREST);
return connection;
}
}
StatefulRedisMasterSlaveConnection 支持读写分离,通过设置ReadFrom控制读是从哪个节点读取.
| 参数 | 含义 |
| MASTER | 从master节点读取 |
| SLAVE | 从slave节点读取 |
MASTER_PREFERRED |
从master节点读取,如果master节点不可以则从slave节点读取 |
SLAVE_PREFERRED |
从slave节点读取,如果slave节点不可用则倒退到master节点读取 |
NEAREST |
从最近到节点读取 |
具体是如何实现到呢? 下面看一下MasterSlaveConnectionProvider相关源码
//根据意图获取连接
public StatefulRedisConnection<K, V> getConnection(Intent intent) {
if (debugEnabled) {
logger.debug("getConnection(" + intent + ")");
}
//如果readFrom不为null且是READ
if (readFrom != null && intent == Intent.READ) {
//根据readFrom配置从已知节点中选择可用节点描述
List<RedisNodeDescription> selection = readFrom.select(new ReadFrom.Nodes() {
@Override
public List<RedisNodeDescription> getNodes() {
return knownNodes;
}
@Override
public Iterator<RedisNodeDescription> iterator() {
return knownNodes.iterator();
}
});
//如果可选择节点集合为空则抛出异常
if (selection.isEmpty()) {
throw new RedisException(String.format("Cannot determine a node to read (Known nodes: %s) with setting %s",
knownNodes, readFrom));
}
try {
//遍历所有可用节点
for (RedisNodeDescription redisNodeDescription : selection) {
//获取节点连接
StatefulRedisConnection<K, V> readerCandidate = getConnection(redisNodeDescription);
//如果节点连接不是打开到连接则继续查找下一个连接
if (!readerCandidate.isOpen()) {
continue;
}
//返回可用连接
return readerCandidate;
}
//如果没有找到可用连接，默认返回第一个
return getConnection(selection.get(0));
} catch (RuntimeException e) {
throw new RedisException(e);
}
}
//如果没有配置readFrom或者不是READ 则返回master连接
return getConnection(getMaster());
}
我们可以看到选择连接到逻辑是通用的,不同的处理就是在selection的处理上,下面看一下不同readFrom策略对于selection的处理
ReadFromSlavePerferred和ReadFromMasterPerferred都是有优先级到概念,看看相关逻辑的处理
static final class ReadFromSlavePreferred extends ReadFrom {
@Override
public List<RedisNodeDescription> select(Nodes nodes) {
List<RedisNodeDescription> result = new ArrayList<>(nodes.getNodes().size());
//优先添加slave节点
for (RedisNodeDescription node : nodes) {
if (node.getRole() == RedisInstance.Role.SLAVE) {
result.add(node);
}
}
//最后添加master节点
for (RedisNodeDescription node : nodes) {
if (node.getRole() == RedisInstance.Role.MASTER) {
result.add(node);
}
}
return result;
}
static final class ReadFromMasterPreferred extends ReadFrom {
@Override
public List<RedisNodeDescription> select(Nodes nodes) {
List<RedisNodeDescription> result = new ArrayList<>(nodes.getNodes().size());
//优先添加master节点
for (RedisNodeDescription node : nodes) {
if (node.getRole() == RedisInstance.Role.MASTER) {
result.add(node);
}
}
//其次在添加slave节点
for (RedisNodeDescription node : nodes) {
if (node.getRole() == RedisInstance.Role.SLAVE) {
result.add(node);
}
}
return result;
}
}
对于ReadFromMaster和ReadFromSlave都是获取指定角色的节点
static final class ReadFromSlave extends ReadFrom {
@Override
public List<RedisNodeDescription> select(Nodes nodes) {
List<RedisNodeDescription> result = new ArrayList<>(nodes.getNodes().size());
//只获取slave节点
for (RedisNodeDescription node : nodes) {
if (node.getRole() == RedisInstance.Role.SLAVE) {
result.add(node);
}
}
return result;
}
}
static final class ReadFromMaster extends ReadFrom {
@Override
public List<RedisNodeDescription> select(Nodes nodes) {
for (RedisNodeDescription node : nodes) {
if (node.getRole() == RedisInstance.Role.MASTER) {
return LettuceLists.newList(node);
}
}
return Collections.emptyList();
}
}
获取最近的节点这个就有点特殊了,它对已知对节点没有做处理,直接返回了它们的节点描述,也就是谁在前面就优先使用谁
static final class ReadFromNearest extends ReadFrom {
@Override
public List<RedisNodeDescription> select(Nodes nodes) {
return nodes.getNodes();
}
}
在SentinelTopologyProvider中可以发现,获取nodes节点总是优先获取Master节点,其次是slave节点,这样Nearest效果就等效与MasterPreferred
public List<RedisNodeDescription> getNodes() {
logger.debug("lookup topology for masterId {}", masterId);
try (StatefulRedisSentinelConnection<String, String> connection = redisClient.connectSentinel(CODEC, sentinelUri)) {
RedisFuture<Map<String, String>> masterFuture = connection.async().master(masterId);
RedisFuture<List<Map<String, String>>> slavesFuture = connection.async().slaves(masterId);
List<RedisNodeDescription> result = new ArrayList<>();
try {
Map<String, String> master = masterFuture.get(timeout.toNanos(), TimeUnit.NANOSECONDS);
List<Map<String, String>> slaves = slavesFuture.get(timeout.toNanos(), TimeUnit.NANOSECONDS);
//添加master节点
result.add(toNode(master, RedisInstance.Role.MASTER));
//添加所有slave节点
result.addAll(slaves.stream().filter(SentinelTopologyProvider::isAvailable)
.map(map -> toNode(map, RedisInstance.Role.SLAVE)).collect(Collectors.toList()));
} catch (ExecutionException | InterruptedException | TimeoutException e) {
throw new RedisException(e);
}
return result;
}
}
自定义负载均衡
通过上文可以发现只需要实现 ReadFrom接口,就可以通过该接口实现Master,Slave负载均衡;下面的示例是通过将nodes节点进行打乱,进而实现
@Bean(destroyMethod = "close")
StatefulRedisMasterSlaveConnection<String, String> statefulRedisMasterSlaveConnection(RedisClient redisClient, RedisURI redisURI) {
StatefulRedisMasterSlaveConnection connection = MasterSlave.connect(redisClient, new Utf8StringCodec(), redisURI);
connection.setReadFrom(new ReadFrom() {
@Override
public List<RedisNodeDescription> select(Nodes nodes) {
List<RedisNodeDescription> list = nodes.getNodes();
Collections.shuffle(list);
return list;
}
});
return connection;
}
在大规模使用的时候会使用多组主备服务,可以通过客户端分片的方式将部分请求路由到指定的服务器上,但是Lettuce没有提供这样的支持,下面是自定义的实现:
public class Sharded< C extends StatefulRedisConnection,V> {
private TreeMap<Long, String> nodes;
private final Hashing algo = Hashing.MURMUR_HASH;
private final Map<String, StatefulRedisConnection> resources = new LinkedHashMap<>();
private RedisClient redisClient;
private String password;
private Set<HostAndPort> sentinels;
private RedisCodec<String, V> codec;
public Sharded(List<String> masters, RedisClient redisClient, String password, Set<HostAndPort> sentinels, RedisCodec<String, V> codec) {
this.redisClient = redisClient;
this.password = password;
this.sentinels = sentinels;
this.codec = codec;
initialize(masters);
}
private void initialize(List<String> masters) {
nodes = new TreeMap<>();
for (int i = 0; i != masters.size(); ++i) {
final String master = masters.get(i);
for (int n = 0; n < 160; n++) {
nodes.put(this.algo.hash("SHARD-" + i + "-NODE-" + n), master);
}
RedisURI.Builder builder = RedisURI.builder();
for (HostAndPort hostAndPort : sentinels) {
builder.withSentinel(hostAndPort.getHostText(), hostAndPort.getPort());
}
RedisURI redisURI = builder.withPassword(password).withSentinelMasterId(master).build();
resources.put(master, MasterSlave.connect(redisClient, codec, redisURI));
}
}
public StatefulRedisConnection getConnectionBy(String key) {
return resources.get(getShardInfo(SafeEncoder.encode(key)));
}
public Collection<StatefulRedisConnection> getAllConnection(){
return Collections.unmodifiableCollection(resources.values());
}
public String getShardInfo(byte[] key) {
SortedMap<Long, String> tail = nodes.tailMap(algo.hash(key));
if (tail.isEmpty()) {
return nodes.get(nodes.firstKey());
}
return tail.get(tail.firstKey());
}
public void close(){
for(StatefulRedisConnection connection: getAllConnection()){
connection.close();
}
}
private static class SafeEncoder {
static byte[] encode(final String str) {
try {
if (str == null) {
throw new IllegalArgumentException("value sent to redis cannot be null");
}
return str.getBytes("UTF-8");
} catch (UnsupportedEncodingException e) {
throw new RuntimeException(e);
}
}
}
private interface Hashing {
Hashing MURMUR_HASH = new MurmurHash();
long hash(String key);
long hash(byte[] key);
}
private static class MurmurHash implements Hashing {
static long hash64A(byte[] data, int seed) {
return hash64A(ByteBuffer.wrap(data), seed);
}
static long hash64A(ByteBuffer buf, int seed) {
ByteOrder byteOrder = buf.order();
buf.order(ByteOrder.LITTLE_ENDIAN);
long m = 0xc6a4a7935bd1e995L;
int r = 47;
long h = seed ^ (buf.remaining() * m);
long k;
while (buf.remaining() >= 8) {
k = buf.getLong();
k *= m;
k ^= k >>> r;
k *= m;
h ^= k;
h *= m;
}
if (buf.remaining() > 0) {
ByteBuffer finish = ByteBuffer.allocate(8).order(ByteOrder.LITTLE_ENDIAN);
// for big-endian version, do this first:
// finish.position(8-buf.remaining());
finish.put(buf).rewind();
h ^= finish.getLong();
h *= m;
}
h ^= h >>> r;
h *= m;
h ^= h >>> r;
buf.order(byteOrder);
return h;
}
public long hash(byte[] key) {
return hash64A(key, 0x1234ABCD);
}
public long hash(String key) {
return hash(SafeEncoder.encode(key));
}
}
}
@Bean(destroyMethod = "close")
Sharded<StatefulRedisMasterSlaveConnection,String> sharded(RedisClient redisClient) {
Set<HostAndPort> hostAndPorts=new HashSet<>();
hostAndPorts.add(HostAndPort.parse("1xx:26009"));
hostAndPorts.add(HostAndPort.parse("1xx:26009"));
return new Sharded<>(Arrays.asList("te009","test68","test67"),redisClient,"password",hostAndPorts, new Utf8StringCodec());
}
使用方式
//只从slave节点中读取
StatefulRedisMasterSlaveConnection redisConnection = (StatefulRedisMasterSlaveConnection) sharded.getConnectionBy("key");
//使用异步模式获取缓存值
System.out.println(redisConnection.sync().get("key"));