个人觉得akka提供的cluster工具中,sharding是最吸引人的。当我们需要把actor分布在不同的节点上时,Cluster sharding非常有用。我们可以使用actor的逻辑标识符与actor进行通信,而不用关心其物理位置。简单来说就是把actor的actorPath或actorRef进一步抽象,用一个字符串表示。
sharding可以表示为DDD领域中的聚合根,此时actor叫做“实体”。这些actor一般都具有持久化的状态。每个实体actor只运行在某个节点,我们可以在不知道actor位置的情况下,给对应的actor发送消息。当然了,这需要通过ShardRegion这个actor来发送,它知道如何路由消息。
介绍到这里,你是不是也有自己写一个sharding的冲动呢?不过我劝你不要这样做,虽然基于akka实现这个功能的核心功能很简单,但需要考虑很多场景,比如实体漂移。但还是要大胆想一想的,怎么屏蔽actor的位置呢?当然是创建一个proxy喽,这是akka的很多组件中经常出现的一种技术。proxy保存各个实体的具体位置(ActorRef或者ActorPath),然后通过proxy发送消息。其实sharding比较难的是实体要尽量分散,还要支持重新均衡,而且还要考虑实体actor漂移之后状态重建的功能。
在进一步分析源码之前,我有必要啰嗦几句akka中分片的相关概念。
首先我们要区分分片和分区的区别。其实我个人理解,分区是对一组数据的划分,即符合某类特征的数据进行聚合作为一个分区;而分片则是将数据分散到节点上的机制,即分片是尽可能的将数据“均匀”的分散到不同节点来达到均衡的目的,当然了还有其他好处。
akka分片是针对actor的,不是传统的对数据进行分片,可以理解成对计算的分片。每个actor都有一个实体ID,该ID在集群中唯一。akka需要提供从消息中提取实体ID的方法,和用于接收消息并计算actor所属分片ID的函数。
集群中分布的分片所在的区域成为分片区域,这些区域作为分片的proxy,参与分片的每个节点将每种类型的分片actor承载一个单独的分片区域。每个区域可以托管多个分片,分片区域内的所有实体由相同类型的actor表示。每个分片可以托管多个实体,实体根据ID的结果,分布在不同的分片上。分片协调器用于管理分片所在的位置,它是作为一个集群单例实现的。分片区域可以与分片协调器进行通信以定位分片,然后这个位置信息会被缓存。
akka的分片还引入了钝化的机制。很显然如果在系统内长时间保持所有的actor是不合理的,而持久性的存在则可以允许我们对actor进行加载和卸载。如果在指定的时间内,actor没有收到消息,则我们可以钝化actor。简单来说就是给实体actor发送特定的消息关闭自己。如果某个消息ID对应的actor不存在,我们还可以重新加载这个actor。
好了,下面开始分析源码,上官网demo。
class Counter extends PersistentActor { import ShardRegion.Passivate context.setReceiveTimeout(120.seconds) // self.path.name is the entity identifier (utf-8 URL-encoded) override def persistenceId: String = "Counter-" + self.path.name var count = 0 def updateState(event: CounterChanged): Unit = count += event.delta override def receiveRecover: Receive = { case evt: CounterChanged ⇒ updateState(evt) } override def receiveCommand: Receive = { case Increment ⇒ persist(CounterChanged(+1))(updateState) case Decrement ⇒ persist(CounterChanged(-1))(updateState) case Get(_) ⇒ sender() ! count case ReceiveTimeout ⇒ context.parent ! Passivate(stopMessage = Stop) case Stop ⇒ context.stop(self) } }
首先Counter是一个需要分散在各个节点的实体actor。很显然这是个PersistentActor。那如何创建一个对Counter的proxyActor呢?在akka中这叫做ShardRegion。
val counterRegion: ActorRef = ClusterSharding(system).start( typeName = "Counter", entityProps = Props[Counter], settings = ClusterShardingSettings(system), extractEntityId = extractEntityId, extractShardId = extractShardId)
上面代码启动了一个ShardRegion,请注意start方法的最后两个参数。它分别定义了实体ID提取器和分片ID提取器。
val extractEntityId: ShardRegion.ExtractEntityId = { case EntityEnvelope(id, payload) ⇒ (id.toString, payload) case msg @ Get(id) ⇒ (id.toString, msg) } val numberOfShards = 100 val extractShardId: ShardRegion.ExtractShardId = { case EntityEnvelope(id, _) ⇒ (id % numberOfShards).toString case Get(id) ⇒ (id % numberOfShards).toString case ShardRegion.StartEntity(id) ⇒ // StartEntity is used by remembering entities feature (id.toLong % numberOfShards).toString }
ShardRegion.ExtractEntityId、ShardRegion.ExtractShardId又是什么呢?
/** * Interface of the partial function used by the [[ShardRegion]] to * extract the entity id and the message to send to the entity from an * incoming message. The implementation is application specific. * If the partial function does not match the message will be * `unhandled`, i.e. posted as `Unhandled` messages on the event stream. * Note that the extracted message does not have to be the same as the incoming * message to support wrapping in message envelope that is unwrapped before * sending to the entity actor. */ type ExtractEntityId = PartialFunction[Msg, (EntityId, Msg)]
ExtractEntityId是一个片函数,它用来从消息中提取实体ID和消息。如果这个片函数无法处理消息,那么就会把这个未处理的消息发送给event stream。注意,输入的消息和输出的消息不一定是同类型的。
/** * Interface of the function used by the [[ShardRegion]] to * extract the shard id from an incoming message. * Only messages that passed the [[ExtractEntityId]] will be used * as input to this function. */ type ExtractShardId = Msg ⇒ ShardId
ExtractShardId是用来从消息中提取分片ID的,只有通过ExtractEntityId的消息才会发送给这个函数。其实在发送消息时,首先从消息中获取实体ID,然后通过实体ID计算分片ID的。
但有一定需要注意,那就是对于相同的实体ID计算出来的分片ID一定要相同,其实就是要保持一致,否则就会同时在多个地方启动相同的实体actor了。
其实一个好的分片算法非常重要。一致性HASH就是同时对消息ID和节点ID进行HASH,把消息路由给最邻近的节点。
下面开始分析start。
/** * Scala API: Register a named entity type by defining the [[akka.actor.Props]] of the entity actor * and functions to extract entity and shard identifier from messages. The [[ShardRegion]] actor * for this type can later be retrieved with the [[shardRegion]] method. * * This method will start a [[ShardRegion]] in proxy mode in case if there is no match between the roles of * the current cluster node and the role specified in [[ClusterShardingSettings]] passed to this method. * * Some settings can be configured as described in the `akka.cluster.sharding` section * of the `reference.conf`. * * @return the actor ref of the [[ShardRegion]] that is to be responsible for the shard */ def start( typeName: String, entityProps: Props, settings: ClusterShardingSettings, extractEntityId: ShardRegion.ExtractEntityId, extractShardId: ShardRegion.ExtractShardId, allocationStrategy: ShardAllocationStrategy, handOffStopMessage: Any): ActorRef = { internalStart(typeName, _ ⇒ entityProps, settings, extractEntityId, extractShardId, allocationStrategy, handOffStopMessage) }
start注册了一个命名的实体类型,也就是实体actor的Props,当然还包括从消息中提取实体ID、分片ID的函数。ShardRegion这个actor会被创建,单独负责这个类型(Props定义的类型),当然了还可以通过shardRegion方法查询获取。如果当前节点角色不属于我们定义的节点,则会创建一个ShardRegion的一个代理,即本地不会有分片actor创建,只是简单的把消息转发给其他节点上的ShardRegion。
@InternalApi private[akka] def internalStart( typeName: String, entityProps: String ⇒ Props, settings: ClusterShardingSettings, extractEntityId: ShardRegion.ExtractEntityId, extractShardId: ShardRegion.ExtractShardId, allocationStrategy: ShardAllocationStrategy, handOffStopMessage: Any): ActorRef = { if (settings.shouldHostShard(cluster)) { implicit val timeout = system.settings.CreationTimeout val startMsg = Start(typeName, entityProps, settings, extractEntityId, extractShardId, allocationStrategy, handOffStopMessage) val Started(shardRegion) = Await.result(guardian ? startMsg, timeout.duration) regions.put(typeName, shardRegion) shardRegion } else { log.debug("Starting Shard Region Proxy [{}] (no actors will be hosted on this node)...", typeName) startProxy( typeName, settings.role, dataCenter = None, // startProxy method must be used directly to start a proxy for another DC extractEntityId, extractShardId) } }
通过源码跟踪,我们找到了internalStart,这个函数只有一个if-else代码块,很显然我们刚开始需要创建ShardRegion,应该会走if代码块,也就是用ask模式给guardian发送Start消息,返回一个ActorRef,其实应该就是ShardRegion的actor的实例。并把这个ActorRef添加到了regions变量中。
private lazy val guardian: ActorRef = { val guardianName: String = system.settings.config.getString("akka.cluster.sharding.guardian-name") val dispatcher = system.settings.config .getString("akka.cluster.sharding.use-dispatcher") match { case "" ⇒ Dispatchers.DefaultDispatcherId case id ⇒ id } system.systemActorOf(Props[ClusterShardingGuardian].withDispatcher(dispatcher), guardianName) }
guardian是一个ClusterShardingGuardian实例,它如何处理Start消息呢?
这个actor只处理两个类型的消息:Start和StartProxy。其实吧,这个就是对ShardRegion的一个监督者,这种用法之前分析过,也特意提出来了,读者自行理解吧。不过这里又叫Guardian,而不叫supervisor了,命名不统一啊,估计是多个开发者导致的。
val rep = replicator(settings) val encName = URLEncoder.encode(typeName, ByteString.UTF_8) val cName = coordinatorSingletonManagerName(encName) val cPath = coordinatorPath(encName) val shardRegion = context.child(encName).getOrElse { if (context.child(cName).isEmpty) { val coordinatorProps = if (settings.stateStoreMode == ClusterShardingSettings.StateStoreModePersistence) ShardCoordinator.props(typeName, settings, allocationStrategy) else ShardCoordinator.props(typeName, settings, allocationStrategy, rep, majorityMinCap) val singletonProps = BackoffSupervisor.props( childProps = coordinatorProps, childName = "coordinator", minBackoff = coordinatorFailureBackoff, maxBackoff = coordinatorFailureBackoff * 5, randomFactor = 0.2) .withDeploy(Deploy.local) val singletonSettings = settings.coordinatorSingletonSettings .withSingletonName("singleton") .withRole(role) context.actorOf( ClusterSingletonManager.props(singletonProps, terminationMessage = PoisonPill, singletonSettings) .withDispatcher(context.props.dispatcher), name = cName) } context.actorOf( ShardRegion.props( typeName = typeName, entityProps = entityProps, settings = settings, coordinatorPath = cPath, extractEntityId = extractEntityId, extractShardId = extractShardId, handOffStopMessage = handOffStopMessage, replicator = rep, majorityMinCap ) .withDispatcher(context.props.dispatcher), name = encName ) } sender() ! Started(shardRegion)
对Start消息的处理非常关键,所以这里把整个源码都贴出来了。很显然这里创建了三种actor:replicator、coordinator、shardRegion。replicator先略过,coordinator只需要记住它是通过ClusterSingletonManager创建的集群单例actor就行了。
StartProxy消息的处理不再分析,跟Start不同的是,它不会创建replicator、coordinator这些实例,只是简单的创建ShardRegion的代理actor。
/** * INTERNAL API * * This actor creates children entity actors on demand for the shards that it is told to be * responsible for. It delegates messages targeted to other shards to the responsible * `ShardRegion` actor on other nodes. * * @see [[ClusterSharding$ ClusterSharding extension]] */ private[akka] class ShardRegion( typeName: String, entityProps: Option[String ⇒ Props], dataCenter: Option[DataCenter], settings: ClusterShardingSettings, coordinatorPath: String, extractEntityId: ShardRegion.ExtractEntityId, extractShardId: ShardRegion.ExtractShardId, handOffStopMessage: Any, replicator: ActorRef, majorityMinCap: Int) extends Actor with ActorLogging
ShardRegion根据需要为分片创建对应的实体actor,并可以把消息路由给其他分片内的实体。既然消息都是发送给ShardRegion这个actor的,那么这个actor的receive定义就比较重要了。
def receive: Receive = { case Terminated(ref) ⇒ receiveTerminated(ref) case ShardInitialized(shardId) ⇒ initializeShard(shardId, sender()) case evt: ClusterDomainEvent ⇒ receiveClusterEvent(evt) case state: CurrentClusterState ⇒ receiveClusterState(state) case msg: CoordinatorMessage ⇒ receiveCoordinatorMessage(msg) case cmd: ShardRegionCommand ⇒ receiveCommand(cmd) case query: ShardRegionQuery ⇒ receiveQuery(query) case msg: RestartShard ⇒ deliverMessage(msg, sender()) case msg: StartEntity ⇒ deliverStartEntity(msg, sender()) case msg if extractEntityId.isDefinedAt(msg) ⇒ deliverMessage(msg, sender()) case unknownMsg ⇒ log.warning("Message does not have an extractor defined in shard [{}] so it was ignored: {}", typeName, unknownMsg) }
很显然,收到本分片能够处理的消息时,会去调用deliverMessage这个方法。
def deliverMessage(msg: Any, snd: ActorRef): Unit = msg match { case RestartShard(shardId) ⇒ regionByShard.get(shardId) match { case Some(ref) ⇒ if (ref == self) getShard(shardId) case None ⇒ if (!shardBuffers.contains(shardId)) { log.debug("Request shard [{}] home. Coordinator [{}]", shardId, coordinator) coordinator.foreach(_ ! GetShardHome(shardId)) } val buf = shardBuffers.getOrEmpty(shardId) log.debug("Buffer message for shard [{}]. Total [{}] buffered messages.", shardId, buf.size + 1) shardBuffers.append(shardId, msg, snd) } case _ ⇒ val shardId = extractShardId(msg) regionByShard.get(shardId) match { case Some(ref) if ref == self ⇒ getShard(shardId) match { case Some(shard) ⇒ if (shardBuffers.contains(shardId)) { // Since now messages to a shard is buffered then those messages must be in right order bufferMessage(shardId, msg, snd) deliverBufferedMessages(shardId, shard) } else shard.tell(msg, snd) case None ⇒ bufferMessage(shardId, msg, snd) } case Some(ref) ⇒ log.debug("Forwarding request for shard [{}] to [{}]", shardId, ref) ref.tell(msg, snd) case None if shardId == null || shardId == "" ⇒ log.warning("Shard must not be empty, dropping message [{}]", msg.getClass.getName) context.system.deadLetters ! msg case None ⇒ if (!shardBuffers.contains(shardId)) { log.debug("Request shard [{}] home. Coordinator [{}]", shardId, coordinator) coordinator.foreach(_ ! GetShardHome(shardId)) } bufferMessage(shardId, msg, snd) } }
很显然,普通消息会走最后一个case代码块。它首先根据extractShardId解析出消息所属分片的ID。然后根据shardId从本地的regionByShard中查找对应的实体,如果找到且属于本地,就把消息转发给它。如果是远程的或不属于当前ShardRegion则会把消息转发给对应的Shard。如果当前ShardRegion没有找到对应的Shard,则会给coordinator发送GetShardHome消息来获取对应的Shard的ActorRef信息,不过这种情况我们先不分析。
regionByShard是用来做什么的呢?其实这是一个索引,它保存分片ID到分区区域的映射,因为分片区域负责管理分片。分片区域(ShardRegion)可以在本地也可以在远程或其他JVM。我们假设法消息给本地的分片区域,很显然刚开始时regionByShard是空的,而且shardBuffers也是空的,所以会给协调器发送GetShardHome消息,并调用bufferMessage消息。bufferMessage简单来说就是把shardId保存到shardBuffers并把消息做一个临时缓存。在Region创建之后再把消息发送过去。
ShardCoordinator分为PersistentShardCoordinator、DDataShardCoordinator,来看看他们是如何处理GetShardHome消息的。
case GetShardHome(shard) ⇒ if (!handleGetShardHome(shard)) { // location not know, yet val activeRegions = state.regions -- gracefulShutdownInProgress if (activeRegions.nonEmpty) { val getShardHomeSender = sender() val regionFuture = allocationStrategy.allocateShard(getShardHomeSender, shard, activeRegions) regionFuture.value match { case Some(Success(region)) ⇒ continueGetShardHome(shard, region, getShardHomeSender) case _ ⇒ // continue when future is completed regionFuture.map { region ⇒ AllocateShardResult(shard, Some(region), getShardHomeSender) }.recover { case _ ⇒ AllocateShardResult(shard, None, getShardHomeSender) }.pipeTo(self) } } }
上面的代码表明,在ShardRegion给ShardCoordinator发送GetShardHome之前,ShardCoordinator就已经知道了Region的信息,那么什么时候知道的呢?我们似乎漏掉了什么。我们发现ShardRegion在收到集群的消息时(ClusterDomainEvent、CurrentClusterState)最终都会触发changeMembers方法,我们知道集群消息是在actor启动的时候就会收到的。
def changeMembers(newMembers: immutable.SortedSet[Member]): Unit = { val before = membersByAge.headOption val after = newMembers.headOption membersByAge = newMembers if (before != after) { if (log.isDebugEnabled) log.debug("Coordinator moved from [{}] to [{}]", before.map(_.address).getOrElse(""), after.map(_.address).getOrElse("")) coordinator = None register() } }
changeMembers中会调用register方法。
def register(): Unit = { coordinatorSelection.foreach(_ ! registrationMessage) if (shardBuffers.nonEmpty && retryCount >= 5) coordinatorSelection match { case Some(actorSelection) ⇒ val coordinatorMessage = if (cluster.state.unreachable(membersByAge.head)) s"Coordinator [${membersByAge.head}] is unreachable." else s"Coordinator [${membersByAge.head}] is reachable." log.warning( "Trying to register to coordinator at [{}], but no acknowledgement. Total [{}] buffered messages. [{}]", actorSelection, shardBuffers.totalSize, coordinatorMessage ) case None ⇒ log.warning( "No coordinator found to register. Probably, no seed-nodes configured and manual cluster join not performed? Total [{}] buffered messages.", shardBuffers.totalSize) } }
register其实就是给coordinatorSelection发送registrationMessage消息。
def registrationMessage: Any = if (entityProps.isDefined) Register(self) else RegisterProxy(self)
registrationMessage只是区分了Register还是RegisterProxy,其实就是区分是注册Region还是RegionProxy。
case Register(region) ⇒ if (isMember(region)) { log.debug("ShardRegion registered: [{}]", region) aliveRegions += region if (state.regions.contains(region)) { region ! RegisterAck(self) allocateShardHomesForRememberEntities() } else { gracefulShutdownInProgress -= region update(ShardRegionRegistered(region)) { evt ⇒ state = state.updated(evt) context.watch(region) region ! RegisterAck(self) allocateShardHomesForRememberEntities() } } } else { log.debug("ShardRegion {} was not registered since the coordinator currently does not know about a node of that region", region) }
上面是ShardCoordinator收到Register时的处理逻辑,其实就是保存了region的相关信息,并发送了RegisterAck消息。但请注意allocateShardHomesForRememberEntities,这个机制后面会再深入分析,因为它影响了分片的性能。
case RegisterAck(coord) ⇒ context.watch(coord) coordinator = Some(coord) requestShardBufferHomes()
ShardRegion收到RegisterAck消息如何处理的呢,其实就是保存了coord的ActorRef,并把缓存的消息发出去了。
我们回到GetShardHome处理的代码块,其实它最终调用了allocationStrategy.allocateShard(getShardHomeSender, shard, activeRegions)创建了Shard。
/** * Invoked when the location of a new shard is to be decided. * @param requester actor reference to the [[ShardRegion]] that requested the location of the * shard, can be returned if preference should be given to the node where the shard was first accessed * @param shardId the id of the shard to allocate * @param currentShardAllocations all actor refs to `ShardRegion` and their current allocated shards, * in the order they were allocated * @return a `Future` of the actor ref of the [[ShardRegion]] that is to be responsible for the shard, must be one of * the references included in the `currentShardAllocations` parameter */ def allocateShard(requester: ActorRef, shardId: ShardId, currentShardAllocations: Map[ActorRef, immutable.IndexedSeq[ShardId]]): Future[ActorRef]
allocateShard是ShardAllocationStrategy这个trait的一个方法。通过上下文,我们知道ShardAllocationStrategy是LeastShardAllocationStrategy的一个实例。
/** * The default implementation of [[ShardCoordinator.LeastShardAllocationStrategy]] * allocates new shards to the `ShardRegion` with least number of previously allocated shards. * It picks shards for rebalancing handoff from the `ShardRegion` with most number of previously allocated shards. * They will then be allocated to the `ShardRegion` with least number of previously allocated shards, * i.e. new members in the cluster. There is a configurable threshold of how large the difference * must be to begin the rebalancing. The number of ongoing rebalancing processes can be limited. */ @SerialVersionUID(1L) class LeastShardAllocationStrategy(rebalanceThreshold: Int, maxSimultaneousRebalance: Int) extends ShardAllocationStrategy with Serializable
override def allocateShard(requester: ActorRef, shardId: ShardId, currentShardAllocations: Map[ActorRef, immutable.IndexedSeq[ShardId]]): Future[ActorRef] = { val (regionWithLeastShards, _) = currentShardAllocations.minBy { case (_, v) ⇒ v.size } Future.successful(regionWithLeastShards) }
LeastShardAllocationStrategy是如何实现allocateShard的呢?其实就是从currentShardAllocations选择保持shard最少的actorRef,那么currentShardAllocations传入的时候值是什么呢?其实就是当前活跃的ShardRegion。很显然我们目前只有一个ShardRegion,根据源码来分析,GetShardHome消息的代码块中会调用continueGetShardHome
def continueGetShardHome(shard: ShardId, region: ActorRef, getShardHomeSender: ActorRef): Unit = if (rebalanceInProgress.contains(shard)) { deferGetShardHomeRequest(shard, getShardHomeSender) } else { state.shards.get(shard) match { case Some(ref) ⇒ getShardHomeSender ! ShardHome(shard, ref) case None ⇒ if (state.regions.contains(region) && !gracefulShutdownInProgress.contains(region)) { update(ShardHomeAllocated(shard, region)) { evt ⇒ state = state.updated(evt) log.debug("Shard [{}] allocated at [{}]", evt.shard, evt.region) sendHostShardMsg(evt.shard, evt.region) getShardHomeSender ! ShardHome(evt.shard, evt.region) } } else log.debug( "Allocated region {} for shard [{}] is not (any longer) one of the registered regions: {}", region, shard, state) } }
continueGetShardHome其实就是从state.shards中获取region的实例,如果找到则返回,如果没有找到则创建ShardHomeAllocated消息,然后调用sendHostShardMsg
def sendHostShardMsg(shard: ShardId, region: ActorRef): Unit = { region ! HostShard(shard) val cancel = context.system.scheduler.scheduleOnce(shardStartTimeout, self, ResendShardHost(shard, region)) unAckedHostShards = unAckedHostShards.updated(shard, cancel) }
sendHostShardMsg就是给ShardRegion发送HostShard消息。后续功能不再分析。
那么读者会问了,为啥ShardRegion还需要通过shardId去问协调器ShardRegion的位置呢?你猜啊。哈哈,因为发给本地ShardRegion的消息,并不一定属于它啊,有可能是其他节点的ShardRegion呢?为啥对于同一个shardId可能存在多个节点上呢?你再猜,哈哈。想想本来就该这样啊,我们之定义了实体ID、分片ID,并没有定义分片区域ID啊,也就是说,分片可以属于任意一个分片区域,而分片区域可以跨节点分布。所以这也就是默认的LeastShardAllocationStrategy中allocateShard的意义:从所有节点上找出管理分片最少的分片区域。也就是说,一个消息发送给哪个分片、再有该分片发送给哪个实体是确定的,但是这个分片在哪个节点就不固定了。这也就是所谓的分片漂移,某个分片到达指定阈值,可能会迁移到其他的分片区域内,当迁移时,本地分片内的实体actor都会被stop掉,等分片迁移成功后,收到某个实体第一个消息时重新创建实体actor。
至此,akka的分片机制分析完毕,当然还有很多细节没有分析。但简单来说就是,一个消息先发送给本地的ShardRegion实例,本地实例会计算消息的sharId,通过这个ID找出对应的Shard(因为Shard可能不在本节点或漂移到了其他节点),当然这是通过协调器找到的。找到shardId对应的Shard之后,就会把消息发送给Shard。
不过我们好像忘了分析Shard了,好尴尬,哈哈。那么就看看。
def receiveCommand: Receive = { case Terminated(ref) ⇒ receiveTerminated(ref) case msg: CoordinatorMessage ⇒ receiveCoordinatorMessage(msg) case msg: ShardCommand ⇒ receiveShardCommand(msg) case msg: ShardRegion.StartEntity ⇒ receiveStartEntity(msg) case msg: ShardRegion.StartEntityAck ⇒ receiveStartEntityAck(msg) case msg: ShardRegionCommand ⇒ receiveShardRegionCommand(msg) case msg: ShardQuery ⇒ receiveShardQuery(msg) case msg if extractEntityId.isDefinedAt(msg) ⇒ deliverMessage(msg, sender()) }
很显然,它收到消息之后,还是调用deliverMessage。
def deliverMessage(msg: Any, snd: ActorRef): Unit = { val (id, payload) = extractEntityId(msg) if (id == null || id == "") { log.warning("Id must not be empty, dropping message [{}]", msg.getClass.getName) context.system.deadLetters ! msg } else if (payload.isInstanceOf[ShardRegion.StartEntity]) { // in case it was wrapped, used in Typed receiveStartEntity(payload.asInstanceOf[ShardRegion.StartEntity]) } else { messageBuffers.contains(id) match { case false ⇒ deliverTo(id, msg, payload, snd) case true if messageBuffers.totalSize >= bufferSize ⇒ log.debug("Buffer is full, dropping message for entity [{}]", id) context.system.deadLetters ! msg case true ⇒ log.debug("Message for entity [{}] buffered", id) messageBuffers.append(id, msg, snd) } } }
deliverMessage源码看似很多,但重点就在最后一个else代码块。messageBuffers刚开始是没有这个实体ID的,所以会调用deliverTo。
def deliverTo(id: EntityId, msg: Any, payload: Msg, snd: ActorRef): Unit = { val name = URLEncoder.encode(id, "utf-8") context.child(name) match { case Some(actor) ⇒ actor.tell(payload, snd) case None ⇒ getEntity(id).tell(payload, snd) } }
deliverTo会在当前的分片中查找对应的实体actor,如果找到就把消息发送给它,如果没找到就通过getEntity创建之后在发送给他。
def getEntity(id: EntityId): ActorRef = { val name = URLEncoder.encode(id, "utf-8") context.child(name).getOrElse { log.debug("Starting entity [{}] in shard [{}]", id, shardId) val a = context.watch(context.actorOf(entityProps(id), name)) idByRef = idByRef.updated(a, id) refById = refById.updated(id, a) state = state.copy(state.entities + id) a } }
getEntity比较简单,就是通过name查找子actor,如果没找到就用entityProps创建对应的actor,然后就是更新相关的状态。
本文介绍了分片的基本机制,但分片的功能远比这个要复杂一点,我们后面会再进一步的分析和研读源码。