前言
在之前两周主要学了HDFS中的一些模块知识,其中的许多都或多或少有我们借鉴学习的地方,现在将目光转向另外一个块,被誉为MRv2,就是yarn,在Yarn中,解决了MR中JobTracker单点的问题,将此拆分成了ResourceManager和NodeManager这样的结构,在每个节点上,还会有ApplicationMaster来管理应用程序的整个生命周期,的确在Yarn中,多了许多优秀的设计,而今天,我主要分享的就是这个ApplicationMaster相关的一整套服务,他是隶属于ResoureManager的内部服务中的.了解了AM的启动机制,你将会更进一步了解Yarn的任务启动过程.
ApplicationMaster管理涉及类
ApplicationMaster管理涉及到了4大类,ApplicationMasterLauncher,AMLivelinessMonitor,ApplicationMasterService,以及ApplicationMaster自身类.下面介绍一下这些类的用途,在Yarn中,每个类都会有自己明确的功能模块的区分.
1.ApplicationMasterLauncher--姑且叫做AM启动关闭事件处理器,他既是一个服务也是一个处理器,在这个类中,只处理2类事件,launch和cleanup事件.分别对应启动应用和关闭应用的情形.
2.AMLivelinessMonitor--这个类从名字上可以看出他是监控类,监控的对象是AM存活状态的监控类,检测的方法与之前的HDFS一样,都是采用heartbeat的方式,如果有节点过期了,将会触发一次过期事件.
3.ApplicationMasterService--AM请求服务处理类.AMS存在于ResourceManager,中,服务的对象是各个节点上的ApplicationMaster,负责接收各个AM的注册请求,更新心跳包信息等.
4.ApplicationMaster--节点应用管理类,简单的说,ApplicationMaster负责管理整个应用的生命周期.
简答的描述完AM管理的相关类,下面从源码级别分析一下几个流程.
AM启动
要想让AM启动,启动的背景当然是有用户提交了新的Application的时候,之后ApplicationMasterLauncher会生成Launch事件,与对应的nodemanager通信,让其准备启动的新的AM的Container.在这里,就用到了ApplicationMasterLauncher这个类,之前在上文中已经提到,此类就处理2类事件,Launch启动和Cleanup清洗事件,先来看看这个类的基本变量设置
//Application应用事件处理器
public class ApplicationMasterLauncher extends AbstractService implements
EventHandler<AMLauncherEvent> {
private static final Log LOG = LogFactory.getLog(
ApplicationMasterLauncher.class);
private final ThreadPoolExecutor launcherPool;
private LauncherThread launcherHandlingThread;
//事件队列
private final BlockingQueue<Runnable> masterEvents
= new LinkedBlockingQueue<Runnable>();
//资源管理器上下文
protected final RMContext context;
public ApplicationMasterLauncher(RMContext context) {
super(ApplicationMasterLauncher.class.getName());
this.context = context;
//初始化线程池
this.launcherPool = new ThreadPoolExecutor(10, 10, 1,
TimeUnit.HOURS, new LinkedBlockingQueue<Runnable>());
//新建处理线程
this.launcherHandlingThread = new LauncherThread();
}还算比较简单,有一个masterEvents事件队列,还有执行线程以及所需的线程池执行环境。在RM相关的服务中，基本都是继承自AbstractService这个抽象服务类的。ApplicationMasterLauncher中主要处理2类事件,就是下面的展示的
@Override
public synchronized void handle(AMLauncherEvent appEvent) {
AMLauncherEventType event = appEvent.getType();
RMAppAttempt application = appEvent.getAppAttempt();
//处理来自ApplicationMaster获取到的请求，分为启动事件和清洗事件2种
switch (event) {
case LAUNCH:
launch(application);
break;
case CLEANUP:
cleanup(application);
default:
break;
}
}//添加应用启动事件
private void launch(RMAppAttempt application) {
Runnable launcher = createRunnableLauncher(application,
AMLauncherEventType.LAUNCH);
//将启动事件加入事件队列中
masterEvents.add(launcher);
}//执行线程实现
private class LauncherThread extends Thread {
public LauncherThread() {
super("ApplicationMaster Launcher");
}
@Override
public void run() {
while (!this.isInterrupted()) {
Runnable toLaunch;
try {
//执行方法为从事件队列中逐一取出事件
toLaunch = masterEvents.take();
//放入线程池池中进行执行
launcherPool.execute(toLaunch);
} catch (InterruptedException e) {
LOG.warn(this.getClass().getName() + " interrupted. Returning.");
return;
}
}
}
}/**
* The launch of the AM itself.
* Application事件执行器
*/
public class AMLauncher implements Runnable {
private static final Log LOG = LogFactory.getLog(AMLauncher.class);
private ContainerManagementProtocol containerMgrProxy;
private final RMAppAttempt application;
private final Configuration conf;
private final AMLauncherEventType eventType;
private final RMContext rmContext;
private final Container masterContainer;
@SuppressWarnings("unchecked")
public void run() {
//AMLauncher分2中事件分别处理
switch (eventType) {
case LAUNCH:
try {
LOG.info("Launching master" + application.getAppAttemptId());
//调用启动方法
launch();
handler.handle(new RMAppAttemptEvent(application.getAppAttemptId(),
RMAppAttemptEventType.LAUNCHED));
...
break;
case CLEANUP:
try {
LOG.info("Cleaning master " + application.getAppAttemptId());
//调用作业清洗方法
cleanup();
...
break;
default:
LOG.warn("Received unknown event-type " + eventType + ". Ignoring.");
break;
}
}@SuppressWarnings({ "unchecked" })
public boolean run() throws YarnException, IOException {
LOG.info("Starting ApplicationMaster");
Credentials credentials =
UserGroupInformation.getCurrentUser().getCredentials();
DataOutputBuffer dob = new DataOutputBuffer();
credentials.writeTokenStorageToStream(dob);
// Now remove the AM->RM token so that containers cannot access it.
Iterator<Token<?>> iter = credentials.getAllTokens().iterator();
while (iter.hasNext()) {
Token<?> token = iter.next();
if (token.getKind().equals(AMRMTokenIdentifier.KIND_NAME)) {
iter.remove();
}
}
allTokens = ByteBuffer.wrap(dob.getData(), 0, dob.getLength());
//与ResourceManager通信，周期性发送心跳信息，包含了应用的最新信息
AMRMClientAsync.CallbackHandler allocListener = new RMCallbackHandler();
amRMClient = AMRMClientAsync.createAMRMClientAsync(1000, allocListener);
amRMClient.init(conf);
amRMClient.start();
.....
// Register self with ResourceManager
// This will start heartbeating to the RM
//启动之后进行AM的注册
appMasterHostname = NetUtils.getHostname();
RegisterApplicationMasterResponse response = amRMClient
.registerApplicationMaster(appMasterHostname, appMasterRpcPort,
appMasterTrackingUrl);
// Dump out information about cluster capability as seen by the
// resource manager
int maxMem = response.getMaximumResourceCapability().getMemory();
LOG.info("Max mem capabililty of resources in this cluster " + maxMem);
// A resource ask cannot exceed the max.
if (containerMemory > maxMem) {
LOG.info("Container memory specified above max threshold of cluster."
+ " Using max value." + ", specified=" + containerMemory + ", max="
+ maxMem);
containerMemory = maxMem;
}AMLiveLinessMonitor监控
在这里把重心从ApplicationMaster转移到AMLivelinessMonitor上，首先这是一个激活状态的监控线程，此类线程都有一个共同的父类
//应用存活状态监控线程
public class AMLivelinessMonitor extends AbstractLivelinessMonitor<ApplicationAttemptId> {//进程存活状态监控类
public abstract class AbstractLivelinessMonitor<O> extends AbstractService {
private static final Log LOG = LogFactory.getLog(AbstractLivelinessMonitor.class);
//thread which runs periodically to see the last time since a heartbeat is
//received.
//检查线程
private Thread checkerThread;
private volatile boolean stopped;
//默认超时时间5分钟
public static final int DEFAULT_EXPIRE = 5*60*1000;//5 mins
//超时时间
private int expireInterval = DEFAULT_EXPIRE;
//监控间隔检测时间，为超时时间的1/3
private int monitorInterval = expireInterval/3;
private final Clock clock;
//保存了心跳检验的结果记录
private Map<O, Long> running = new HashMap<O, Long>();//新的节点注册心跳监控
public synchronized void register(O ob) {
running.put(ob, clock.getTime());
}
//节点移除心跳监控
public synchronized void unregister(O ob) {
running.remove(ob);
}//更新心跳监控检测最新时间
public synchronized void receivedPing(O ob) {
//only put for the registered objects
if (running.containsKey(ob)) {
running.put(ob, clock.getTime());
}
}AM与AMS
在ApplicationMaster运行之后，会周期性的向ApplicationMasterService发送心跳信息，心跳信息包含有许多资源描述信息。
//ApplicationMaster心跳信息更新
@Override
public AllocateResponse allocate(AllocateRequest request)
throws YarnException, IOException {
ApplicationAttemptId appAttemptId = authorizeRequest();
//进行心跳信息时间的更新
this.amLivelinessMonitor.receivedPing(appAttemptId);
....
//ApplicationMaster在ApplicationMasterService上服务上进行应用注册
@Override
public RegisterApplicationMasterResponse registerApplicationMaster(
RegisterApplicationMasterRequest request) throws YarnException,
IOException {
ApplicationAttemptId applicationAttemptId = authorizeRequest();
ApplicationId appID = applicationAttemptId.getApplicationId();
.....
//在存活监控线程上进行心跳记录，更新检测时间，key为应用ID
this.amLivelinessMonitor.receivedPing(applicationAttemptId);
RMApp app = this.rmContext.getRMApps().get(appID);
// Setting the response id to 0 to identify if the
// application master is register for the respective attemptid
lastResponse.setResponseId(0);
responseMap.put(applicationAttemptId, lastResponse);
LOG.info("AM registration " + applicationAttemptId);
this.rmContext//进程存活状态监控类
public abstract class AbstractLivelinessMonitor<O> extends AbstractService {
...
//thread which runs periodically to see the last time since a heartbeat is
//received.
//检查线程
private Thread checkerThread;
....
//默认超时时间5分钟
public static final int DEFAULT_EXPIRE = 5*60*1000;//5 mins
//超时时间
private int expireInterval = DEFAULT_EXPIRE;
//监控间隔检测时间，为超时时间的1/3
private int monitorInterval = expireInterval/3;
....
//保存了心跳检验的结果记录
private Map<O, Long> running = new HashMap<O, Long>();
...
private class PingChecker implements Runnable {
@Override
public void run() {
while (!stopped && !Thread.currentThread().isInterrupted()) {
synchronized (AbstractLivelinessMonitor.this) {
Iterator<Map.Entry<O, Long>> iterator =
running.entrySet().iterator();
//avoid calculating current time everytime in loop
long currentTime = clock.getTime();
while (iterator.hasNext()) {
Map.Entry<O, Long> entry = iterator.next();
//进行超时检测
if (currentTime > entry.getValue() + expireInterval) {
iterator.remove();
//调用超时处理方法，将处理事件交由调度器处理
expire(entry.getKey());
LOG.info("Expired:" + entry.getKey().toString() +
" Timed out after " + expireInterval/1000 + " secs");
}
}
}
//应用存活状态监控线程
public class AMLivelinessMonitor extends AbstractLivelinessMonitor<ApplicationAttemptId> {
//中央调度处理器
private EventHandler dispatcher;
...
@Override
protected void expire(ApplicationAttemptId id) {
//一旦应用过期，处理器处理过期事件处理
dispatcher.handle(
new RMAppAttemptEvent(id, RMAppAttemptEventType.EXPIRE));
}
}
全部代码的分析请点击链接https://github.com/linyiqun/hadoop-yarn,后续将会继续更新YARN其他方面的代码分析。
参考文献
《Hadoop技术内部–HDFS结构设计与实现原理》.蔡斌等