YARN报错
2017-08-25 03:51:58,815 WARN org.apache.hadoop.ipc.Server: Large response size 4739374 for call org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplications from 10.135.8.101:38352 Call#33361 Retry#0
2017-08-25 03:53:39,255 WARN org.apache.hadoop.ipc.Server: Large response size 4739374 for call org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplications from 10.135.8.101:38456 Call#33364 Retry#0 2017-08-25 03:55:19,700 WARN org.apache.hadoop.ipc.Server: Large response size 4739374 for call org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplications from 10.135.8.101:38556 Call#33367 Retry#0 2017-08-25 03:57:00,262 WARN org.apache.hadoop.ipc.Server: Large response size 4739374 for call org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplications from 10.135.8.101:38674 Call#33370 Retry#0 2017-08-25 03:58:40,687 WARN org.apache.hadoop.ipc.Server: Large response size 4739374 for call org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplications from 10.135.8.101:38804 Call#33373 Retry#0
解决办法:
1、在hdfs-site中添加如下参数
<property> <name>ipc.server.max.response.size</name> <value>5242880</value> </property>
2、可能造成OOM问题
增大-xmx参数的大小
其他问题
正常来说这里的IPC时间返回大概是10s/1min这个级别,如果返回的太频繁就可能会出现RM OOM的问题。
这个问题需要深入源码去分析,待有结论再更新上来。
链接
2、https://issues.apache.org/jira/browse/HADOOP-14858
3、https://mapr.com/community/s/question/0D50L00006BIt35SAD/why-yarn-crashes-