类似错误信息:
2020-08-21 15:30:28,713 INFO mapreduce.Job: Task Id : attempt_1596166140899_0045_m_000004_2, Status : FAILED
[2020-08-21 15:30:29.974]Container [pid=17160,containerID=container_1596166140899_0045_01_000023] is running 458619392B beyond the 'VIRTUAL' memory limit. Current usage: 74.7 MB of 1 GB physical memory used; 2.5 GB of 2.1 GB virtual memory used. Killing container.
[2020-08-21 15:13:45.757]Container killed on request. Exit code is 143
[2020-08-21 15:13:45.846]Container exited with a non-zero exit code 143.
2020-08-21 15:13:45,298 INFO mapreduce.Job: Task Id : attempt_1596166140899_0044_m_000010_1, Status : FAILED
[2020-08-21 15:13:47.447]Container [pid=2415,containerID=container_1596166140899_0044_01_000023] is running 450304512B beyond the 'VIRTUAL' memory limit. Current usage: 60.8 MB of 1 GB physical memory used; 2.5 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1596166140899_0044_01_000023 :
|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
|- 2415 2414 2415 2415 (bash) 0 0 115900416 307
参考:
https://stackoverflow.com/questions/21005643/container-is-running-beyond-memory-limits
修改yarn-site.xml配置增加:
<property>
<name>yarn.nodemanager.vmem-check-enabled</name>
<value>false</value>
</property>