union all 12个报表，SQL为什么会出现报错

union all 12个报表，SQL为什么会出现这个报错：

Error while compiling statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Application application_1685007310357_314615 failed 2 times due to AM Container for appattempt_1685007310357_314615_000002 exited with  exitCode: -104Failing this attempt.Diagnostics: [2023-06-06 15:18:15.156]Container [pid=188988,containerID=container_e127_1685007310357_314615_02_000001] is running 27553792B beyond the 'PHYSICAL' memory limit. Current usage: 2.0 GB of 2 GB physical memory used; 4.0 GB of 4.2 GB virtual memory used. Killing container.Dump of the process-tree for container_e127_1685007310357_314615_02_000001 :        |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE        |- 188988 188986 188988 188988 (bash) 0 0 118185984 367 /bin/bash -c /usr/java/jdk1.8.0_232-cloudera_bak/bin/java  -Xmx1638m -Djava.io.tmpdir=/data4/yarn/nm/usercache/hive/appcache/application_1685007310357_314615/container_e127_1685007310357_314615_02_000001/tmp -server -Djava.net.preferIPv4Stack=true -XX:+PrintGCDetails -verbose:gc -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=/data8/yarn/container-logs/application_1685007310357_314615/container_e127_1685007310357_314615_02_000001 -Dtez.root.logger=INFO,CLA -Dsun.nio.ch.bugLevel='' org.apache.tez.dag.app.DAGAppMaster --session 1>/data8/yarn/container-logs/application_1685007310357_314615/container_e127_1685007310357_314615_02_000001/stdout 2>/data8/yarn/container-logs/application_1685007310357_314615/container_e127_1685007310357_314615_02_000001/stderr          |- 188996 188988 188988 188988 (java) 17272 688 4166348800 530648 /usr/java/jdk1.8.0_232-cloudera_bak/bin/java -Xmx1638m -Djava.io.tmpdir=/data4/yarn/nm/usercache/hive/appcache/application_1685007310357_314615/container_e127_1685007310357_314615_02_000001/tmp -server -Djava.net.preferIPv4Stack=true -XX:+PrintGCDetails -verbose:gc -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=/data8/yarn/container-logs/application_1685007310357_314615/container_e127_1685007310357_314615_02_000001 -Dtez.root.logger=INFO,CLA -Dsun.nio.ch.bugLevel= org.apache.tez.dag.app.DAGAppMaster --session [2023-06-06 15:18:15.173]Container killed on request. Exit code is 143[2023-06-06 15:18:15.173]Container exited with a non-zero exit code 143. For more detailed output, check the application tracking page: http://gac-bd-pro-nn02.bd.gac.co ... 685007310357_314615 Then click on links to logs of each attempt.. Failing the application.

太阳福音 发表于 2023-6-6 15:41
为啥减少union all的表就可以正常出现呢

分配的物理内存限制

"Container [pid=188988, containerID=container_e127_1685007310357_314615_02_000001] is running 27553792B beyond the 'PHYSICAL' memory limit. Current usage: 2.0 GB of 2 GB physical memory used; 4.0 GB of 4.2 GB virtual memory used. Killing container."

这个错误提示表明运行查询的容器超过了分配的物理内存限制，导致容器被终止。容器进程的ID为188988（一个bash进程）被杀死，因为它超过了物理内存限制。

要解决此问题，您有几个选项：

1. 优化Hive查询：检查您的查询并优化它以使用更少的资源。这可能涉及重新编写查询或调整Tez或Hive的配置设置，以更有效地使用内存。

2. 增加容器内存：如果集群上可用的资源允许，您可以增加分配给运行Hive查询的容器的内存。这可以通过修改YARN的相关配置来实现，例如增加YARN的容器内存分配上限。

请注意，这些解决方案可能需要根据您的特定环境和资源配置进行调整。在调整配置之前，请确保您了解系统的资源状况，并确保所做的更改符合集群的资源限制。

yhdata_yzm 发表于 2023-6-6 15:32
"Container  is running 27553792B beyond the 'PHYSICAL' memory limit. Current usage: 2.0 GB of 2 GB ...

为啥减少union all的表就可以正常出现呢