转:http://blog.csdn.net/yydcj/article/details/8482916
我自己平时处理是: ps 获取这个服务的进程号,全部kill掉,再sh startup.sh 启动进程并确认启动成功
1 问题现象
执行shutdown.sh之后,Web已经无法访问:
Tomcat关联java进程仍然存活:
- #/home/apache-tomcat-7.0.34/bin/shutdown.sh
- Using CATALINA_BASE: /home/apache-tomcat-7.0.34
- Using CATALINA_HOME: /home/apache-tomcat-7.0.34
- Using CATALINA_TMPDIR: /home/apache-tomcat-7.0.34/temp
- Using JRE_HOME: /home/jdk1.6.0_34
- Using CLASSPATH: /home/apache-tomcat-7.0.34/bin/bootstrap.jar:/home/apache-tomcat-7.0.34/bin/tomcat-juli.jar
- # ps aux | grep 'java.*tomcat' | grep -v grep
- root 22465 15.3 8.6 1294300 346264 pts/0 Sl 14:55 1:08 /home/jdk1.6.0_34/bin/java -Djava.util.logging.config.file=/home/apache-tomcat-7.0.34/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/home/apache-tomcat-7.0.34/endorsed -classpath /home/apache-tomcat-7.0.34/bin/bootstrap.jar:/home/apache-tomcat-7.0.34/bin/tomcat-juli.jar -Dcatalina.base=/home/apache-tomcat-7.0.34 -Dcatalina.home=/home/apache-tomcat-7.0.34 -Djava.io.tmpdir=/home/apache-tomcat-7.0.34/temp org.apache.catalina.startup.Bootstrap start
2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader clearReferencesJdbccatalina.out报内存泄露警告:
- 严重: The web application [] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
- 严重: The web application [] appears to have started a thread named [Thread-2] but has failed to stop it. This is very likely to create a memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
- 严重: The web application [] appears to have started a thread named [thread-snatch-picture] but has failed to stop it. This is very likely to create a memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
- 严重: The web application [] appears to have started a thread named [Xmemcached-Reactor-0] but has failed to stop it. This is very likely to create a memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
- 严重: The web application [] appears to have started a thread named [Xmemcached-Reactor-1] but has failed to stop it. This is very likely to create a memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
- 严重: The web application [] appears to have started a thread named [Xmemcached-Reactor-2] but has failed to stop it. This is very likely to create a memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
- 严重: The web application [] appears to have started a thread named [Xmemcached-Reactor-3] but has failed to stop it. This is very likely to create a memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
- 严重: The web application [] appears to have started a thread named [Xmemcached-Reactor-4] but has failed to stop it. This is very likely to create a memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
- 严重: The web application [] appears to have started a thread named [Xmemcached-Reactor-5] but has failed to stop it. This is very likely to create a memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
- 严重: The web application [] appears to have started a thread named [Xmemcached-Reactor-6] but has failed to stop it. This is very likely to create a memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
- 严重: The web application [] appears to have started a thread named [Xmemcached-Reactor-7] but has failed to stop it. This is very likely to create a memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
- 严重: The web application [] appears to have started a thread named [Heal-Session-Thread] but has failed to stop it. This is very likely to create a memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
- 严重: The web application [] appears to have started a thread named [pool-2-thread-1] but has failed to stop it. This is very likely to create a memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
- 严重: The web application [] appears to have started a thread named [pool-2-thread-2] but has failed to stop it. This is very likely to create a memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
- 严重: The web application [] appears to have started a thread named [pool-2-thread-3] but has failed to stop it. This is very likely to create a memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
- 严重: The web application [] appears to have started a thread named [pool-2-thread-4] but has failed to stop it. This is very likely to create a memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
- 严重: The web application [] appears to have started a thread named [Timer-0] but has failed to stop it. This is very likely to create a memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
- 严重: The web application [] appears to have started a thread named [MySQL Statement Cancellation Timer] but has failed to stop it. This is very likely to create a memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
- 严重: The web application [] created a ThreadLocal with key of type [com.opensymphony.xwork2.inject.ContainerImpl$10] (value [com.opensymphony.xwork2.inject.ContainerImpl$10@b6c562]) and a value of type [java.lang.Object[]] (value [[Ljava.lang.Object;@1d97383]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
- 严重: The web application [] created a ThreadLocal with key of type [com.opensymphony.xwork2.inject.ContainerImpl$10] (value [com.opensymphony.xwork2.inject.ContainerImpl$10@b6c562]) and a value of type [java.lang.Object[]] (value [[Ljava.lang.Object;@9a827]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
- 严重: The web application [] created a ThreadLocal with key of type [com.opensymphony.xwork2.inject.ContainerImpl$10] (value [com.opensymphony.xwork2.inject.ContainerImpl$10@b6c562]) and a value of type [java.lang.Object[]] (value [[Ljava.lang.Object;@12e71f1]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
- 严重: The web application [] created a ThreadLocal with key of type [com.opensymphony.xwork2.inject.ContainerImpl$10] (value [com.opensymphony.xwork2.inject.ContainerImpl$10@b6c562]) and a value of type [java.lang.Object[]] (value [[Ljava.lang.Object;@855793]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
- 2013-1-8 15:02:53 org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
- 严重: The web application [] created a ThreadLocal with key of type [com.opensymphony.xwork2.inject.ContainerImpl$10] (value [com.opensymphony.xwork2.inject.ContainerImpl$10@b6c562]) and a value of type [java.lang.Object[]] (value [[Ljava.lang.Object;@5164df]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
2 问题分析
这个在windows下没有碰到过,因为此前跑Tomcat都是以服务而不是命令脚本的形式跑的,而且已经换了一个项目,所以暂时不考察windows下是否存在该问题。
难道是Tomcat版本问题?或者用带内存泄漏保护的Tomcat 7可以解决该问题?尝试将web应用跑在apache-tomcat-6.0.18、apache-tomcat-6.0.35、apache-tomcat-7.0.34,发现均存在无法shutdown.sh进程问题。
难道在CentOS(Linux)下,Tomcat无法用shutdown.sh停掉进程?显然不可能的。于是又把web应用从webapps中移走,换回tomcat自带的ROOT,果然启动后再shutdown.sh,查找不到原来的tomcat进程了,也就证明了是自己web应用的问题了。
3 解决方案
现在已经确定是web应用的问题了,所以可以提解决方案了。
3.1 Kill进程
忽略日志中的严重警告,因为这是关闭tomcat时候引起的,正常情况下不会发生这种内存泄露情况,而且Tomcat6.18以上版本的Tomcat已经做了内存泄露保护,交给Tomcat完成吧,我们只需要在shutdown.sh之后,补上一个kill -9 pid即可。要是嫌这样太麻烦了,可以如下这样改:
- ==============================bin/shutdown.sh
- exec "$PRGDIR"/"$EXECUTABLE" stop -force "$@" #加上 -force
- ==============================bin/catalina.sh
- 在PRGDIR=`dirname "$PRG"`后面加上
- if [ -z "$CATALINA_PID" ]; then
- CATALINA_PID=$PRGDIR/CATALINA_PID
- cat $CATALINA_PID
- fi
3.2 web应用中关闭后台线程
shutdown.sh后存在tomcat僵尸进程的原因是shutdown.sh后web应用没有去关闭其启用的后台线程,所以处理的办法是监听到tomcat关闭事件时,关闭掉其启用的后台线程。
Pstree -p pid打印进程的线程树,查看shutdown.sh后未关闭的后台线程:
通过Java VisualVM查看shutdown.sh后未关闭的活动线程
在web应用中,如果有后台线程,这时用shutdown.sh是关闭不掉Tomcat的。我们需要重写相关的ServletContextListener的contextDestroyed(ServletContextEventarg0)方法,在它里面关闭后台线程。
但是,现在很多web应用都是由spring来管理bean的,实际情况下我们可能需要注入很多第三方jar中的bean。比如在applicationContext.xml中配置如下bean:
<bean id ="dataSource" class ="org.apache.commons.dbcp.BasicDataSource"destroy-method="close">
则在org.apache.commons.dbcp.BasicDataSource需要做好销毁数据库线程池的逻辑。
当然我们也可以自己设置销毁逻辑。在web.xml文件中增加监听器。
- <listener>
- <listener-class>com.mysite.MySpecialListener</listener-class>
- </listener>
MySpecialListener实现如下:
- public class MySpecialListener extends ApplicationContextListener {
- @Override
- public void contextInitialized(ServletContextEvent sce) {
- // On Application Startup, please…
- // Usually I'll make a singleton in here, set up my pool, etc.
- }
- @Override
- public void contextDestroyed(ServletContextEvent sce) {
- // On Application Shutdown, please…
- // 1. Go fetch that DataSource
- Context initContext = new InitialContext();
- Context envContext = (Context)initContext.lookup("java:/comp/env");
- DataSource datasource = (DataSource)envContext.lookup("jdbc/database");
- // 2. Call the close() method on it
- datasource.close();
- // 3. Deregister Driver
- try {
- java.sql.Driver mySqlDriver = DriverManager.getDriver("jdbc:mysql://localhost:3306/");
- DriverManager.deregisterDriver(mySqlDriver);
- } catch (SQLException ex) {
- logger.info("Could not deregister driver:".concat(ex.getMessage()));
- }
- dataSource = null;
- }
- }