监控Tomcat解决方案（监控应用服务器系列文章）

前言：做了一个监控应用服务器的项目（支持Tocmat、WebSphere、WebLogic各版本）， 过程也算是磕磕绊绊，由于网上缺少相关资料，或者深陷于知识的海洋难以寻觅到有效的资料，因而走过不少弯路，遇过不少困难。为了留下点印记，给后来人留下 点经验之谈，助之少走弯路，故将这些经验整理出来，与大家分享。水平有限，难免疏漏，还望指正。如有疑问，欢迎留言，或者加入Q群参与讨论：35526521。

对监控Tomcat的前期方案调研，共有两种方案，比较如下

使用Tomcat提供的manager应用进行数据采集
  ◆ 可以使用现有的成熟代码，减少工作量
  ◆ 支持各不同版本时无差别
  ◆ 可能存在特殊需求而manager不能满足的情况
  ◆ 最重要的一个优点是，配置比较简单

使用JMX接口开发监控程序
  ◆ 全部代码需要从零开始，代码量较大
  ◆ 支持各不同版本比较麻烦，每个版本可能有差异
  ◆ 可支配性强
  ◆ 最重要的一个缺点是，配置比较麻烦

 

方案一、使用Tomcat提供的manager应用进行数据采集

ManageEngine Applications Manager（又称opManager）就是通过这种方式实现的。
使用这种方式，所监控Tomcat必须运行manager应用，缺省情况下，该应用总是运行在服务器中的。

增加manager角色用户

访问manager应用的用户的角色权限必须是 manager.
修改<TOMCAT_HOME>/conf目录下的tomcat-users.xml文件，在<tomcat-users>节点下添加一个user节点，即可创建一个用户。

Tomcat版本不同配置也有差异，5.x和6.x创建的用户角色应为manager，7.x创建的用户角色为manager-jmx，举例如下：

1、在5.x和6.x中创建一个manager角色的用户，用户名为admin，密码为admin123：
    <user username="chenfeng" password="chenfeng" roles="manager"/>

2、在7.x中创建一个manager角色的用户，用户名为admin，密码为admin123：
    <user username="chenfeng" password="chenfeng" roles="manager-jmx,manager-script,manager-status"/>

修改配置后，需要重新启动 Tomcat 服务器。
    连接manager时将用户名/密码指定为chenfeng/chenfeng

通过浏览器访问JMX Proxy Servlet

详见官方说明文档：
    http://tomcat.apache.org/tomcat-6.0-doc/manager-howto.html#Using_the_JMX_Proxy_Servlet

What is JMX Proxy Servlet

The JMX Proxy Servlet is a lightweight proxy to get and set the tomcat internals. (Or any class that has been exposed via an MBean) Its usage is not very user friendly but the UI is extremely help for integrating command line scripts for monitoring and changing the internals of tomcat. You can do two things with the proxy: get information and set information. For you to really understand the JMX Proxy Servlet, you should have a general understanding of JMX. If you don't know what JMX is, then prepare to be confused.

JMX Query command

This takes the form:
    http://webserver/manager/jmxproxy/?qry=STUFF
Where STUFF is the JMX query you wish to perform. For example, here are some queries you might wish to run:
   ◆ qry=*%3Atype%3DRequestProcessor%2C* --> type=RequestProcessorwhich will locate all workers which can process requests and report their state.
   ◆ qry=*%3Aj2eeType=Servlet%2c* --> j2eeType=Servletwhich return all loaded servlets.
   ◆ qry=Catalina%3Atype%3DEnvironment%2Cresourcetype%3DGlobal%2Cname%3DsimpleValue --> Catalina:type=Environment,resourcetype=Global,name=simpleValuewhich look for a specific MBean by the given name.
You'll need to experiment with this to really understand its capabilites. If you provide no qry parameter, then all of the MBeans will be displayed. We really recommend looking at the tomcat source code and understand the JMX spec to get a better understanding of all the queries you may run.

通过浏览器访问http://localhost:8080/manager/jmxproxy ，输入用户名密码，然后就可以看到返回了所有的监控信息

添加查询参数，返回特定的监控信息：
    例如 http://localhost:8080/manager/jmxproxy?qry=*%3Atype%3DRequestProcessor%2C*
    其中 *%3Atype%3DRequestProcessor%2C* 其实就是 *:type=RequestProcessor,*
    又如 http://localhost:8080/manager/jmxproxy?qry=*%3Aj2eeType%3DWebModule%2Cname%3D//localhost/ajaxrpc%2C*

在代码中访问JMX Proxy Servlet

通过浏览器访问JMX Proxy Servlet需要输入用户名密码，所以通过Java访问JMX Proxy Servlet的URL也需要授权访问：

URL url = new URL("http://localhost:8080/manager/jmxproxy?qry=*%3Atype%3DManager%2C*");
URLConnection conn = (URLConnection) url.openConnection();

// URL授权访问 -- Begin
String password = "admin:chenfeng"; // manager角色的用户
String encodedPassword = new BASE64Encoder().encode(password.getBytes());
conn.setRequestProperty("Authorization", "Basic " + encodedPassword);
// URL授权访问 -- End

InputStream is = conn.getInputStream();
BufferedReader bufreader = new BufferedReader(new InputStreamReader(is));
String line = null;
while ((line = bufreader.readLine()) != null) {
    System.out.println(line);
}

几个具体的例子

下面展示两个例子，一个是采集服务器基本信息，一个是采集Web应用列表信息，注意Tomcat 7.x和Tomcat 5.x、6.x之间存在很大的区别。

◆ 采集服务器基本信息

通过serverinfo命令查看服务器基本信息
    http://localhost:8080/manager/serverinfo

Tomcat 7.x的查询URL有变化：
    http://localhost:8080/manager/text/serverinfo

返回信息：

OK - Server info
Tomcat Version: Apache Tomcat/7.0.11
OS Name: Windows Vista
OS Version: 6.1
OS Architecture: x86
JVM Version: 1.6.0_13-b03
JVM Vendor: Sun Microsystems Inc.

◆ 采集Web应用列表信息

通过list命令查看Web应用列表和会话数信息
    http://localhost:8080/manager/list

Tomcat 7.x的查询URL有变化：
    http://localhost:8080/manager/text/list

返回信息：

OK - Listed applications for virtual host localhost
/:running:0:ROOT
/manager:running:1:manager
/docs:running:0:docs
/examples:running:0:examples
/host-manager:running:0:host-manager

方案二、使用JMX接口开发监控程序

Tomcat激活JMX远程配置

① ■ 先修改Tomcat的启动脚本，window下tomcat的bin/catalina.bat（linux为catalina.sh），添加以下内容，8999是jmxremote使用的端口号，第二个false表示不需要鉴权：

set JMX_REMOTE_CONFIG=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8999 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false
set CATALINA_OPTS=%CATALINA_OPTS% %JMX_REMOTE_CONFIG%

 

可以加在if "%OS%" == "Windows_NT" setlocal 一句后的大段的注释后面。

参考官方说明：
    http://tomcat.apache.org/tomcat-6.0-doc/monitoring.html#Enabling_JMX_Remote

② ■ 上面的配置是不需要鉴权的，如果需要鉴权则添加的内容为：

set JMX_REMOTE_CONFIG=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8999 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=true -Dcom.sun.management.jmxremote.password.file=../conf/jmxremote.password -Dcom.sun.management.jmxremote.access.file=../conf/jmxremote.access
set CATALINA_OPTS=%CATALINA_OPTS% %JMX_REMOTE_CONFIG%

 

  然后复制并修改授权文件，$JAVA_HOME/jre/lib/management下有jmxremote.access和jmxremote.password的模板文件，将两个文件复制到$CATALINA_BASE/conf目录下
  ● 修改$CATALINA_BASE/conf/jmxremote.access 添加内容：
     monitorRole readonly
     controlRole readwrite

  ● 修改$CATALINA_BASE/conf/jmxremote.password 添加内容：
     monitorRole tomcat
     controlRole tomcat

注意：如果只做第一步没有问题，进行了第二步Tomcat就启动不了，那么很可能是密码文件的权限问题
    需要修改jmxremote.password文件的权限，只有运行Tomcat的用户有访问权限
    Windows的NTFS文件系统下，选中文件，点右键 -->“属性”-->“安全”--> 点“高级”--> 点“更改权限”--> 去掉“从父项继承....”--> 弹出窗口中选“删除”，这样就删除了所有访问权限。再选“添加”--> “高级”--> “立即查找”，选中你的用户，例administrator，点“确定"，“确定"。来到权限项目窗口，勾选“完全控制”，点“确定”，OK了。

官方的提示：
    The password file should be read-only and only accessible by the operating system user Tomcat is running as.

③ ■ 重新启动Tomcat，在Windows命令行输入“netstat -a”查看配置的端口号是否已打开，如果打开，说明上面的配置成功了。

④ ■ 使用jconsole测试JMX。

   运行$JAVA_HOME/bin目录下的jconsole.exe，打开J2SE监视和管理控制台，然后建立连接，如果是本地的Tomcat则直接选择然后点击连接，如果是远程的，则进入远程选项卡，填写地址、端口号、用户名、口令即可连接。Mbean属性页中给出了相应的数据，Catalina中是tomcat的，java.lang是jvm的。对于加粗的黑体属性值，需双击一下才可看内容。

使用JMX监控Tomcat示例代码

String jmxURL = "service:jmx:rmi:///jndi/rmi://192.168.10.93:8999/jmxrmi";
JMXServiceURL serviceURL = new JMXServiceURL(jmxURL);

Map map = new HashMap();
// 用户名密码，在jmxremote.password文件中查看
String[] credentials = new String[] { "monitorRole", "tomcat" };
map.put("jmx.remote.credentials", credentials);
JMXConnector connector = JMXConnectorFactory.connect(serviceURL, map);
MBeanServerConnection mbsc = connector.getMBeanServerConnection();

// 端口最好是动态取得
ObjectName threadObjName = new ObjectName("Catalina:type=ThreadPool,name=http-8080");
MBeanInfo mbInfo = mbsc.getMBeanInfo(threadObjName);

// tomcat的线程数对应的属性值
String attrName = "currentThreadCount";
MBeanAttributeInfo[] mbAttributes = mbInfo.getAttributes();
System.out.println("currentThreadCount:" + mbsc.getAttribute(threadObjName, attrName));

完整的示例代码文件

import java.lang.management.MemoryUsage;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.Formatter;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;
import java.util.Set;

import javax.management.MBeanAttributeInfo;
import javax.management.MBeanInfo;
import javax.management.MBeanServerConnection;
import javax.management.ObjectInstance;
import javax.management.ObjectName;
import javax.management.openmbean.CompositeDataSupport;
import javax.management.remote.JMXConnector;
import javax.management.remote.JMXConnectorFactory;
import javax.management.remote.JMXServiceURL;

/**
 * 使用JMX监控Tomcat示例
 *
 * @author 陳峰
 */
public class JMXTest {

    /**
     * @param args
     */
    public static void main(String[] args) {
        try {

            String jmxURL = "service:jmx:rmi:///jndi/rmi://127.0.0.1:8999/jmxrmi";

            JMXServiceURL serviceURL = new JMXServiceURL(jmxURL);

            Map map = new HashMap();
            String[] credentials = new String[] { "monitorRole", "tomcat" };
            map.put("jmx.remote.credentials", credentials);
            JMXConnector connector = JMXConnectorFactory.connect(serviceURL,
                    map);
            MBeanServerConnection mbsc = connector.getMBeanServerConnection();

            // 端口最好是动态取得
            ObjectName threadObjName = new ObjectName(
                    "Catalina:type=ThreadPool,name=http-8080");
            MBeanInfo mbInfo = mbsc.getMBeanInfo(threadObjName);

            String attrName = "currentThreadCount";// tomcat的线程数对应的属性值
            MBeanAttributeInfo[] mbAttributes = mbInfo.getAttributes();
            System.out.println("currentThreadCount:"
                    + mbsc.getAttribute(threadObjName, attrName));

            // heap
            for (int j = 0; j < mbsc.getDomains().length; j++) {
                System.out.println("###########" + mbsc.getDomains()[j]);
            }
            Set MBeanset = mbsc.queryMBeans(null, null);
            System.out.println("MBeanset.size() : " + MBeanset.size());
            Iterator MBeansetIterator = MBeanset.iterator();
            while (MBeansetIterator.hasNext()) {
                ObjectInstance objectInstance = (ObjectInstance) MBeansetIterator
                        .next();
                ObjectName objectName = objectInstance.getObjectName();
                String canonicalName = objectName.getCanonicalName();
                System.out.println("canonicalName : " + canonicalName);
                if (canonicalName
                        .equals("Catalina:host=localhost,type=Cluster")) {
                    // Get details of cluster MBeans
                    System.out.println("Cluster MBeans Details:");
                    System.out
                            .println("=========================================");
                    // getMBeansDetails(canonicalName);
                    String canonicalKeyPropList = objectName
                            .getCanonicalKeyPropertyListString();
                }
            }
            // ------------------------- system ----------------------
            ObjectName runtimeObjName = new ObjectName("java.lang:type=Runtime");
            System.out.println("厂商:"
                    + (String) mbsc.getAttribute(runtimeObjName, "VmVendor"));
            System.out.println("程序:"
                    + (String) mbsc.getAttribute(runtimeObjName, "VmName"));
            System.out.println("版本:"
                    + (String) mbsc.getAttribute(runtimeObjName, "VmVersion"));
            Date starttime = new Date((Long) mbsc.getAttribute(runtimeObjName,
                    "StartTime"));
            SimpleDateFormat df = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
            System.out.println("启动时间:" + df.format(starttime));

            Long timespan = (Long) mbsc.getAttribute(runtimeObjName, "Uptime");
            System.out.println("连续工作时间:" + JMXTest.formatTimeSpan(timespan));
            // ------------------------ JVM -------------------------
            // 堆使用率
            ObjectName heapObjName = new ObjectName("java.lang:type=Memory");
            MemoryUsage heapMemoryUsage = MemoryUsage
                    .from((CompositeDataSupport) mbsc.getAttribute(heapObjName,
                            "HeapMemoryUsage"));
            long maxMemory = heapMemoryUsage.getMax();// 堆最大
            long commitMemory = heapMemoryUsage.getCommitted();// 堆当前分配
            long usedMemory = heapMemoryUsage.getUsed();
            System.out.println("heap:" + (double) usedMemory * 100
                    / commitMemory + "%");// 堆使用率

            MemoryUsage nonheapMemoryUsage = MemoryUsage
                    .from((CompositeDataSupport) mbsc.getAttribute(heapObjName,
                            "NonHeapMemoryUsage"));
            long noncommitMemory = nonheapMemoryUsage.getCommitted();
            long nonusedMemory = heapMemoryUsage.getUsed();
            System.out.println("nonheap:" + (double) nonusedMemory * 100
                    / noncommitMemory + "%");

            ObjectName permObjName = new ObjectName(
                    "java.lang:type=MemoryPool,name=Perm Gen");
            MemoryUsage permGenUsage = MemoryUsage
                    .from((CompositeDataSupport) mbsc.getAttribute(permObjName,
                            "Usage"));
            long committed = permGenUsage.getCommitted();// 持久堆大小
            long used = heapMemoryUsage.getUsed();//
            System.out.println("perm gen:" + (double) used * 100 / committed
                    + "%");// 持久堆使用率

            // -------------------- Session ---------------
            ObjectName managerObjName = new ObjectName(
                    "Catalina:type=Manager,*");
            Set<ObjectName> s = mbsc.queryNames(managerObjName, null);
            for (ObjectName obj : s) {
                System.out.println("应用名:" + obj.getKeyProperty("path"));
                ObjectName objname = new ObjectName(obj.getCanonicalName());
                System.out.println("最大会话数:"
                        + mbsc.getAttribute(objname, "maxActiveSessions"));
                System.out.println("会话数:"
                        + mbsc.getAttribute(objname, "activeSessions"));
                System.out.println("活动会话数:"
                        + mbsc.getAttribute(objname, "sessionCounter"));
            }

            // ----------------- Thread Pool ----------------
            ObjectName threadpoolObjName = new ObjectName(
                    "Catalina:type=ThreadPool,*");
            Set<ObjectName> s2 = mbsc.queryNames(threadpoolObjName, null);
            for (ObjectName obj : s2) {
                System.out.println("端口名:" + obj.getKeyProperty("name"));
                ObjectName objname = new ObjectName(obj.getCanonicalName());
                System.out.println("最大线程数:"
                        + mbsc.getAttribute(objname, "maxThreads"));
                System.out.println("当前线程数:"
                        + mbsc.getAttribute(objname, "currentThreadCount"));
                System.out.println("繁忙线程数:"
                        + mbsc.getAttribute(objname, "currentThreadsBusy"));
            }

        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public static String formatTimeSpan(long span) {
        long minseconds = span % 1000;

        span = span / 1000;
        long seconds = span % 60;

        span = span / 60;
        long mins = span % 60;

        span = span / 60;
        long hours = span % 24;

        span = span / 24;
        long days = span;
        return (new Formatter()).format("%1$d天 %2$02d:%3$02d:%4$02d.%5$03d",
                days, hours, mins, seconds, minseconds).toString();
    }
}

说明： 本文为原创，原发表在ITEye，详见本人专栏： 监控应用服务器