• 关于springmvc的helloworld的压测报告


      都说hello world 很简单,应该能承受很大的请求压力,那么到底有多大?你知道吗?如果知道,那咱们就不继续了。如果不知道,我们来看一下!

    1. 准备工作,快速建立一个基于springmvc的helloworld

    1.1. 在pom.xml引入spring必须的包级日志组件

    <?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
        <modelVersion>4.0.0</modelVersion>
    
        <groupId>com.yougewe</groupId>
        <artifactId>mvn-local-test</artifactId>
        <version>1.0-SNAPSHOT</version>
        <packaging>war</packaging>
    
        <properties>
            <org.springframework.version>4.3.20.RELEASE</org.springframework.version>
            <freemarker.version>2.3.23</freemarker.version>
            <slf4j.version>1.7.12</slf4j.version>
            <mybatis.version>3.4.5</mybatis.version>
            <aspectj.version>1.8.13</aspectj.version>
        </properties>
    
        <dependencies>
                <!-- https://mvnrepository.com/artifact/com.alibaba/fastjson -->
                <dependency>
                    <groupId>com.alibaba</groupId>
                    <artifactId>fastjson</artifactId>
                    <version>1.2.44</version>
                </dependency>
    
            <dependency>
                <groupId>javax.servlet</groupId>
                <artifactId>javax.servlet-api</artifactId>
                <version>3.1.0</version>
            </dependency>
            <!-- https://mvnrepository.com/artifact/org.springframework/spring-core -->
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-core</artifactId>
                <version>${org.springframework.version}</version>
            </dependency>
    
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-context</artifactId>
                <version>${org.springframework.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-orm</artifactId>
                <version>${org.springframework.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-web</artifactId>
                <version>${org.springframework.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-webmvc</artifactId>
                <version>${org.springframework.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-aop</artifactId>
                <version>${org.springframework.version}</version>
            </dependency>
            <dependency>
                <groupId>org.aspectj</groupId>
                <artifactId>aspectjweaver</artifactId>
                <version>1.6.8</version>
            </dependency>
    
                <!-- https://mvnrepository.com/artifact/junit/junit -->
                <dependency>
                    <groupId>junit</groupId>
                    <artifactId>junit</artifactId>
                    <version>4.12</version>
                    <!-- 不关注位置先 -->
                    <!--<scope>test</scope>-->
                </dependency>
                <!-- https://mvnrepository.com/artifact/org.projectlombok/lombok -->
                <dependency>
                    <groupId>org.projectlombok</groupId>
                    <artifactId>lombok</artifactId>
                    <version>1.16.18</version>
                    <scope>provided</scope>
                </dependency>
                <!-- https://mvnrepository.com/artifact/com.rabbitmq/amqp-client -->
                <dependency>
                    <groupId>com.rabbitmq</groupId>
                    <artifactId>amqp-client</artifactId>
                    <version>5.1.1</version>
                </dependency>
    
            <!-- 配合slf4j使用  -->
            <!-- 日志记录 -->
            <dependency>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
                <version>1.2.17</version>
            </dependency>
            <dependency>
                <groupId>org.slf4j</groupId>
                <artifactId>jcl-over-slf4j</artifactId>
                <version>${slf4j.version}</version>
            </dependency>
            <dependency>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-api</artifactId>
                <version>${slf4j.version}</version>
            </dependency>
            <dependency>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
                <version>${slf4j.version}</version>
            </dependency>
            <dependency>
                <groupId>commons-logging</groupId>
                <artifactId>commons-logging</artifactId>
                <version>1.2</version>
            </dependency>
            <!-- https://mvnrepository.com/artifact/redis.clients/jedis -->
            <dependency>
                <groupId>redis.clients</groupId>
                <artifactId>jedis</artifactId>
                <version>2.9.0</version>
            </dependency>
    
    
        </dependencies>
    
        <build>
            <!--<finalName>sjd-yzbank-api</finalName>-->
            <plugins>
    
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-resources-plugin</artifactId>
                    <version>2.6</version>
                    <configuration>
                        <encoding>UTF-8</encoding>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>2.20.1</version>
                    <configuration>
                        <skipTests>false</skipTests>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-report-plugin</artifactId>
                    <version>2.20.1</version>
                    <!--<configuration>
                        <includes>
                            <include>**/*Test.java</include>
                        </includes>
                    </configuration>-->
                </plugin>
    
                <!-- clean插件 -->
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-clean-plugin</artifactId>
                </plugin>
    
                <!-- install插件 -->
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-install-plugin</artifactId>
                </plugin>
    
                <!-- deploy插件 -->
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-deploy-plugin</artifactId>
                </plugin>
    
                <!-- dependency插件 -->
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-dependency-plugin</artifactId>
                </plugin>
    
                <plugin>
                    <!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-compiler-plugin -->
                    <!-- 用于更好的编译,如jdk版本太低等问题 -->
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>3.6.1</version>
                    <configuration>
                        <source>1.8</source>
                        <target>1.8</target>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </project>

    1.2. 添加一个 web.xml, 只加一个dispatcherServlet 和一个字符集转换过滤器

        <filter>
            <filter-name>SpringEncodingFilter</filter-name>
            <filter-class>org.springframework.web.filter.CharacterEncodingFilter</filter-class>
            <init-param>
                <param-name>encoding</param-name>
                <param-value>UTF-8</param-value>
            </init-param>
            <init-param>
                <param-name>forceEncoding</param-name>
                <param-value>true</param-value>
            </init-param>
        </filter>
        <filter-mapping>
            <filter-name>SpringEncodingFilter</filter-name>
            <url-pattern>/*</url-pattern>
        </filter-mapping>
        <!-- 防止Spring内存溢出监听器 -->
        <listener>
            <listener-class>org.springframework.web.util.IntrospectorCleanupListener</listener-class>
        </listener>
        <!-- 如下 listener 会查找 WEB-INF/applicationContext.xml 文件 -->
        <!--<listener>
            <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
        </listener>-->
        <!-- springMVC核心配置 -->
        <servlet>
            <servlet-name>dispatcherServlet</servlet-name>
            <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
            <init-param>
                <param-name>contextConfigLocation</param-name>
                <param-value>classpath:applicationContext.xml</param-value>
            </init-param>
            <load-on-startup>1</load-on-startup>
        </servlet>
        <!-- 拦截设置 -->
        <servlet-mapping>
            <servlet-name>dispatcherServlet</servlet-name>
            <url-pattern>/*</url-pattern>
        </servlet-mapping>
        <welcome-file-list>
            <welcome-file>index.html</welcome-file>
            <welcome-file>index.jsp</welcome-file>
        </welcome-file-list>

    1.3. 添加log4j.properties日志配置文件

    log4j.rootLogger=DEBUG,console,im,logFile
    log4j.additivity.org.apache=true
    # 控制台(console)
    log4j.appender.console=org.apache.log4j.ConsoleAppender
    log4j.appender.console.Threshold=DEBUG
    log4j.appender.console.ImmediateFlush=true
    log4j.appender.console.Target=System.out
    log4j.appender.console.layout=org.apache.log4j.PatternLayout
    log4j.appender.console.layout.ConversionPattern=%d(%r) [%t] %-5p %l: %m %x %n
    # 日志文件(logFile)
    log4j.appender.logFile=org.apache.log4j.FileAppender
    log4j.appender.logFile.Threshold=DEBUG
    log4j.appender.logFile.ImmediateFlush=true
    log4j.appender.logFile.Append=true
    log4j.appender.logFile.File=D:/logs/log.log4j
    log4j.appender.logFile.layout=org.apache.log4j.PatternLayout
    log4j.appender.logFile.layout.ConversionPattern=%d(%r) [%t] %-5p %l: %m %x %n
    # 回滚文件(rollingFile)
    log4j.appender.rollingFile=org.apache.log4j.RollingFileAppender
    log4j.appender.rollingFile.Threshold=DEBUG
    log4j.appender.rollingFile.ImmediateFlush=true
    log4j.appender.rollingFile.Append=true
    log4j.appender.rollingFile.File=D:/logs/log.log4j
    log4j.appender.rollingFile.MaxFileSize=200KB
    log4j.appender.rollingFile.MaxBackupIndex=50
    log4j.appender.rollingFile.layout=org.apache.log4j.PatternLayout
    log4j.appender.rollingFile.layout.ConversionPattern=%d(%r) [%t] %-5p %l: %m %x %n

    1.4. 添加一个HelloController, 返回一个 字符串

    @RestController
    @RequestMapping("/hello")
    public class HelloController {
    
        @RequestMapping(value = "/world", method = {RequestMethod.GET, RequestMethod.POST}, produces = "application/json")
        @ResponseBody
        public Object world(@ModelAttribute UserInfo info) {
            return "hello world!";
        }
    }

      好了,一切准备就绪!是时候让我们来看一下它的能力如何了!

    测试工具: jmeter + 台式测试机一台

    前提1:
      机器配置: 4c8g 笔记本
      网卡: 高通 QCA9377, 共享带宽: 100M, 网络测速: 下载:3.5MB/s 上传:475KB/s
      web容器: tomcat7, 运行模式: apr(apache portable runtime)

    压测过程如下:

      并发100-5组连续请求, TPS: 146.2, error: 0, 平均响应时间: 0.381s, 最大响应时间: 2.44s

      server端cpu有一瞬间的飙高,内存几乎无变化!

      下面,按照规律,翻倍并发,200-5组连续请求!看下数据!

      TPS增加了,为275;平均响应时间慢了点,0.429;没有 error。

      再翻倍并发量:400-5组连续请求:

      TPS下降了,为208;平均响应时间翻番,1.1秒;不过幸好还是没有error;

      再翻倍并发量:800-5组连续请求:

      TPS再次下降,为144;平均响应时间再翻番,3.6秒,这在生产环境已经不符合要求了!error仍为0;

       再翻倍: 1600-5组: server cpu 一直高居不下,直接jvm宕机。

     

      注意,此时已经有error出现了,1.09%的错误率! TPS继续下降: 93.4,平均响应时间继续翻倍:8.9秒;

      综上,springmvc的helloworld 能力差不多也就在1600了,因为已经有错误出现,在实际生产中已经完全不能接受了!

      不过,我还是想看一下server到底能承受多大压力,也就是jvm完全宕机!

      压到3000并发-5组:

      看起来还能响应,其实再server端,jvm已经挂掉了!所以结论是,tomcat7(apr模式)是扛不住3000并发的!

    jvm 宕机日志样例如下:

    十二月 12, 2018 11:54:25 上午 org.apache.tomcat.util.net.AprEndpoint$Poller run
    警告: Unexpected poller error
    java.lang.NullPointerException
        at org.apache.tomcat.util.net.AprEndpoint$AprSocketWrapper.access$400(AprEndpoint.java:2537)
        at org.apache.tomcat.util.net.AprEndpoint$Poller.run(AprEndpoint.java:1887)
        at java.lang.Thread.run(Thread.java:745)
    
    十二月 12, 2018 11:54:25 上午 org.apache.tomcat.util.net.AprEndpoint$Poller run
    警告: Unexpected poller error
    java.lang.NullPointerException
        at org.apache.tomcat.util.net.AprEndpoint$AprSocketWrapper.access$400(AprEndpoint.java:2537)
        at org.apache.tomcat.util.net.AprEndpoint$Poller.run(AprEndpoint.java:1887)
        at java.lang.Thread.run(Thread.java:745)
    
    #
    # A fatal error has been detected by the Java Runtime Environment:
    #
    #  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000180008005, pid=13444, tid=0x0000000000002e34
    #
    # JRE version: Java(TM) SE Runtime Environment (8.0_101-b13) (build 1.8.0_101-b13)
    # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.101-b13 mixed mode windows-amd64 compressed oops)
    # Problematic frame:
    # C  [tcnative-1.dll+0x8005]
    #
    # Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
    #
    # An error report file with more information is saved as:
    # D:servers	omcat-7.0.70inhs_err_pid13444.log
    #
    # If you would like to submit a bug report, please visit:
    #   http://bugreport.java.com/bugreport/crash.jsp
    # The crash happened outside the Java Virtual Machine in native code.
    # See problematic frame for where to report the bug.
    #
    Disconnected from the target VM, address: '127.0.0.1:58835', transport: 'socket'
    Disconnected from server

    其中,也指示了错误记录文件:hs_err_pid13444.log,其中大概是一些线程、栈、内存、cpu的总结内容!

      大概就是说, 

    #
    # A fatal error has been detected by the Java Runtime Environment:
    #
    #  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000180008005, pid=13444, tid=0x0000000000002e34
    #
    # JRE version: Java(TM) SE Runtime Environment (8.0_101-b13) (build 1.8.0_101-b13)
    # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.101-b13 mixed mode windows-amd64 compressed oops)
    # Problematic frame:
    # C  [tcnative-1.dll+0x8005]
    #
    # Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
    #
    # If you would like to submit a bug report, please visit:
    #   http://bugreport.java.com/bugreport/crash.jsp
    # The crash happened outside the Java Virtual Machine in native code.
    # See problematic frame for where to report the bug.
    #
    
    ---------------  T H R E A D  ---------------
    
    Current thread (0x000000001a1e5800):  JavaThread "http-apr-8080-Poller" daemon [_thread_in_native, id=11828, stack(0x000000001c420000,0x000000001c520000)]
    
    siginfo: ExceptionCode=0xc0000005, writing address 0x0000000000004774
    
    Registers:
    RAX=0x0000000036a64ec0, RBX=0x00000000356a2d60, RCX=0x0000000000004774, RDX=0x0000000036a64df8
    RSP=0x000000001c51f010, RBP=0x00000000000025e0, RSI=0x000000000000025e, RDI=0x00000000003ae258
    R8 =0x0000000000000000, R9 =0x00000000003960c0, R10=0x0000000000004000, R11=0x0000000036a64ec0
    R12=0x00000000000025e8, R13=0x00057ccb27bc9160, R14=0x00000000003ae2a8, R15=0x0000000000000001
    RIP=0x0000000180008005, EFLAGS=0x0000000000010246
    
    Top of Stack: (sp=0x000000001c51f010)
    0x000000001c51f010:   000000000000025e 00000000003ae258
    0x000000001c51f020:   000000001c51f088 000000001c51f090
    0x000000001c51f030:   000000001c51f500 0000000088a1ccb0
    0x000000001c51f040:   000000001a1e5800 000000001c51f0e0
    0x000000001c51f050:   0000000089172b08 0000000000000000
    0x000000001c51f060:   0000000000000001 000000001c51f0f0

     戳详情:

    #
    # A fatal error has been detected by the Java Runtime Environment:
    #
    #  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000180008005, pid=13444, tid=0x0000000000002e34
    #
    # JRE version: Java(TM) SE Runtime Environment (8.0_101-b13) (build 1.8.0_101-b13)
    # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.101-b13 mixed mode windows-amd64 compressed oops)
    # Problematic frame:
    # C  [tcnative-1.dll+0x8005]
    #
    # Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
    #
    # If you would like to submit a bug report, please visit:
    #   http://bugreport.java.com/bugreport/crash.jsp
    # The crash happened outside the Java Virtual Machine in native code.
    # See problematic frame for where to report the bug.
    #
    
    ---------------  T H R E A D  ---------------
    
    Current thread (0x000000001a1e5800):  JavaThread "http-apr-8080-Poller" daemon [_thread_in_native, id=11828, stack(0x000000001c420000,0x000000001c520000)]
    
    siginfo: ExceptionCode=0xc0000005, writing address 0x0000000000004774
    
    Registers:
    RAX=0x0000000036a64ec0, RBX=0x00000000356a2d60, RCX=0x0000000000004774, RDX=0x0000000036a64df8
    RSP=0x000000001c51f010, RBP=0x00000000000025e0, RSI=0x000000000000025e, RDI=0x00000000003ae258
    R8 =0x0000000000000000, R9 =0x00000000003960c0, R10=0x0000000000004000, R11=0x0000000036a64ec0
    R12=0x00000000000025e8, R13=0x00057ccb27bc9160, R14=0x00000000003ae2a8, R15=0x0000000000000001
    RIP=0x0000000180008005, EFLAGS=0x0000000000010246
    
    Top of Stack: (sp=0x000000001c51f010)
    0x000000001c51f010:   000000000000025e 00000000003ae258
    0x000000001c51f020:   000000001c51f088 000000001c51f090
    0x000000001c51f030:   000000001c51f500 0000000088a1ccb0
    0x000000001c51f040:   000000001a1e5800 000000001c51f0e0
    0x000000001c51f050:   0000000089172b08 0000000000000000
    0x000000001c51f060:   0000000000000001 000000001c51f0f0
    0x000000001c51f070:   0000000000000000 0000000002d9f968
    0x000000001c51f080:   000000001a1e59f8 00000000000003e4
    0x000000001c51f090:   00000000003aac38 0000000089033de8
    0x000000001c51f0a0:   000000001c51f0c0 0000000000000001
    0x000000001c51f0b0:   36a8db6836187b68 00000000ede7bdd0
    0x000000001c51f0c0:   000000008912ea98 0000000052cd606f
    0x000000001c51f0d0:   01d491ce629c2dc0 000000008912ce88
    0x000000001c51f0e0:   0000000088a1ccb0 0000000000000202
    0x000000001c51f0f0:   000000001c51f500 000000000389d324
    0x000000001c51f100:   01d491ce629c2dc0 0000000052c569bd 
    
    Instructions: (pc=0x0000000180008005)
    0x0000000180007fe5:   48 8b 94 24 80 00 00 00 48 8b 4f 10 e8 ca 8a 01
    0x0000000180007ff5:   00 4c 8b 5b 38 49 8b 13 48 8b 43 38 48 8b 48 08
    0x0000000180008005:   48 89 11 48 8b 43 38 48 8b 50 08 48 8b 43 38 48
    0x0000000180008015:   8b 08 48 89 51 08 48 8b 43 38 4c 89 30 48 8b 4f 
    
    
    Register to memory mapping:
    
    RAX=0x0000000036a64ec0 is an unknown value
    RBX=0x00000000356a2d60 is an unknown value
    RCX=0x0000000000004774 is an unknown value
    RDX=0x0000000036a64df8 is an unknown value
    RSP=0x000000001c51f010 is pointing into the stack for thread: 0x000000001a1e5800
    RBP=0x00000000000025e0 is an unknown value
    RSI=0x000000000000025e is an unknown value
    RDI=0x00000000003ae258 is an unknown value
    R8 =0x0000000000000000 is an unknown value
    R9 =0x00000000003960c0 is an unknown value
    R10=0x0000000000004000 is an unknown value
    R11=0x0000000036a64ec0 is an unknown value
    R12=0x00000000000025e8 is an unknown value
    R13=0x00057ccb27bc9160 is an unknown value
    R14=0x00000000003ae2a8 is an unknown value
    R15=0x0000000000000001 is an unknown value
    
    
    Stack: [0x000000001c420000,0x000000001c520000],  sp=0x000000001c51f010,  free space=1020k
    Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
    C  [tcnative-1.dll+0x8005]
    C  0x0000000002d9f968
    
    Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
    J 3241  org.apache.tomcat.jni.Poll.poll(JJ[JZ)I (0 bytes) @ 0x0000000002d9f8e2 [0x0000000002d9f880+0x62]
    J 6143% C1 org.apache.tomcat.util.net.AprEndpoint$Poller.run()V (2241 bytes) @ 0x000000000389d324 [0x0000000003899ec0+0x3464]
    j  java.lang.Thread.run()V+11
    v  ~StubRoutines::call_stub
    
    ---------------  P R O C E S S  ---------------
    
    Java Threads: ( => current thread )
      0x0000000020785000 JavaThread "http-apr-8080-exec-770" daemon [_thread_blocked, id=14488, stack(0x0000000035820000,0x0000000035920000)]
      0x0000000020784800 JavaThread "http-apr-8080-exec-769" daemon [_thread_blocked, id=5880, stack(0x0000000035930000,0x0000000035a30000)]
      0x0000000020783800 JavaThread "http-apr-8080-exec-768" daemon [_thread_blocked, id=8632, stack(0x0000000034db0000,0x0000000034eb0000)]
      0x0000000020783000 JavaThread "http-apr-8080-exec-767" daemon [_thread_blocked, id=11628, stack(0x0000000034500000,0x0000000034600000)]
      0x0000000020782000 JavaThread "http-apr-8080-exec-766" daemon [_thread_blocked, id=13600, stack(0x0000000034bd0000,0x0000000034cd0000)]
      0x0000000020781800 JavaThread "http-apr-8080-exec-765" daemon [_thread_blocked, id=11512, stack(0x00000000348b0000,0x00000000349b0000)]
      0x0000000020780800 JavaThread "http-apr-8080-exec-764" daemon [_thread_blocked, id=4848, stack(0x0000000032980000,0x0000000032a80000)]
      0x0000000020780000 JavaThread "http-apr-8080-exec-763" daemon [_thread_blocked, id=10048, stack(0x0000000033ef0000,0x0000000033ff0000)]
      0x000000002077f000 JavaThread "http-apr-8080-exec-762" daemon [_thread_blocked, id=12072, stack(0x00000000347a0000,0x00000000348a0000)]
      0x000000002395f000 JavaThread "http-apr-8080-exec-761" daemon [_thread_blocked, id=10588, stack(0x0000000034650000,0x0000000034750000)]
      0x000000002395e000 JavaThread "http-apr-8080-exec-760" daemon [_thread_blocked, id=14920, stack(0x00000000343e0000,0x00000000344e0000)]
      0x000000002395d800 JavaThread "http-apr-8080-exec-759" daemon [_thread_blocked, id=12232, stack(0x00000000341f0000,0x00000000342f0000)]
      0x000000002395c800 JavaThread "http-apr-8080-exec-758" daemon [_thread_blocked, id=9540, stack(0x0000000034000000,0x0000000034100000)]
      0x000000002395c000 JavaThread "http-apr-8080-exec-757" daemon [_thread_blocked, id=13092, stack(0x0000000031d30000,0x0000000031e30000)]
      0x000000002395b000 JavaThread "http-apr-8080-exec-756" daemon [_thread_blocked, id=4624, stack(0x0000000033c50000,0x0000000033d50000)]
      0x000000002395a800 JavaThread "http-apr-8080-exec-755" daemon [_thread_blocked, id=9964, stack(0x0000000033dd0000,0x0000000033ed0000)]
      0x0000000023959800 JavaThread "http-apr-8080-exec-754" daemon [_thread_blocked, id=9704, stack(0x0000000033b30000,0x0000000033c30000)]
      0x0000000023959000 JavaThread "http-apr-8080-exec-753" daemon [_thread_blocked, id=7484, stack(0x0000000033940000,0x0000000033a40000)]
      0x0000000023958000 JavaThread "http-apr-8080-exec-752" daemon [_thread_blocked, id=11976, stack(0x0000000033660000,0x0000000033760000)]
      0x0000000023957800 JavaThread "http-apr-8080-exec-751" daemon [_thread_blocked, id=15160, stack(0x00000000337c0000,0x00000000338c0000)]
      0x0000000023956800 JavaThread "http-apr-8080-exec-750" daemon [_thread_blocked, id=1320, stack(0x00000000332d0000,0x00000000333d0000)]
      0x0000000023956000 JavaThread "http-apr-8080-exec-749" daemon [_thread_blocked, id=8720, stack(0x0000000033560000,0x0000000033660000)]
      0x0000000023955000 JavaThread "http-apr-8080-exec-748" daemon [_thread_blocked, id=14136, stack(0x0000000033410000,0x0000000033510000)]
      0x0000000023954800 JavaThread "http-apr-8080-exec-747" daemon [_thread_blocked, id=10148, stack(0x00000000321c0000,0x00000000322c0000)]
      0x0000000023953800 JavaThread "http-apr-8080-exec-746" daemon [_thread_blocked, id=7368, stack(0x00000000331b0000,0x00000000332b0000)]
      0x0000000023953000 JavaThread "http-apr-8080-exec-745" daemon [_thread_blocked, id=11400, stack(0x0000000032fd0000,0x00000000330d0000)]
      0x0000000023952000 JavaThread "http-apr-8080-exec-744" daemon [_thread_blocked, id=10248, stack(0x0000000032de0000,0x0000000032ee0000)]
      0x0000000023951800 JavaThread "http-apr-8080-exec-743" daemon [_thread_blocked, id=5200, stack(0x0000000032c50000,0x0000000032d50000)]
      0x0000000023950800 JavaThread "http-apr-8080-exec-740" daemon [_thread_blocked, id=12396, stack(0x0000000032a80000,0x0000000032b80000)]
      0x0000000023950000 JavaThread "http-apr-8080-exec-742" daemon [_thread_blocked, id=6516, stack(0x0000000032880000,0x0000000032980000)]
      0x000000001e40b000 JavaThread "http-apr-8080-exec-741" daemon [_thread_blocked, id=8412, stack(0x0000000032710000,0x0000000032810000)]
      0x000000001e40a000 JavaThread "http-apr-8080-exec-739" daemon [_thread_blocked, id=8996, stack(0x00000000325e0000,0x00000000326e0000)]
      0x000000001e409800 JavaThread "http-apr-8080-exec-738" daemon [_thread_blocked, id=7712, stack(0x0000000031f40000,0x0000000032040000)]
      0x000000001e408800 JavaThread "http-apr-8080-exec-737" daemon [_thread_blocked, id=12004, stack(0x000000002ed90000,0x000000002ee90000)]
      0x000000001e408000 JavaThread "http-apr-8080-exec-736" daemon [_thread_blocked, id=9200, stack(0x0000000032470000,0x0000000032570000)]
      0x000000001e407000 JavaThread "http-apr-8080-exec-735" daemon [_thread_blocked, id=12752, stack(0x00000000322d0000,0x00000000323d0000)]
      0x000000001e406800 JavaThread "http-apr-8080-exec-734" daemon [_thread_blocked, id=12632, stack(0x0000000032040000,0x0000000032140000)]
      0x000000001e405800 JavaThread "http-apr-8080-exec-733" daemon [_thread_blocked, id=11596, stack(0x000000002f860000,0x000000002f960000)]
      0x000000001e405000 JavaThread "http-apr-8080-exec-732" daemon [_thread_blocked, id=4564, stack(0x00000000309f0000,0x0000000030af0000)]
      0x000000001e404000 JavaThread "http-apr-8080-exec-730" daemon [_thread_blocked, id=14540, stack(0x0000000031e30000,0x0000000031f30000)]
      0x000000001ac6b000 JavaThread "http-apr-8080-exec-731" daemon [_thread_blocked, id=2600, stack(0x0000000030e20000,0x0000000030f20000)]
      0x000000001ac6a000 JavaThread "http-apr-8080-exec-729" daemon [_thread_blocked, id=13624, stack(0x0000000031c20000,0x0000000031d20000)]
      0x000000001ac69800 JavaThread "http-apr-8080-exec-728" daemon [_thread_blocked, id=6428, stack(0x0000000031a60000,0x0000000031b60000)]
      0x000000001ac68800 JavaThread "http-apr-8080-exec-727" daemon [_thread_blocked, id=6384, stack(0x0000000025860000,0x0000000025960000)]
      0x000000001ac68000 JavaThread "http-apr-8080-exec-726" daemon [_thread_blocked, id=7200, stack(0x0000000031960000,0x0000000031a60000)]
      0x000000001ac67000 JavaThread "http-apr-8080-exec-725" daemon [_thread_blocked, id=14740, stack(0x00000000314c0000,0x00000000315c0000)]
      0x000000001ac66800 JavaThread "http-apr-8080-exec-724" daemon [_thread_blocked, id=12892, stack(0x00000000304f0000,0x00000000305f0000)]
      0x000000001ac65800 JavaThread "http-apr-8080-exec-723" daemon [_thread_blocked, id=7992, stack(0x0000000031780000,0x0000000031880000)]
      0x000000001ac65000 JavaThread "http-apr-8080-exec-722" daemon [_thread_blocked, id=4788, stack(0x00000000315e0000,0x00000000316e0000)]
      0x000000001ac64000 JavaThread "http-apr-8080-exec-721" daemon [_thread_blocked, id=10808, stack(0x00000000313b0000,0x00000000314b0000)]
      0x000000001b040000 JavaThread "http-apr-8080-exec-720" daemon [_thread_blocked, id=14832, stack(0x0000000030c00000,0x0000000030d00000)]
      0x000000001b03f800 JavaThread "http-apr-8080-exec-719" daemon [_thread_blocked, id=13892, stack(0x00000000311d0000,0x00000000312d0000)]
      0x000000001b03e800 JavaThread "http-apr-8080-exec-718" daemon [_thread_blocked, id=10408, stack(0x0000000030f30000,0x0000000031030000)]
      0x000000001b03e000 JavaThread "http-apr-8080-exec-717" daemon [_thread_blocked, id=7220, stack(0x0000000030d10000,0x0000000030e10000)]
      0x000000001b03d000 JavaThread "http-apr-8080-exec-716" daemon [_thread_blocked, id=14512, stack(0x0000000030af0000,0x0000000030bf0000)]
      0x000000001b03c800 JavaThread "http-apr-8080-exec-715" daemon [_thread_blocked, id=3068, stack(0x0000000030740000,0x0000000030840000)]
      0x000000001b03b800 JavaThread "http-apr-8080-exec-714" daemon [_thread_blocked, id=1004, stack(0x00000000305f0000,0x00000000306f0000)]
      0x000000001b03b000 JavaThread "http-apr-8080-exec-713" daemon [_thread_blocked, id=11156, stack(0x0000000029be0000,0x0000000029ce0000)]
      0x000000001b03a000 JavaThread "http-apr-8080-exec-712" daemon [_thread_blocked, id=6740, stack(0x00000000303f0000,0x00000000304f0000)]
      0x000000001b039800 JavaThread "http-apr-8080-exec-709" daemon [_thread_blocked, id=11852, stack(0x0000000030250000,0x0000000030350000)]
      0x000000002167e800 JavaThread "http-apr-8080-exec-711" daemon [_thread_blocked, id=13836, stack(0x00000000300f0000,0x00000000301f0000)]
      0x000000002167e000 JavaThread "http-apr-8080-exec-710" daemon [_thread_blocked, id=9528, stack(0x000000002fe40000,0x000000002ff40000)]
      0x000000002167d000 JavaThread "http-apr-8080-exec-708" daemon [_thread_blocked, id=9636, stack(0x000000002ff40000,0x0000000030040000)]
      0x000000002167c800 JavaThread "http-apr-8080-exec-707" daemon [_thread_blocked, id=7904, stack(0x000000002fc90000,0x000000002fd90000)]
      0x000000002167b800 JavaThread "http-apr-8080-exec-706" daemon [_thread_blocked, id=13048, stack(0x000000002f630000,0x000000002f730000)]
      0x000000002167b000 JavaThread "http-apr-8080-exec-705" daemon [_thread_blocked, id=9536, stack(0x000000002faf0000,0x000000002fbf0000)]
      0x000000002167a000 JavaThread "http-apr-8080-exec-703" daemon [_thread_blocked, id=3868, stack(0x000000002f960000,0x000000002fa60000)]
      0x0000000021679800 JavaThread "http-apr-8080-exec-704" daemon [_thread_blocked, id=13344, stack(0x000000002f760000,0x000000002f860000)]
      0x0000000021678800 JavaThread "http-apr-8080-exec-702" daemon [_thread_blocked, id=6812, stack(0x000000002f520000,0x000000002f620000)]
      0x0000000021678000 JavaThread "http-apr-8080-exec-701" daemon [_thread_blocked, id=6804, stack(0x000000002f360000,0x000000002f460000)]
      0x000000001bb23800 JavaThread "http-apr-8080-exec-700" daemon [_thread_blocked, id=10328, stack(0x000000002cc30000,0x000000002cd30000)]
      0x000000001bb22800 JavaThread "http-apr-8080-exec-699" daemon [_thread_blocked, id=14236, stack(0x000000002f210000,0x000000002f310000)]
      0x000000001bb22000 JavaThread "http-apr-8080-exec-698" daemon [_thread_blocked, id=3444, stack(0x000000002f080000,0x000000002f180000)]
      0x000000001bb21000 JavaThread "http-apr-8080-exec-697" daemon [_thread_blocked, id=2548, stack(0x000000002ee90000,0x000000002ef90000)]
      0x000000001bb20800 JavaThread "http-apr-8080-exec-696" daemon [_thread_blocked, id=13268, stack(0x000000002ec90000,0x000000002ed90000)]
      0x000000001bb1f800 JavaThread "http-apr-8080-exec-695" daemon [_thread_blocked, id=14668, stack(0x000000002eb50000,0x000000002ec50000)]
      0x000000001bb1f000 JavaThread "http-apr-8080-exec-694" daemon [_thread_blocked, id=10132, stack(0x000000002e2d0000,0x000000002e3d0000)]
      0x000000001bb1e000 JavaThread "http-apr-8080-exec-689" daemon [_thread_blocked, id=10860, stack(0x000000002e980000,0x000000002ea80000)]
      0x000000001bb1d800 JavaThread "http-apr-8080-exec-693" daemon [_thread_blocked, id=8664, stack(0x000000002e840000,0x000000002e940000)]
      0x000000001bb1c800 JavaThread "http-apr-8080-exec-692" daemon [_thread_blocked, id=3928, stack(0x000000002e730000,0x000000002e830000)]
      0x0000000021666800 JavaThread "http-apr-8080-exec-691" daemon [_thread_blocked, id=6380, stack(0x000000002e610000,0x000000002e710000)]
      0x0000000021665800 JavaThread "http-apr-8080-exec-690" daemon [_thread_blocked, id=8848, stack(0x000000002db20000,0x000000002dc20000)]
      0x0000000021665000 JavaThread "http-apr-8080-exec-688" daemon [_thread_blocked, id=7752, stack(0x000000002e1b0000,0x000000002e2b0000)]
      0x0000000021664000 JavaThread "http-apr-8080-exec-687" daemon [_thread_blocked, id=11700, stack(0x000000002dff0000,0x000000002e0f0000)]
      0x0000000021663800 JavaThread "http-apr-8080-exec-686" daemon [_thread_blocked, id=12084, stack(0x000000002de20000,0x000000002df20000)]
      0x0000000021662800 JavaThread "http-apr-8080-exec-685" daemon [_thread_blocked, id=13356, stack(0x000000002d6c0000,0x000000002d7c0000)]
      0x0000000021662000 JavaThread "http-apr-8080-exec-684" daemon [_thread_blocked, id=13484, stack(0x000000002da10000,0x000000002db10000)]
      0x0000000021661000 JavaThread "http-apr-8080-exec-683" daemon [_thread_blocked, id=8748, stack(0x000000002d900000,0x000000002da00000)]
      0x0000000021660800 JavaThread "http-apr-8080-exec-682" daemon [_thread_blocked, id=13808, stack(0x000000002d7c0000,0x000000002d8c0000)]
      0x000000002165f800 JavaThread "http-apr-8080-exec-681" daemon [_thread_blocked, id=8964, stack(0x000000002d580000,0x000000002d680000)]
      0x000000001a1e4800 JavaThread "http-apr-8080-exec-680" daemon [_thread_blocked, id=13084, stack(0x000000002d390000,0x000000002d490000)]
      0x00000000242ec000 JavaThread "http-apr-8080-exec-679" daemon [_thread_blocked, id=11716, stack(0x000000002c800000,0x000000002c900000)]
      0x00000000242ea800 JavaThread "http-apr-8080-exec-678" daemon [_thread_blocked, id=5008, stack(0x000000002a950000,0x000000002aa50000)]
      0x00000000242e7800 JavaThread "http-apr-8080-exec-677" daemon [_thread_blocked, id=3672, stack(0x000000002d200000,0x000000002d300000)]
      0x00000000242e5800 JavaThread "http-apr-8080-exec-676" daemon [_thread_blocked, id=6772, stack(0x000000002d090000,0x000000002d190000)]
      0x00000000242eb800 JavaThread "http-apr-8080-exec-675" daemon [_thread_blocked, id=8288, stack(0x000000002cf20000,0x000000002d020000)]
      0x00000000242e9000 JavaThread "http-apr-8080-exec-674" daemon [_thread_blocked, id=7048, stack(0x000000002ca30000,0x000000002cb30000)]
      0x00000000242ea000 JavaThread "http-apr-8080-exec-673" daemon [_thread_blocked, id=13932, stack(0x000000002cd40000,0x000000002ce40000)]
      0x00000000242e8800 JavaThread "http-apr-8080-exec-672" daemon [_thread_blocked, id=13524, stack(0x000000002c910000,0x000000002ca10000)]
      0x000000001a4f9000 JavaThread "http-apr-8080-exec-671" daemon [_thread_blocked, id=8072, stack(0x000000002c6b0000,0x000000002c7b0000)]
      0x000000001a4f6800 JavaThread "http-apr-8080-exec-670" daemon [_thread_blocked, id=9524, stack(0x000000002c570000,0x000000002c670000)]
      0x000000001a4fa800 JavaThread "http-apr-8080-exec-669" daemon [_thread_blocked, id=10388, stack(0x000000002c3d0000,0x000000002c4d0000)]
      0x000000001a4f9800 JavaThread "http-apr-8080-exec-668" daemon [_thread_blocked, id=7176, stack(0x000000002c1e0000,0x000000002c2e0000)]
      0x000000001a4fb000 JavaThread "http-apr-8080-exec-667" daemon [_thread_blocked, id=10288, stack(0x000000002bd40000,0x000000002be40000)]
      0x000000001a4f7800 JavaThread "http-apr-8080-exec-666" daemon [_thread_blocked, id=7480, stack(0x000000002c050000,0x000000002c150000)]
      0x000000001a4fc000 JavaThread "http-apr-8080-exec-665" daemon [_thread_blocked, id=2744, stack(0x000000002be90000,0x000000002bf90000)]
      0x000000001a4fc800 JavaThread "http-apr-8080-exec-664" daemon [_thread_blocked, id=6640, stack(0x000000002bc30000,0x000000002bd30000)]
      0x000000001d735800 JavaThread "http-apr-8080-exec-663" daemon [_thread_blocked, id=5784, stack(0x000000002b7f0000,0x000000002b8f0000)]
      0x000000001d736000 JavaThread "http-apr-8080-exec-662" daemon [_thread_blocked, id=10984, stack(0x000000002baf0000,0x000000002bbf0000)]
      0x000000001d733000 JavaThread "http-apr-8080-exec-661" daemon [_thread_blocked, id=5592, stack(0x000000002b930000,0x000000002ba30000)]
      0x000000001d731800 JavaThread "http-apr-8080-exec-660" daemon [_thread_blocked, id=5832, stack(0x000000002b680000,0x000000002b780000)]
      0x000000001d737000 JavaThread "http-apr-8080-exec-659" daemon [_thread_blocked, id=15260, stack(0x000000002a5c0000,0x000000002a6c0000)]
      0x000000001d734000 JavaThread "http-apr-8080-exec-658" daemon [_thread_blocked, id=12784, stack(0x000000002b4e0000,0x000000002b5e0000)]
      0x000000001d734800 JavaThread "http-apr-8080-exec-657" daemon [_thread_blocked, id=2488, stack(0x000000002b280000,0x000000002b380000)]
      0x000000001d737800 JavaThread "http-apr-8080-exec-656" daemon [_thread_blocked, id=12492, stack(0x000000002b120000,0x000000002b220000)]
      0x000000001d738800 JavaThread "http-apr-8080-exec-655" daemon [_thread_blocked, id=11388, stack(0x000000002a090000,0x000000002a190000)]
      0x000000001ed5f000 JavaThread "http-apr-8080-exec-654" daemon [_thread_blocked, id=4904, stack(0x000000002afe0000,0x000000002b0e0000)]
      0x000000001ed5b000 JavaThread "http-apr-8080-exec-653" daemon [_thread_blocked, id=6436, stack(0x000000002aee0000,0x000000002afe0000)]
      0x000000001ed5e000 JavaThread "http-apr-8080-exec-652" daemon [_thread_blocked, id=11532, stack(0x000000002ad50000,0x000000002ae50000)]
      0x000000001ed5c800 JavaThread "http-apr-8080-exec-651" daemon [_thread_blocked, id=12584, stack(0x000000002ac10000,0x000000002ad10000)]
      0x000000001ed61000 JavaThread "http-apr-8080-exec-650" daemon [_thread_blocked, id=13068, stack(0x000000002aa50000,0x000000002ab50000)]
      0x000000001ed5f800 JavaThread "http-apr-8080-exec-649" daemon [_thread_blocked, id=2000, stack(0x000000002a850000,0x000000002a950000)]
      0x000000001ed5b800 JavaThread "http-apr-8080-exec-648" daemon [_thread_blocked, id=7348, stack(0x000000002a6f0000,0x000000002a7f0000)]
      0x000000001ed60800 JavaThread "http-apr-8080-exec-647" daemon [_thread_blocked, id=6920, stack(0x000000002a1a0000,0x000000002a2a0000)]
      0x000000001ed5a000 JavaThread "http-apr-8080-exec-646" daemon [_thread_blocked, id=9844, stack(0x0000000029520000,0x0000000029620000)]
      0x000000001a0eb800 JavaThread "http-apr-8080-exec-645" daemon [_thread_blocked, id=13168, stack(0x0000000029e30000,0x0000000029f30000)]
      0x000000001a0ef800 JavaThread "http-apr-8080-exec-644" daemon [_thread_blocked, id=1520, stack(0x0000000029f30000,0x000000002a030000)]
      0x000000001a0f1000 JavaThread "http-apr-8080-exec-643" daemon [_thread_blocked, id=1268, stack(0x0000000029ce0000,0x0000000029de0000)]
      0x000000001a0f1800 JavaThread "http-apr-8080-exec-642" daemon [_thread_blocked, id=2644, stack(0x0000000029ae0000,0x0000000029be0000)]
      0x000000001a0f0000 JavaThread "http-apr-8080-exec-641" daemon [_thread_blocked, id=8884, stack(0x0000000029950000,0x0000000029a50000)]
      0x000000001a0ee000 JavaThread "http-apr-8080-exec-640" daemon [_thread_blocked, id=10704, stack(0x0000000029760000,0x0000000029860000)]
      0x000000001a0ed000 JavaThread "http-apr-8080-exec-639" daemon [_thread_blocked, id=11100, stack(0x0000000028e60000,0x0000000028f60000)]
      0x000000001a0ee800 JavaThread "http-apr-8080-exec-625" daemon [_thread_blocked, id=7720, stack(0x0000000029620000,0x0000000029720000)]
      0x000000001a0ec000 JavaThread "http-apr-8080-exec-638" daemon [_thread_blocked, id=1264, stack(0x0000000029410000,0x0000000029510000)]
      0x000000001a5a3000 JavaThread "http-apr-8080-exec-637" daemon [_thread_blocked, id=13576, stack(0x0000000028620000,0x0000000028720000)]
      0x000000001a5a2800 JavaThread "http-apr-8080-exec-636" daemon [_thread_blocked, id=4856, stack(0x0000000029280000,0x0000000029380000)]
      0x000000001a5a6000 JavaThread "http-apr-8080-exec-635" daemon [_thread_blocked, id=4580, stack(0x0000000029140000,0x0000000029240000)]
      0x000000001a5a7800 JavaThread "http-apr-8080-exec-634" daemon [_thread_blocked, id=11844, stack(0x0000000028f90000,0x0000000029090000)]
      0x000000001a5a4800 JavaThread "http-apr-8080-exec-633" daemon [_thread_blocked, id=11368, stack(0x0000000028ba0000,0x0000000028ca0000)]
      0x000000001a5a7000 JavaThread "http-apr-8080-exec-632" daemon [_thread_blocked, id=8928, stack(0x0000000028d10000,0x0000000028e10000)]
      0x000000001a5a9000 JavaThread "http-apr-8080-exec-631" daemon [_thread_blocked, id=6272, stack(0x0000000028960000,0x0000000028a60000)]
      0x000000001a5a4000 JavaThread "http-apr-8080-exec-630" daemon [_thread_blocked, id=7332, stack(0x0000000028830000,0x0000000028930000)]
      0x000000001a5a8800 JavaThread "http-apr-8080-exec-629" daemon [_thread_blocked, id=7672, stack(0x0000000028720000,0x0000000028820000)]
      0x000000001f38b800 JavaThread "http-apr-8080-exec-628" daemon [_thread_blocked, id=14600, stack(0x0000000028500000,0x0000000028600000)]
      0x000000001f38d800 JavaThread "http-apr-8080-exec-627" daemon [_thread_blocked, id=8272, stack(0x0000000027f20000,0x0000000028020000)]
      0x000000001f38c000 JavaThread "http-apr-8080-exec-626" daemon [_thread_blocked, id=14688, stack(0x00000000283b0000,0x00000000284b0000)]
      0x000000001f38a800 JavaThread "http-apr-8080-exec-624" daemon [_thread_blocked, id=10456, stack(0x00000000281e0000,0x00000000282e0000)]
      0x000000001f38f000 JavaThread "http-apr-8080-exec-623" daemon [_thread_blocked, id=15012, stack(0x0000000028060000,0x0000000028160000)]
      0x000000001f389800 JavaThread "http-apr-8080-exec-622" daemon [_thread_blocked, id=9692, stack(0x0000000027900000,0x0000000027a00000)]
      0x000000001f38e800 JavaThread "http-apr-8080-exec-621" daemon [_thread_blocked, id=6484, stack(0x0000000026b30000,0x0000000026c30000)]
      0x000000001f389000 JavaThread "http-apr-8080-exec-620" daemon [_thread_blocked, id=5840, stack(0x0000000027710000,0x0000000027810000)]
      0x000000001f388000 JavaThread "http-apr-8080-exec-619" daemon [_thread_blocked, id=12740, stack(0x000000001d250000,0x000000001d350000)]
      0x000000001e0c4800 JavaThread "http-apr-8080-exec-613" daemon [_thread_blocked, id=13252, stack(0x0000000027280000,0x0000000027380000)]
      0x000000001e0be800 JavaThread "http-apr-8080-exec-618" daemon [_thread_blocked, id=14384, stack(0x0000000027570000,0x0000000027670000)]
      0x000000001e0b6800 JavaThread "http-apr-8080-exec-617" daemon [_thread_blocked, id=13464, stack(0x00000000273b0000,0x00000000274b0000)]
      0x000000001e0bf800 JavaThread "http-apr-8080-exec-616" daemon [_thread_blocked, id=8000, stack(0x0000000027130000,0x0000000027230000)]
      0x000000001e0c2800 JavaThread "http-apr-8080-exec-615" daemon [_thread_blocked, id=7684, stack(0x0000000026e40000,0x0000000026f40000)]
      0x000000001e0bb800 JavaThread "http-apr-8080-exec-614" daemon [_thread_blocked, id=4924, stack(0x0000000026c60000,0x0000000026d60000)]
      0x000000001e0c0000 JavaThread "http-apr-8080-exec-612" daemon [_thread_blocked, id=5340, stack(0x0000000026780000,0x0000000026880000)]
      0x000000001e0bc800 JavaThread "http-apr-8080-exec-611" daemon [_thread_blocked, id=10568, stack(0x0000000026910000,0x0000000026a10000)]
      0x000000001e0b8000 JavaThread "http-apr-8080-exec-610" daemon [_thread_blocked, id=512, stack(0x0000000026650000,0x0000000026750000)]
      0x000000001e0c4000 JavaThread "http-apr-8080-exec-609" daemon [_thread_blocked, id=3196, stack(0x0000000026550000,0x0000000026650000)]
      0x000000001e0b7000 JavaThread "http-apr-8080-exec-608" daemon [_thread_blocked, id=11280, stack(0x0000000026380000,0x0000000026480000)]
      0x000000001e0b9800 JavaThread "http-apr-8080-exec-607" daemon [_thread_blocked, id=9860, stack(0x0000000026030000,0x0000000026130000)]
      0x000000001e0bb000 JavaThread "http-apr-8080-exec-606" daemon [_thread_blocked, id=14944, stack(0x0000000026240000,0x0000000026340000)]
      0x000000001e0c1000 JavaThread "http-apr-8080-exec-605" daemon [_thread_blocked, id=6944, stack(0x0000000026140000,0x0000000026240000)]
      0x000000001e0bd000 JavaThread "http-apr-8080-exec-604" daemon [_thread_blocked, id=9876, stack(0x0000000025e70000,0x0000000025f70000)]
      0x000000001e0c3000 JavaThread "http-apr-8080-exec-603" daemon [_thread_blocked, id=13192, stack(0x0000000025cd0000,0x0000000025dd0000)]
      0x000000001e0c1800 JavaThread "http-apr-8080-exec-602" daemon [_thread_blocked, id=11868, stack(0x00000000251e0000,0x00000000252e0000)]
      0x000000001e0ba000 JavaThread "http-apr-8080-exec-601" daemon [_thread_blocked, id=7288, stack(0x0000000025af0000,0x0000000025bf0000)]
      0x000000001e0c5800 JavaThread "http-apr-8080-exec-600" daemon [_thread_blocked, id=8228, stack(0x0000000025960000,0x0000000025a60000)]
      0x000000001e2e1800 JavaThread "http-apr-8080-exec-599" daemon [_thread_blocked, id=14164, stack(0x0000000025760000,0x0000000025860000)]
      0x000000001e2ee000 JavaThread "http-apr-8080-exec-598" daemon [_thread_blocked, id=10924, stack(0x0000000025600000,0x0000000025700000)]
      0x000000001e2e8000 JavaThread "http-apr-8080-exec-597" daemon [_thread_blocked, id=14928, stack(0x00000000254d0000,0x00000000255d0000)]
      0x000000001e2e5800 JavaThread "http-apr-8080-exec-596" daemon [_thread_blocked, id=8364, stack(0x0000000025310000,0x0000000025410000)]
      0x000000001e2ee800 JavaThread "http-apr-8080-exec-595" daemon [_thread_blocked, id=7508, stack(0x0000000024f80000,0x0000000025080000)]
      0x000000001e2ec800 JavaThread "http-apr-8080-exec-594" daemon [_thread_blocked, id=11068, stack(0x00000000250c0000,0x00000000251c0000)]
      0x000000001e2e8800 JavaThread "http-apr-8080-exec-593" daemon [_thread_blocked, id=10552, stack(0x0000000024e10000,0x0000000024f10000)]
      0x000000001e2eb000 JavaThread "http-apr-8080-exec-592" daemon [_thread_blocked, id=14956, stack(0x0000000024a10000,0x0000000024b10000)]
      0x000000001e2e7000 JavaThread "http-apr-8080-exec-591" daemon [_thread_blocked, id=8128, stack(0x0000000024650000,0x0000000024750000)]
      0x000000001e2ea000 JavaThread "http-apr-8080-exec-590" daemon [_thread_blocked, id=7880, stack(0x0000000024890000,0x0000000024990000)]
      0x000000001e2eb800 JavaThread "http-apr-8080-exec-589" daemon [_thread_blocked, id=2904, stack(0x0000000024760000,0x0000000024860000)]
      0x000000001e2e4800 JavaThread "http-apr-8080-exec-588" daemon [_thread_blocked, id=13972, stack(0x000000001aaf0000,0x000000001abf0000)]
      0x000000001e2ed000 JavaThread "http-apr-8080-exec-587" daemon [_thread_blocked, id=9868, stack(0x0000000023370000,0x0000000023470000)]
      0x000000001e2ef800 JavaThread "http-apr-8080-exec-586" daemon [_thread_blocked, id=12524, stack(0x00000000244f0000,0x00000000245f0000)]
      0x000000001e2e3000 JavaThread "http-apr-8080-exec-585" daemon [_thread_blocked, id=4500, stack(0x0000000019260000,0x0000000019360000)]
      0x000000001e2e0000 JavaThread "http-apr-8080-exec-584" daemon [_thread_blocked, id=3644, stack(0x0000000021e00000,0x0000000021f00000)]
      0x000000001e2e6800 JavaThread "http-apr-8080-exec-583" daemon [_thread_blocked, id=11864, stack(0x0000000019c60000,0x0000000019d60000)]
      0x000000001e2e4000 JavaThread "http-apr-8080-exec-582" daemon [_thread_blocked, id=8668, stack(0x000000001d050000,0x000000001d150000)]
      0x000000001e2e1000 JavaThread "http-apr-8080-exec-581" daemon [_thread_blocked, id=6356, stack(0x000000001bdf0000,0x000000001bef0000)]
      0x000000001f38d000 JavaThread "http-apr-8080-exec-538" daemon [_thread_blocked, id=4396, stack(0x00000000310a0000,0x00000000311a0000)]
      0x000000001a0f2800 JavaThread "http-apr-8080-exec-531" daemon [_thread_blocked, id=6632, stack(0x00000000308e0000,0x00000000309e0000)]
      0x000000001a4f8000 JavaThread "http-apr-8080-exec-505" daemon [_thread_blocked, id=13804, stack(0x000000002e4a0000,0x000000002e5a0000)]
      0x000000001a4f6000 JavaThread "http-apr-8080-exec-502" daemon [_thread_blocked, id=5144, stack(0x000000002dca0000,0x000000002dda0000)]
      0x000000001ed5d000 JavaThread "http-apr-8080-exec-475" daemon [_thread_blocked, id=14108, stack(0x000000002b3a0000,0x000000002b4a0000)]
      0x00000000242e7000 JavaThread "http-apr-8080-exec-468" daemon [_thread_blocked, id=8840, stack(0x000000002a390000,0x000000002a490000)]
      0x00000000242e6000 JavaThread "http-apr-8080-exec-463" daemon [_thread_blocked, id=2764, stack(0x000000002a4c0000,0x000000002a5c0000)]
      0x000000001e2e9800 JavaThread "http-apr-8080-exec-441" daemon [_thread_blocked, id=11860, stack(0x0000000028a60000,0x0000000028b60000)]
      0x000000001e2e2800 JavaThread "http-apr-8080-exec-430" daemon [_thread_blocked, id=10620, stack(0x0000000026f60000,0x0000000027060000)]
      0x000000001e0b8800 JavaThread "http-apr-8080-exec-420" daemon [_thread_blocked, id=12060, stack(0x0000000026a30000,0x0000000026b30000)]
      0x000000001a1ea000 JavaThread "ajp-apr-8019-AsyncTimeout" daemon [_thread_blocked, id=12688, stack(0x000000001d150000,0x000000001d250000)]
      0x000000001a1e9000 JavaThread "ajp-apr-8019-Acceptor-0" daemon [_thread_in_native, id=10124, stack(0x000000001cee0000,0x000000001cfe0000)]
      0x000000001a1e8800 JavaThread "ajp-apr-8019-Poller" daemon [_thread_blocked, id=3552, stack(0x000000001cd30000,0x000000001ce30000)]
      0x000000001a1e7800 JavaThread "http-apr-8080-AsyncTimeout" daemon [_thread_blocked, id=14876, stack(0x000000001cb60000,0x000000001cc60000)]
      0x000000001a1e7000 JavaThread "http-apr-8080-Acceptor-0" daemon [_thread_in_native, id=12780, stack(0x000000001ca10000,0x000000001cb10000)]
      0x000000001a1e6000 JavaThread "http-apr-8080-Sendfile" daemon [_thread_blocked, id=11020, stack(0x000000001c8e0000,0x000000001c9e0000)]
    =>0x000000001a1e5800 JavaThread "http-apr-8080-Poller" daemon [_thread_in_native, id=11828, stack(0x000000001c420000,0x000000001c520000)]
      0x000000001a1e4000 JavaThread "ContainerBackgroundProcessor[StandardEngine[Catalina]]" daemon [_thread_blocked, id=4780, stack(0x000000001c540000,0x000000001c640000)]
      0x000000001a1e3000 JavaThread "GC Daemon" daemon [_thread_blocked, id=7360, stack(0x000000001c0b0000,0x000000001c1b0000)]
      0x000000001a1d7800 JavaThread "RMI Scheduler(0)" daemon [_thread_blocked, id=15236, stack(0x000000001a970000,0x000000001aa70000)]
      0x0000000018884800 JavaThread "RMI TCP Accept-0" daemon [_thread_in_native, id=11560, stack(0x00000000198c0000,0x00000000199c0000)]
      0x0000000018873800 JavaThread "RMI TCP Accept-1099" daemon [_thread_in_native, id=12684, stack(0x0000000019b30000,0x0000000019c30000)]
      0x000000001881f800 JavaThread "RMI TCP Accept-0" daemon [_thread_in_native, id=1868, stack(0x0000000019620000,0x0000000019720000)]
      0x00000000185fb800 JavaThread "Service Thread" daemon [_thread_blocked, id=13392, stack(0x0000000019510000,0x0000000019610000)]
      0x0000000018575800 JavaThread "C1 CompilerThread2" daemon [_thread_blocked, id=6968, stack(0x0000000019410000,0x0000000019510000)]
      0x0000000018570000 JavaThread "C2 CompilerThread1" daemon [_thread_blocked, id=6488, stack(0x0000000019030000,0x0000000019130000)]
      0x000000001856a800 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=7528, stack(0x0000000019160000,0x0000000019260000)]
      0x000000001855c000 JavaThread "JDWP Command Reader" daemon [_thread_in_native, id=13196, stack(0x0000000018ed0000,0x0000000018fd0000)]
      0x0000000018557800 JavaThread "JDWP Event Helper Thread" daemon [_thread_blocked, id=15308, stack(0x0000000018da0000,0x0000000018ea0000)]
      0x000000001714f000 JavaThread "JDWP Transport Listener: dt_socket" daemon [_thread_blocked, id=6536, stack(0x0000000018930000,0x0000000018a30000)]
      0x0000000017148800 JavaThread "Attach Listener" daemon [_thread_blocked, id=8788, stack(0x0000000017f40000,0x0000000018040000)]
      0x0000000017144000 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=6084, stack(0x0000000018a90000,0x0000000018b90000)]
      0x000000001712c000 JavaThread "Finalizer" daemon [_thread_blocked, id=2112, stack(0x0000000018430000,0x0000000018530000)]
      0x00000000170e4800 JavaThread "Reference Handler" daemon [_thread_blocked, id=14692, stack(0x0000000018290000,0x0000000018390000)]
      0x0000000001e9f000 JavaThread "main" [_thread_in_native, id=10256, stack(0x00000000026e0000,0x00000000027e0000)]
    
    Other Threads:
      0x00000000170dd000 VMThread [stack: 0x00000000180a0000,0x00000000181a0000] [id=13616]
      0x0000000018889800 WatcherThread [stack: 0x0000000019d80000,0x0000000019e80000] [id=9576]
    
    VM state:not at safepoint (normal execution)
    
    VM Mutex/Monitor currently owned by a thread: None
    
    Heap:
     PSYoungGen      total 583680K, used 465274K [0x00000000d8380000, 0x00000000ffc80000, 0x0000000100000000)
      eden space 518656K, 89% used [0x00000000d8380000,0x00000000f4896a70,0x00000000f7e00000)
      from space 65024K, 2% used [0x00000000f7e00000,0x00000000f7f48000,0x00000000fbd80000)
      to   space 62976K, 0% used [0x00000000fbf00000,0x00000000fbf00000,0x00000000ffc80000)
     ParOldGen       total 133632K, used 61052K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 45% used [0x0000000088a00000,0x000000008c59f140,0x0000000090c80000)
     Metaspace       used 33765K, capacity 34314K, committed 34816K, reserved 1079296K
      class space    used 3802K, capacity 4019K, committed 4096K, reserved 1048576K
    
    Card table byte_map: [0x0000000011ba0000,0x0000000011f60000] byte_map_base: 0x000000001175b000
    
    Marking Bits: (ParMarkBitMap*) 0x000000005325a6d0
     Begin Bits: [0x00000000127e0000, 0x00000000145b8000)
     End Bits:   [0x00000000145b8000, 0x0000000016390000)
    
    Polling page: 0x0000000001e70000
    
    CodeCache: size=245760Kb used=19834Kb max_used=21178Kb free=225925Kb
     bounds [0x00000000027e0000, 0x0000000003d10000, 0x00000000117e0000]
     total_blobs=5219 nmethods=4771 adapters=367
     compilation: enabled
    
    Compilation events (10 events):
    Event: 2108.116 Thread 0x0000000018575800 6307 % !   3       org.apache.tomcat.util.net.AprEndpoint$Poller::run @ 672 (2241 bytes)
    Event: 2108.137 Thread 0x0000000018575800 nmethod 6307% 0x0000000003cf3210 code [0x0000000003cf4080, 0x0000000003d00608]
    Event: 2110.648 Thread 0x0000000018570000 6308       4       java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject::addConditionWaiter (64 bytes)
    Event: 2110.650 Thread 0x0000000018570000 nmethod 6308 0x0000000003aa89d0 code [0x0000000003aa8b20, 0x0000000003aa8ca8]
    Event: 2121.424 Thread 0x000000001856a800 6309       4       java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject::doSignal (43 bytes)
    Event: 2121.442 Thread 0x000000001856a800 nmethod 6309 0x0000000003715010 code [0x0000000003715180, 0x00000000037154e8]
    Event: 2125.553 Thread 0x0000000018575800 6310       1       java.util.concurrent.ConcurrentHashMap::spread (10 bytes)
    Event: 2125.553 Thread 0x0000000018575800 nmethod 6310 0x0000000003aa8710 code [0x0000000003aa8860, 0x0000000003aa8950]
    Event: 2125.553 Thread 0x0000000018575800 6311       3       java.util.concurrent.ConcurrentHashMap::get (162 bytes)
    Event: 2125.555 Thread 0x0000000018575800 nmethod 6311 0x000000000362b2d0 code [0x000000000362b4e0, 0x000000000362c0b8]
    
    GC Heap History (10 events):
    Event: 61.902 GC heap before
    {Heap before GC invocations=18 (full 3):
     PSYoungGen      total 572928K, used 510464K [0x00000000d8380000, 0x00000000ffa00000, 0x0000000100000000)
      eden space 510464K, 100% used [0x00000000d8380000,0x00000000f7600000,0x00000000f7600000)
      from space 62464K, 0% used [0x00000000fbd00000,0x00000000fbd00000,0x00000000ffa00000)
      to   space 67584K, 0% used [0x00000000f7600000,0x00000000f7600000,0x00000000fb800000)
     ParOldGen       total 133632K, used 59508K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 44% used [0x0000000088a00000,0x000000008c41d130,0x0000000090c80000)
     Metaspace       used 33530K, capacity 34122K, committed 34560K, reserved 1079296K
      class space    used 3794K, capacity 4019K, committed 4096K, reserved 1048576K
    Event: 61.952 GC heap after
    Heap after GC invocations=18 (full 3):
     PSYoungGen      total 578048K, used 1568K [0x00000000d8380000, 0x0000000100000000, 0x0000000100000000)
      eden space 510464K, 0% used [0x00000000d8380000,0x00000000d8380000,0x00000000f7600000)
      from space 67584K, 2% used [0x00000000f7600000,0x00000000f7788000,0x00000000fb800000)
      to   space 67584K, 0% used [0x00000000fbe00000,0x00000000fbe00000,0x0000000100000000)
     ParOldGen       total 133632K, used 59516K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 44% used [0x0000000088a00000,0x000000008c41f130,0x0000000090c80000)
     Metaspace       used 33530K, capacity 34122K, committed 34560K, reserved 1079296K
      class space    used 3794K, capacity 4019K, committed 4096K, reserved 1048576K
    }
    Event: 90.777 GC heap before
    {Heap before GC invocations=19 (full 3):
     PSYoungGen      total 578048K, used 512032K [0x00000000d8380000, 0x0000000100000000, 0x0000000100000000)
      eden space 510464K, 100% used [0x00000000d8380000,0x00000000f7600000,0x00000000f7600000)
      from space 67584K, 2% used [0x00000000f7600000,0x00000000f7788000,0x00000000fb800000)
      to   space 67584K, 0% used [0x00000000fbe00000,0x00000000fbe00000,0x0000000100000000)
     ParOldGen       total 133632K, used 59516K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 44% used [0x0000000088a00000,0x000000008c41f130,0x0000000090c80000)
     Metaspace       used 33562K, capacity 34122K, committed 34560K, reserved 1079296K
      class space    used 3799K, capacity 4019K, committed 4096K, reserved 1048576K
    Event: 90.819 GC heap after
    Heap after GC invocations=19 (full 3):
     PSYoungGen      total 583168K, used 640K [0x00000000d8380000, 0x00000000fff00000, 0x0000000100000000)
      eden space 516608K, 0% used [0x00000000d8380000,0x00000000d8380000,0x00000000f7c00000)
      from space 66560K, 0% used [0x00000000fbe00000,0x00000000fbea0000,0x00000000fff00000)
      to   space 67072K, 0% used [0x00000000f7c00000,0x00000000f7c00000,0x00000000fbd80000)
     ParOldGen       total 133632K, used 60396K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 45% used [0x0000000088a00000,0x000000008c4fb140,0x0000000090c80000)
     Metaspace       used 33562K, capacity 34122K, committed 34560K, reserved 1079296K
      class space    used 3799K, capacity 4019K, committed 4096K, reserved 1048576K
    }
    Event: 269.712 GC heap before
    {Heap before GC invocations=20 (full 3):
     PSYoungGen      total 583168K, used 517248K [0x00000000d8380000, 0x00000000fff00000, 0x0000000100000000)
      eden space 516608K, 100% used [0x00000000d8380000,0x00000000f7c00000,0x00000000f7c00000)
      from space 66560K, 0% used [0x00000000fbe00000,0x00000000fbea0000,0x00000000fff00000)
      to   space 67072K, 0% used [0x00000000f7c00000,0x00000000f7c00000,0x00000000fbd80000)
     ParOldGen       total 133632K, used 60396K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 45% used [0x0000000088a00000,0x000000008c4fb140,0x0000000090c80000)
     Metaspace       used 33593K, capacity 34122K, committed 34560K, reserved 1079296K
      class space    used 3799K, capacity 4019K, committed 4096K, reserved 1048576K
    Event: 270.048 GC heap after
    Heap after GC invocations=20 (full 3):
     PSYoungGen      total 583680K, used 1088K [0x00000000d8380000, 0x00000000ffe80000, 0x0000000100000000)
      eden space 516608K, 0% used [0x00000000d8380000,0x00000000d8380000,0x00000000f7c00000)
      from space 67072K, 1% used [0x00000000f7c00000,0x00000000f7d10000,0x00000000fbd80000)
      to   space 66048K, 0% used [0x00000000fbe00000,0x00000000fbe00000,0x00000000ffe80000)
     ParOldGen       total 133632K, used 60452K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 45% used [0x0000000088a00000,0x000000008c509140,0x0000000090c80000)
     Metaspace       used 33593K, capacity 34122K, committed 34560K, reserved 1079296K
      class space    used 3799K, capacity 4019K, committed 4096K, reserved 1048576K
    }
    Event: 315.217 GC heap before
    {Heap before GC invocations=21 (full 3):
     PSYoungGen      total 583680K, used 517696K [0x00000000d8380000, 0x00000000ffe80000, 0x0000000100000000)
      eden space 516608K, 100% used [0x00000000d8380000,0x00000000f7c00000,0x00000000f7c00000)
      from space 67072K, 1% used [0x00000000f7c00000,0x00000000f7d10000,0x00000000fbd80000)
      to   space 66048K, 0% used [0x00000000fbe00000,0x00000000fbe00000,0x00000000ffe80000)
     ParOldGen       total 133632K, used 60452K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 45% used [0x0000000088a00000,0x000000008c509140,0x0000000090c80000)
     Metaspace       used 33601K, capacity 34122K, committed 34560K, reserved 1079296K
      class space    used 3801K, capacity 4019K, committed 4096K, reserved 1048576K
    Event: 315.239 GC heap after
    Heap after GC invocations=21 (full 3):
     PSYoungGen      total 583168K, used 384K [0x00000000d8380000, 0x00000000ffd00000, 0x0000000100000000)
      eden space 518656K, 0% used [0x00000000d8380000,0x00000000d8380000,0x00000000f7e00000)
      from space 64512K, 0% used [0x00000000fbe00000,0x00000000fbe60000,0x00000000ffd00000)
      to   space 65024K, 0% used [0x00000000f7e00000,0x00000000f7e00000,0x00000000fbd80000)
     ParOldGen       total 133632K, used 61028K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 45% used [0x0000000088a00000,0x000000008c599140,0x0000000090c80000)
     Metaspace       used 33601K, capacity 34122K, committed 34560K, reserved 1079296K
      class space    used 3801K, capacity 4019K, committed 4096K, reserved 1048576K
    }
    Event: 1570.093 GC heap before
    {Heap before GC invocations=22 (full 3):
     PSYoungGen      total 583168K, used 519040K [0x00000000d8380000, 0x00000000ffd00000, 0x0000000100000000)
      eden space 518656K, 100% used [0x00000000d8380000,0x00000000f7e00000,0x00000000f7e00000)
      from space 64512K, 0% used [0x00000000fbe00000,0x00000000fbe60000,0x00000000ffd00000)
      to   space 65024K, 0% used [0x00000000f7e00000,0x00000000f7e00000,0x00000000fbd80000)
     ParOldGen       total 133632K, used 61028K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 45% used [0x0000000088a00000,0x000000008c599140,0x0000000090c80000)
     Metaspace       used 33675K, capacity 34250K, committed 34560K, reserved 1079296K
      class space    used 3801K, capacity 4019K, committed 4096K, reserved 1048576K
    Event: 1570.102 GC heap after
    Heap after GC invocations=22 (full 3):
     PSYoungGen      total 583680K, used 1312K [0x00000000d8380000, 0x00000000ffc80000, 0x0000000100000000)
      eden space 518656K, 0% used [0x00000000d8380000,0x00000000d8380000,0x00000000f7e00000)
      from space 65024K, 2% used [0x00000000f7e00000,0x00000000f7f48000,0x00000000fbd80000)
      to   space 62976K, 0% used [0x00000000fbf00000,0x00000000fbf00000,0x00000000ffc80000)
     ParOldGen       total 133632K, used 61052K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 45% used [0x0000000088a00000,0x000000008c59f140,0x0000000090c80000)
     Metaspace       used 33675K, capacity 34250K, committed 34560K, reserved 1079296K
      class space    used 3801K, capacity 4019K, committed 4096K, reserved 1048576K
    }
    
    Deoptimization events (10 events):
    Event: 388.516 Thread 0x0000000021649800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x000000000331c6d4 method=java.util.concurrent.ThreadPoolExecutor.getTask()Ljava/lang/Runnable; @ 113
    Event: 901.816 Thread 0x000000001a1d7800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x0000000002b4bdf4 method=java.util.HashMap$HashIterator.<init>(Ljava/util/HashMap;)V @ 45
    Event: 901.816 Thread 0x000000001a1d7800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x000000000366e29c method=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.addConditionWaiter()Ljava/util/concurrent/locks/AbstractQueuedSynchronizer$Node; @ 15
    Event: 2104.757 Thread 0x000000001a1e5800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x0000000003ce559c method=org.apache.tomcat.util.net.AprEndpoint$Poller.run()V @ 1362
    Event: 2104.992 Thread 0x000000001a1e5800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x0000000003a18e3c method=org.apache.tomcat.util.net.AprEndpoint$Poller.addToPoller(JI)Z @ 23
    Event: 2105.392 Thread 0x000000001a1e5800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x00000000038ef390 method=java.lang.Throwable.<init>()V @ 24
    Event: 2105.399 Thread 0x000000001a1e5800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x00000000038ef390 method=java.lang.Throwable.<init>()V @ 24
    Event: 2105.421 Thread 0x000000001a1e5800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x00000000038ef390 method=java.lang.Throwable.<init>()V @ 24
    Event: 2125.243 Thread 0x000000001a1e7000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00000000032866a0 method=java.util.concurrent.ConcurrentHashMap.putVal(Ljava/lang/Object;Ljava/lang/Object;Z)Ljava/lang/Object; @ 331
    Event: 2125.553 Thread 0x000000001a1e5800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x000000000329a180 method=java.util.concurrent.ConcurrentHashMap.get(Ljava/lang/Object;)Ljava/lang/Object; @ 86
    
    Internal exceptions (10 events):
    Event: 2121.066 Thread 0x000000001a1e5800 Implicit null exception at 0x00000000039f91ac to 0x00000000039f91bb
    Event: 2121.066 Thread 0x000000001a1e5800 Exception <a 'java/lang/NullPointerException'> (0x00000000f4811098) thrown at [C:
    eworkspace8-2-build-windows-amd64-cygwinjdk8u1017261hotspotsrcsharevm
    untimesharedRuntime.cpp, line 605]
    Event: 2121.069 Thread 0x000000001a1e5800 Implicit null exception at 0x00000000039f91ac to 0x00000000039f91bb
    Event: 2121.070 Thread 0x000000001a1e5800 Exception <a 'java/lang/NullPointerException'> (0x00000000f481c240) thrown at [C:
    eworkspace8-2-build-windows-amd64-cygwinjdk8u1017261hotspotsrcsharevm
    untimesharedRuntime.cpp, line 605]
    Event: 2121.100 Thread 0x000000001a1e5800 Implicit null exception at 0x00000000039f91ac to 0x00000000039f91bb
    Event: 2121.101 Thread 0x000000001a1e5800 Exception <a 'java/lang/NullPointerException'> (0x00000000f4827648) thrown at [C:
    eworkspace8-2-build-windows-amd64-cygwinjdk8u1017261hotspotsrcsharevm
    untimesharedRuntime.cpp, line 605]
    Event: 2121.104 Thread 0x000000001a1e5800 Implicit null exception at 0x00000000039f91ac to 0x00000000039f91bb
    Event: 2121.104 Thread 0x000000001a1e5800 Exception <a 'java/lang/NullPointerException'> (0x00000000f4832818) thrown at [C:
    eworkspace8-2-build-windows-amd64-cygwinjdk8u1017261hotspotsrcsharevm
    untimesharedRuntime.cpp, line 605]
    Event: 2121.108 Thread 0x000000001a1e5800 Implicit null exception at 0x00000000039f91ac to 0x00000000039f91bb
    Event: 2121.108 Thread 0x000000001a1e5800 Exception <a 'java/lang/NullPointerException'> (0x00000000f483d9a8) thrown at [C:
    eworkspace8-2-build-windows-amd64-cygwinjdk8u1017261hotspotsrcsharevm
    untimesharedRuntime.cpp, line 605]
    
    Events (10 events):
    Event: 2125.243 Thread 0x000000001a1e7000 Uncommon trap: trap_request=0xffffff65 fr.pc=0x00000000032866a0
    Event: 2125.243 Thread 0x000000001a1e7000 DEOPT PACKING pc=0x00000000032866a0 sp=0x000000001cb0ec60
    Event: 2125.243 Thread 0x000000001a1e7000 DEOPT UNPACKING pc=0x000000000282582a sp=0x000000001cb0ec08 mode 2
    Event: 2125.243 loading class java/util/concurrent/ConcurrentHashMap$TreeNode
    Event: 2125.244 loading class java/util/concurrent/ConcurrentHashMap$TreeNode done
    Event: 2125.244 loading class java/util/concurrent/ConcurrentHashMap$TreeBin
    Event: 2125.244 loading class java/util/concurrent/ConcurrentHashMap$TreeBin done
    Event: 2125.552 Thread 0x000000001a1e5800 Uncommon trap: trap_request=0xffffff65 fr.pc=0x000000000329a180
    Event: 2125.553 Thread 0x000000001a1e5800 DEOPT PACKING pc=0x000000000329a180 sp=0x000000001c51f380
    Event: 2125.553 Thread 0x000000001a1e5800 DEOPT UNPACKING pc=0x000000000282582a sp=0x000000001c51f320 mode 2
    
    
    Dynamic libraries:
    0x000000013f910000 - 0x000000013f947000     D:Program FilesJavajdk1.8.0_101injava.exe
    0x00000000774a0000 - 0x000000007764a000     C:WindowsSYSTEM32
    tdll.dll
    0x0000000077280000 - 0x000000007739f000     C:Windowssystem32kernel32.dll
    0x000007fefd2e0000 - 0x000007fefd34a000     C:Windowssystem32KERNELBASE.dll
    0x000007feff6d0000 - 0x000007feff7ab000     C:Windowssystem32ADVAPI32.dll
    0x000007fefeaf0000 - 0x000007fefeb8f000     C:Windowssystem32msvcrt.dll
    0x000007fefd6e0000 - 0x000007fefd6ff000     C:WindowsSYSTEM32sechost.dll
    0x000007feff1e0000 - 0x000007feff30d000     C:Windowssystem32RPCRT4.dll
    0x00000000773a0000 - 0x000000007749a000     C:Windowssystem32USER32.dll
    0x000007fefed20000 - 0x000007fefed87000     C:Windowssystem32GDI32.dll
    0x000007fefed90000 - 0x000007fefed9e000     C:Windowssystem32LPK.dll
    0x000007feff110000 - 0x000007feff1da000     C:Windowssystem32USP10.dll
    0x000007fefbc40000 - 0x000007fefbe34000     C:WindowsWinSxSamd64_microsoft.windows.common-controls_6595b64144ccf1df_6.0.7601.18837_none_fa3b1e3d17594757COMCTL32.dll
    0x000007feff390000 - 0x000007feff401000     C:Windowssystem32SHLWAPI.dll
    0x000007fefda70000 - 0x000007fefda9e000     C:Windowssystem32IMM32.DLL
    0x000007fefd700000 - 0x000007fefd809000     C:Windowssystem32MSCTF.dll
    0x000007fefd1a0000 - 0x000007fefd1d0000     C:Windowssystem32
    vinitx.dll
    0x000007fefd190000 - 0x000007fefd19c000     C:Windowssystem32VERSION.dll
    0x00000000744c0000 - 0x00000000744c6000     C:Program FilesNVIDIA CorporationCoProcManagerdetoured.dll
    0x000007fefb190000 - 0x000007fefb1c8000     C:Program FilesNVIDIA CorporationCoProcManager
    vd3d9wrapx.dll
    0x000007feff4f0000 - 0x000007feff6c7000     C:Windowssystem32SETUPAPI.dll
    0x000007fefd2a0000 - 0x000007fefd2d6000     C:Windowssystem32CFGMGR32.dll
    0x000007feff410000 - 0x000007feff4ea000     C:Windowssystem32OLEAUT32.dll
    0x000007fefe880000 - 0x000007fefea7c000     C:Windowssystem32ole32.dll
    0x000007fefd480000 - 0x000007fefd49a000     C:Windowssystem32DEVOBJ.dll
    0x000007fefb060000 - 0x000007fefb084000     C:Program FilesNVIDIA CorporationCoProcManager
    vdxgiwrapx.dll
    0x000000006a3a0000 - 0x000000006a472000     D:Program FilesJavajdk1.8.0_101jreinmsvcr100.dll
    0x0000000052a40000 - 0x00000000532da000     D:Program FilesJavajdk1.8.0_101jreinserverjvm.dll
    0x000007fefb0f0000 - 0x000007fefb0f9000     C:Windowssystem32WSOCK32.dll
    0x000007fefe830000 - 0x000007fefe87d000     C:Windowssystem32WS2_32.dll
    0x000007fefeae0000 - 0x000007fefeae8000     C:Windowssystem32NSI.dll
    0x000007fefadf0000 - 0x000007fefae2b000     C:Windowssystem32WINMM.dll
    0x0000000077670000 - 0x0000000077677000     C:Windowssystem32PSAPI.DLL
    0x000000006c230000 - 0x000000006c23f000     D:Program FilesJavajdk1.8.0_101jreinverify.dll
    0x000000006c0c0000 - 0x000000006c0e9000     D:Program FilesJavajdk1.8.0_101jreinjava.dll
    0x000000006ba30000 - 0x000000006ba65000     D:Program FilesJavajdk1.8.0_101jreinjdwp.dll
    0x000000006c030000 - 0x000000006c038000     D:Program FilesJavajdk1.8.0_101jrein
    pt.dll
    0x000000006c1e0000 - 0x000000006c1f6000     D:Program FilesJavajdk1.8.0_101jreinzip.dll
    0x000007fefdaa0000 - 0x000007fefe82a000     C:Windowssystem32SHELL32.dll
    0x000007fefd270000 - 0x000007fefd27f000     C:Windowssystem32profapi.dll
    0x000000006ba20000 - 0x000000006ba29000     D:Program FilesJavajdk1.8.0_101jreindt_socket.dll
    0x000007fefc9f0000 - 0x000007fefca45000     C:Windowssystem32mswsock.dll
    0x000007fefc3c0000 - 0x000007fefc3c7000     C:WindowsSystem32wshtcpip.dll
    0x0000000073120000 - 0x000000007312d000     D:Program FilesJavajdk1.8.0_101jreinmanagement.dll
    0x000000006c1c0000 - 0x000000006c1da000     D:Program FilesJavajdk1.8.0_101jrein
    et.dll
    0x000007fefc9e0000 - 0x000007fefc9e7000     C:WindowsSystem32wship6.dll
    0x000000006be10000 - 0x000000006be21000     D:Program FilesJavajdk1.8.0_101jrein
    io.dll
    0x000007fefac10000 - 0x000007fefac25000     C:Windowssystem32NLAapi.dll
    0x000007feeed90000 - 0x000007feeeda5000     C:Windowssystem32
    apinsp.dll
    0x000007feeed70000 - 0x000007feeed89000     C:Windowssystem32pnrpnsp.dll
    0x000007fefc870000 - 0x000007fefc8cb000     C:Windowssystem32DNSAPI.dll
    0x000007feeed60000 - 0x000007feeed6b000     C:WindowsSystem32winrnr.dll
    0x000007feeed50000 - 0x000007feeed60000     C:Windowssystem32wshbth.dll
    0x000007fefa030000 - 0x000007fefa057000     C:Windowssystem32IPHLPAPI.DLL
    0x000007fefa020000 - 0x000007fefa02b000     C:Windowssystem32WINNSI.DLL
    0x000007fef4580000 - 0x000007fef4588000     C:Windowssystem32
    asadhlp.dll
    0x000007fef9c30000 - 0x000007fef9c83000     C:WindowsSystem32fwpuclnt.dll
    0x000007fefca50000 - 0x000007fefca68000     C:Windowssystem32CRYPTSP.dll
    0x000007fefc750000 - 0x000007fefc797000     C:Windowssystem32
    saenh.dll
    0x0000000002440000 - 0x000000000245e000     C:Windowssystem32USERENV.dll
    0x000007fefd0c0000 - 0x000007fefd0cf000     C:Windowssystem32CRYPTBASE.dll
    0x000007fef9c00000 - 0x000007fef9c18000     C:Windowssystem32dhcpcsvc.DLL
    0x000007fef9be0000 - 0x000007fef9bf1000     C:Windowssystem32dhcpcsvc6.DLL
    0x0000000180000000 - 0x000000018020a000     D:servers	omcat-7.0.70in	cnative-1.dll
    0x0000000069dd0000 - 0x0000000069df4000     D:Program FilesJavajdk1.8.0_101jreinsunec.dll
    0x000000006b9a0000 - 0x000000006b9ab000     D:Program FilesJavajdk1.8.0_101jreinsunmscapi.dll
    0x000007fefd4b0000 - 0x000007fefd61d000     C:Windowssystem32CRYPT32.dll
    0x000007fefd260000 - 0x000007fefd26f000     C:Windowssystem32MSASN1.dll
    0x000007fee5320000 - 0x000007fee5445000     C:Windowssystem32dbghelp.dll
    
    VM Arguments:
    jvm_args: -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:58835,suspend=y,server=n -Dcom.sun.management.jmxremote= -Dcom.sun.management.jmxremote.port=1099 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Djava.rmi.server.hostname=127.0.0.1 -Djdk.tls.ephemeralDHKeySize=2048 -Djava.util.logging.config.file=C:Usersxxx.IntelliJIdea2016.1system	omcatUnnamed_mvn_test_localconflogging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=D:servers	omcat-7.0.70endorsed -Dcatalina.base=C:Usersxxx.IntelliJIdea2016.1system	omcatUnnamed_mvn_test_local -Dcatalina.home=D:servers	omcat-7.0.70 -Djava.io.tmpdir=D:servers	omcat-7.0.70	emp 
    java_command: org.apache.catalina.startup.Bootstrap start
    java_class_path (initial): D:servers	omcat-7.0.70inootstrap.jar;D:servers	omcat-7.0.70in	omcat-juli.jar
    Launcher Type: SUN_STANDARD
    
    Environment Variables:
    JAVA_HOME=D:Program FilesJavajdk1.8.0_101
    JRE_HOME=D:Program FilesJavajdk1.8.0_101
    CLASSPATH=D:servers	omcat-7.0.70inootstrap.jar;D:servers	omcat-7.0.70in	omcat-juli.jar
    PATH=C:ProgramDataOracleJavajavapath;C:Windowssystem32;C:Windows;C:WindowsSystem32Wbem;C:WindowsSystem32WindowsPowerShellv1.0;C:Program Files (x86)NVIDIA CorporationPhysXCommon;D:Program FilesGitcmd;D:Program FilesTortoiseGitin;D:Program FilesJavajdk1.8.0_101in;D:Program FilesJavajdk1.8.0_101jrein;C:Program FilesTortoiseSVNin;D:Program Filesapache-maven-3.3.9in;D:xamppphp;C:ProgramDataComposerSetupin;D:Program Files
    odejs;D:Program Files
    odejs;D:Program Filesgradle-4.1in;C:UsersxxxAppDataLocalProgramsPythonPython36Scripts;C:UsersxxxAppDataLocalProgramsPythonPython36;C:UsersxxxAppDataRoamingComposervendorin;C:UsersxxxAppDataRoaming
    pm;D:Program Files
    odejs;C:Program Files (x86)OpenVPNin;C:UsersxxxAppDataLocalProgramsFiddler;C:UsersxxxAppDataLocalProgramsPythonPython36;
    USERNAME=xxx
    OS=Windows_NT
    PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 142 Stepping 9, GenuineIntel
    
    
    
    ---------------  S Y S T E M  ---------------
    
    OS: Windows 7 , 64 bit Build 7601 (6.1.7601.23714)
    
    CPU:total 4 (2 cores per cpu, 2 threads per core) family 6 model 142 stepping 9, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, erms, 3dnowpref, lzcnt, ht, tsc, tscinvbit, bmi1, bmi2, adx
    
    Memory: 4k page, physical 7822340k(3261452k free), swap 15642820k(8502936k free)
    
    vm_info: Java HotSpot(TM) 64-Bit Server VM (25.101-b13) for windows-amd64 JRE (1.8.0_101-b13), built on Jun 22 2016 01:21:29 by "java_re" with MS VC++ 10.0 (VS2010)
    
    time: Wed Dec 12 11:54:30 2018
    elapsed time: 2125 seconds (0d 0h 35m 25s)
    View Code

      降到2800,也依然jvm挂了!

       降到2600,jvm没挂,但是错误量较多,由于错误导致并发只跑到12000就未能继续进行了,数据不准:

       改了下失败策略后,2600,还是挂了,重启后可以扛住压力!

      压到4840个请求就挂了!

      2500并发,挂!

      2300并发,挂!

      2200并发,操作系统提示jre没有响应,被迫关闭java进程!

      2000并发,挂!

      1900并发,扛住了!cpu在80左右跳动!内存不变!

      看来,1900还行,极限就2000吧!

      好了,tomcat7看来是没辙了!

      换tomcat8 的 nio 看下效果!

    前提2:

      tomcat8, nio 模式运行!

     NIO介绍如下:

    Java NIO: Channels and Buffers(通道和缓冲区)

    标准的IO基于字节流和字符流进行操作的,而NIO是基于通道(Channel)和缓冲区(Buffer)进行操作,数据总是从通道读取到缓冲区中,或者从缓冲区写入到通道中。

    Java NIO: Non-blocking IO(非阻塞IO)

    Java NIO可以让你非阻塞的使用IO,例如:当线程从通道读取数据到缓冲区时,线程还是可以进行其他事情。当数据被写入到缓冲区时,线程可以继续处理它。从缓冲区写入通道也类似。

    Java NIO: Selectors(选择器)

    Java NIO引入了选择器的概念,选择器用于监听多个通道的事件(比如:连接打开,数据到达)。因此,单个的线程可以监听多个数据通道。

    直接从2000并发开测!

    并发2000-5组连续请求!

      server ok, 但是,仍然存在数错误,有几个并发请求卡死! TPS: 62.9, 平均响应: 14.7s。

      server 的cpu,内存情况如下:

      server ok, TPS: 77.2, error: 55%, 平均响应时间: 25s。只能说,服务端没死,但是基本已经不怎么可用了!

      到最后,我压到 6000 的并发时,server 仍然没有挂掉!

      所以,nio,是比较强悍的!

  • 相关阅读:
    CSS3实现0.5px的边框
    解决nvm导致终端启动慢的问题
    解决zsh: command not found: gitk,将git路径/usr/bin/git修改为/usr/local/bin/git
    MAC之zsh终端使用nvm安装指定版本node
    解决npm包node-sass下载失败问题
    forEach中使用async await的问题
    使用module-alias别名简化CommonJS的require路径
    关系型数据库的ACID
    Node.js ORM 框架对比
    mysql字符集 utf8 和utf8mb4 的区别
  • 原文地址:https://www.cnblogs.com/yougewe/p/10118681.html
Copyright © 2020-2023  润新知