什么是JNI内存泄露,基本的避免方法 :
http://www.ibm.com/developerworks/cn/java/j-lo-jnileak/
最近的课题中需要用到Spark,同组同学负责的算法用c++写的,数据库读取及调用算法用的是java,因此在整合的时候用到了jni。
因为任务需要并行,因此想到了用多线程实现。
程序在单机单线程的情况下运行没问题,当开两个线程的时候就出现了如下错误:
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x7c930cce, pid=6224, tid=736
#
# JRE version: Java(TM) SE Runtime Environment (8.0_11-b12) (build 1.8.0_11-b12)
# Java VM: Java HotSpot(TM) Client VM (25.11-b03 mixed mode windows-x86 )
# Problematic frame:
# [thread 4240 also had an error]
C [ntdll.dll+0x10cce]
#
# Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# E:eclipse4EEworkspaceMongoLochs_err_pid6224.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.sun.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
java虚拟机崩溃。
JVM错误日志文件如下:
# # A fatal error has been detected by the Java Runtime Environment: # # EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000e, pid=6240, tid=1404 # # JRE version: Java(TM) SE Runtime Environment (8.0_11-b12) (build 1.8.0_11-b12) # Java VM: Java HotSpot(TM) Client VM (25.11-b03 mixed mode windows-x86 ) # Problematic frame: # C 0x0000000e # # Failed to write core dump. Minidumps are not enabled by default on client versions of Windows # # If you would like to submit a bug report, please visit: # http://bugreport.sun.com/bugreport/crash.jsp # The crash happened outside the Java Virtual Machine in native code. # See problematic frame for where to report the bug. # --------------- T H R E A D --------------- Current thread (0x0b1f3400): JavaThread "Thread-0" [_thread_in_native, id=1404, stack(0x0b5d0000,0x0b620000)] siginfo: ExceptionCode=0xc0000005, reading address 0x0000000e Registers: EAX=0x0000000e, EBX=0x003e8748, ECX=0x474e5543, EDX=0x432b2b00 ESP=0x0b61f4ec, EBP=0x0b61f548, ESI=0x0b61f528, EDI=0x0b1f3400 EIP=0x0000000e, EFLAGS=0x00010202 Top of Stack: (sp=0x0b61f4ec) 0x0b61f4ec: 66396350 00000001 00000001 432b2b00 0x0b61f4fc: 474e5543 003e8748 0b61f528 ffffffff 0x0b61f50c: 7c9301db 77bfc3c9 00000064 003e8768 0x0b61f51c: 0b66f6cc 0b1f3400 0b61f534 0bc450b8 0x0b61f52c: 0b66f80c 0bd486a0 66383940 664055d0 0x0b61f53c: 003e8748 0bc450b8 0b1f3400 0b61f568 0x0b61f54c: 6638588e 003e8748 00000000 00000000 0x0b61f55c: 77bfc42e 00000028 0bc450b8 0b61f588 Instructions: (pc=0x0000000e) 0xffffffee: Register to memory mapping: EAX=0x0000000e is an unknown value EBX=0x003e8748 is an unknown value ECX=0x474e5543 is an unknown value EDX=0x432b2b00 is an unknown value ESP=0x0b61f4ec is pointing into the stack for thread: 0x0b1f3400 EBP=0x0b61f548 is pointing into the stack for thread: 0x0b1f3400 ESI=0x0b61f528 is pointing into the stack for thread: 0x0b1f3400 EDI=0x0b1f3400 is a thread Stack: [0x0b5d0000,0x0b620000], sp=0x0b61f4ec, free space=317k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) C 0x0000000e C [SpeedEstimate.dll+0x588e] C [SpeedEstimate.dll+0x4824] C [SpeedEstimate.dll+0x261c8] C [SpeedEstimate.dll+0x58ca6] C [SpeedEstimate.dll+0x58dbc] C [SpeedEstimate.dll+0x7468f] C [SpeedEstimate.dll+0x228d0] C [SpeedEstimate.dll+0x1b8c] C [SpeedEstimate.dll+0x20b24] C [SpeedEstimate.dll+0x2aa0] C [SpeedEstimate.dll+0x2bf5] j tong.mongo.loction.LinkedC.StreetEstimate(Ltong/mongo/defclass/MapLoc;Ltong/mongo/defclass/Car;)Ltong/mongo/defclass/StreetEstimateOutput;+0 j tong.mongo.loction.MdbThread2.run()V+1207 j java.lang.Thread.run()V+11 v ~StubRoutines::call_stub V [jvm.dll+0x142295] V [jvm.dll+0x2073ae] V [jvm.dll+0x14232e] V [jvm.dll+0x1424b6] V [jvm.dll+0x142527] V [jvm.dll+0xedebf] V [jvm.dll+0x1646ac] V [jvm.dll+0x164f3a] V [jvm.dll+0x1a9316] C [msvcr100.dll+0x5c556] C [msvcr100.dll+0x5c600] C [kernel32.dll+0xb729] Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) j tong.mongo.loction.LinkedC.StreetEstimate(Ltong/mongo/defclass/MapLoc;Ltong/mongo/defclass/Car;)Ltong/mongo/defclass/StreetEstimateOutput;+0 j tong.mongo.loction.MdbThread2.run()V+1207 j java.lang.Thread.run()V+11 v ~StubRoutines::call_stub --------------- P R O C E S S --------------- Java Threads: ( => current thread ) 0x0b6c1800 JavaThread "MongoCleaner1963207" daemon [_thread_blocked, id=6808, stack(0x0bb80000,0x0bbd0000)] 0x0b6b4800 JavaThread "MongoCleaner19725142" daemon [_thread_blocked, id=7236, stack(0x0bb30000,0x0bb80000)] 0x0b6ac400 JavaThread "cluster-2-127.0.0.1:27017" daemon [_thread_blocked, id=8120, stack(0x0bae0000,0x0bb30000)] 0x0b6ab800 JavaThread "cluster-1-127.0.0.1:27017" daemon [_thread_blocked, id=7252, stack(0x0ba90000,0x0bae0000)] 0x0b1f4800 JavaThread "Thread-1" [_thread_in_native, id=3284, stack(0x0b620000,0x0b670000)] =>0x0b1f3400 JavaThread "Thread-0" [_thread_in_native, id=1404, stack(0x0b5d0000,0x0b620000)] 0x0b1a8800 JavaThread "Service Thread" daemon [_thread_blocked, id=7368, stack(0x0b440000,0x0b490000)] 0x0b174400 JavaThread "C1 CompilerThread0" daemon [_thread_blocked, id=6272, stack(0x0b3f0000,0x0b440000)] 0x0b172800 JavaThread "Attach Listener" daemon [_thread_blocked, id=6572, stack(0x0b3a0000,0x0b3f0000)] 0x0b187400 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=7200, stack(0x0b350000,0x0b3a0000)] 0x02b4e400 JavaThread "Finalizer" daemon [_thread_blocked, id=6180, stack(0x0b100000,0x0b150000)] 0x02b48400 JavaThread "Reference Handler" daemon [_thread_blocked, id=7344, stack(0x0b0b0000,0x0b100000)] 0x0086a000 JavaThread "main" [_thread_in_native, id=5080, stack(0x00950000,0x009a0000)] Other Threads: 0x02b45000 VMThread [stack: 0x0b060000,0x0b0b0000] [id=6344] 0x0b1aa800 WatcherThread [stack: 0x0b490000,0x0b4e0000] [id=3048] VM state:not at safepoint (normal execution) VM Mutex/Monitor currently owned by a thread: None Heap: def new generation total 58944K, used 22049K [0x02c00000, 0x06bf0000, 0x06c00000) eden space 52416K, 42% used [0x02c00000, 0x041884a0, 0x05f30000) from space 6528K, 0% used [0x05f30000, 0x05f30000, 0x06590000) to space 6528K, 0% used [0x06590000, 0x06590000, 0x06bf0000) tenured generation total 64K, used 0K [0x06c00000, 0x06c10000, 0x0ac00000) the space 64K, 0% used [0x06c00000, 0x06c00000, 0x06c00200, 0x06c10000) Metaspace used 4583K, capacity 4630K, committed 4736K, reserved 5504K Card table byte_map: [0x02b50000,0x02ba0000] byte_map_base: 0x02b3a000 Polling page: 0x009a0000 CodeCache: size=32768Kb used=1110Kb max_used=1110Kb free=31657Kb bounds [0x009d0000, 0x00ae8000, 0x029d0000] total_blobs=430 nmethods=266 adapters=96 compilation: enabled Compilation events (10 events): Event: 4.095 Thread 0x0b174400 258 org.bson.io.OutputBuffer::writeInt (32 bytes) Event: 4.095 Thread 0x0b174400 nmethod 258 0x00ae4608 code [0x00ae4730, 0x00ae4840] Event: 4.102 Thread 0x0b174400 259 java.util.TreeMap::getFirstEntry (26 bytes) Event: 4.102 Thread 0x0b174400 nmethod 259 0x00ae4908 code [0x00ae4a00, 0x00ae4aa0] Event: 4.119 Thread 0x0b174400 261 tong.mongo.defclass.Line::<init> (5 bytes) Event: 4.119 Thread 0x0b174400 nmethod 261 0x00ae4e08 code [0x00ae4f00, 0x00ae4f80] Event: 4.137 Thread 0x0b174400 262 java.util.TreeMap$KeyIterator::next (8 bytes) Event: 4.137 Thread 0x0b174400 nmethod 262 0x00ae4fc8 code [0x00ae50d0, 0x00ae516c] Event: 4.148 Thread 0x0b174400 263 java.util.AbstractSet::<init> (5 bytes) Event: 4.148 Thread 0x0b174400 nmethod 263 0x00ae5208 code [0x00ae5300, 0x00ae5380] GC Heap History (0 events): No events Deoptimization events (0 events): No events Internal exceptions (10 events): Event: 4.007 Thread 0x0b1f4800 Exception <a 'java/security/PrivilegedActionException'> (0x0364b298) thrown at [D: epuppetworkspace8-2-build-windows-i586-cygwinjdk8u11648hotspotsrcsharevmprimsjvm.cpp, line 1248] Event: 4.008 Thread 0x0b1f3400 Exception <a 'java/security/PrivilegedActionException'> (0x03318b20) thrown at [D: epuppetworkspace8-2-build-windows-i586-cygwinjdk8u11648hotspotsrcsharevmprimsjvm.cpp, line 1248] Event: 4.010 Thread 0x0b1f3400 Exception <a 'java/security/PrivilegedActionException'> (0x0331bde0) thrown at [D: epuppetworkspace8-2-build-windows-i586-cygwinjdk8u11648hotspotsrcsharevmprimsjvm.cpp, line 1248] Event: 4.011 Thread 0x0b1f3400 Exception <a 'java/security/PrivilegedActionException'> (0x0331f908) thrown at [D: epuppetworkspace8-2-build-windows-i586-cygwinjdk8u11648hotspotsrcsharevmprimsjvm.cpp, line 1248] Event: 4.011 Thread 0x0b1f3400 Exception <a 'java/security/PrivilegedActionException'> (0x03322820) thrown at [D: epuppetworkspace8-2-build-windows-i586-cygwinjdk8u11648hotspotsrcsharevmprimsjvm.cpp, line 1248] Event: 4.019 Thread 0x0b1f3400 Exception <a 'java/security/PrivilegedActionException'> (0x03755ea8) thrown at [D: epuppetworkspace8-2-build-windows-i586-cygwinjdk8u11648hotspotsrcsharevmprimsjvm.cpp, line 1248] Event: 4.023 Thread 0x0b1f3400 Exception <a 'java/security/PrivilegedActionException'> (0x0375ec58) thrown at [D: epuppetworkspace8-2-build-windows-i586-cygwinjdk8u11648hotspotsrcsharevmprimsjvm.cpp, line 1248] Event: 4.066 Thread 0x0b1f4800 Exception <a 'java/security/PrivilegedActionException'> (0x03871610) thrown at [D: epuppetworkspace8-2-build-windows-i586-cygwinjdk8u11648hotspotsrcsharevmprimsjvm.cpp, line 1248] Event: 4.067 Thread 0x0b1f4800 Exception <a 'java/security/PrivilegedActionException'> (0x03875760) thrown at [D: epuppetworkspace8-2-build-windows-i586-cygwinjdk8u11648hotspotsrcsharevmprimsjvm.cpp, line 1248] Event: 4.131 Thread 0x0b1f3400 Exception <a 'java/security/PrivilegedActionException'> (0x03f5b5b8) thrown at [D: epuppetworkspace8-2-build-windows-i586-cygwinjdk8u11648hotspotsrcsharevmprimsjvm.cpp, line 1248] Events (10 events): Event: 4.023 loading class tong/mongo/defclass/Line Event: 4.023 loading class tong/mongo/defclass/Line done Event: 4.066 loading class com/mongodb/QueryResultIterator$OptionalFinalizer Event: 4.066 loading class com/mongodb/QueryResultIterator$OptionalFinalizer done Event: 4.067 loading class com/mongodb/ServerAddressSelector Event: 4.067 loading class com/mongodb/ServerAddressSelector done Event: 4.131 loading class tong/mongo/loction/LinkedC Event: 4.131 loading class tong/mongo/loction/LinkedC done Event: 4.132 loading class java/lang/ClassLoaderHelper Event: 4.132 loading class java/lang/ClassLoaderHelper done Dynamic libraries: 0x00400000 - 0x00430000 C:Program FilesJavajdk8injavaw.exe 0x7c920000 - 0x7c9b6000 C:WINDOWSsystem32 tdll.dll 0x7c800000 - 0x7c91e000 C:WINDOWSsystem32kernel32.dll 0x77da0000 - 0x77e49000 C:WINDOWSsystem32ADVAPI32.dll 0x77e50000 - 0x77ee3000 C:WINDOWSsystem32RPCRT4.dll 0x77fc0000 - 0x77fd1000 C:WINDOWSsystem32Secur32.dll 0x77d10000 - 0x77da0000 C:WINDOWSsystem32USER32.dll 0x77ef0000 - 0x77f39000 C:WINDOWSsystem32GDI32.dll 0x77180000 - 0x77283000 C:WINDOWSWinSxSx86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.2600.6028_x-ww_61e65202COMCTL32.dll 0x77be0000 - 0x77c38000 C:WINDOWSsystem32msvcrt.dll 0x77f40000 - 0x77fb6000 C:WINDOWSsystem32SHLWAPI.dll 0x76300000 - 0x7631d000 C:WINDOWSsystem32IMM32.DLL 0x62c20000 - 0x62c29000 C:WINDOWSsystem32LPK.DLL 0x73fa0000 - 0x7400b000 C:WINDOWSsystem32USP10.dll 0x78aa0000 - 0x78b5f000 C:Program FilesJavajdk8jreinmsvcr100.dll 0x6d250000 - 0x6d5e7000 C:Program FilesJavajdk8jreinclientjvm.dll 0x71a40000 - 0x71a4b000 C:WINDOWSsystem32WSOCK32.dll 0x71a20000 - 0x71a37000 C:WINDOWSsystem32WS2_32.dll 0x71a10000 - 0x71a18000 C:WINDOWSsystem32WS2HELP.dll 0x76b10000 - 0x76b3a000 C:WINDOWSsystem32WINMM.dll 0x76bc0000 - 0x76bcb000 C:WINDOWSsystem32PSAPI.DLL 0x6f650000 - 0x6f65c000 C:Program FilesJavajdk8jreinverify.dll 0x6dac0000 - 0x6dae1000 C:Program FilesJavajdk8jreinjava.dll 0x6f6e0000 - 0x6f6f3000 C:Program FilesJavajdk8jreinzip.dll 0x7d590000 - 0x7dd84000 C:WINDOWSsystem32SHELL32.dll 0x759d0000 - 0x75a7f000 C:WINDOWSsystem32USERENV.dll 0x6ee40000 - 0x6ee54000 C:Program FilesJavajdk8jrein et.dll 0x10000000 - 0x10038000 C:Documents and SettingsAdministratorLocal SettingsApplication DataTudouFeisuTudouikutm.dll 0x76990000 - 0x76ace000 C:WINDOWSsystem32ole32.dll 0x0b4e0000 - 0x0b51e000 C:Documents and SettingsAdministratorLocal SettingsApplication DataTudouFeisuTudouikutmco.dll 0x719c0000 - 0x719fe000 C:WINDOWSsystem32mswsock.dll 0x60fd0000 - 0x61025000 C:WINDOWSsystem32hnetcfg.dll 0x71a00000 - 0x71a08000 C:WINDOWSSystem32wshtcpip.dll 0x76d30000 - 0x76d48000 C:WINDOWSsystem32IPHLPAPI.DLL 0x76d10000 - 0x76d28000 C:WINDOWSsystem32MPRAPI.dll 0x77c90000 - 0x77cc2000 C:WINDOWSsystem32ACTIVEDS.dll 0x76de0000 - 0x76e05000 C:WINDOWSsystem32adsldpc.dll 0x5fdd0000 - 0x5fe25000 C:WINDOWSsystem32NETAPI32.dll 0x76f30000 - 0x76f5c000 C:WINDOWSsystem32WLDAP32.dll 0x76af0000 - 0x76b01000 C:WINDOWSsystem32ATL.DLL 0x770f0000 - 0x7717b000 C:WINDOWSsystem32OLEAUT32.dll 0x76e50000 - 0x76e5e000 C:WINDOWSsystem32 tutils.dll 0x71b70000 - 0x71b83000 C:WINDOWSsystem32SAMLIB.dll 0x76060000 - 0x761b6000 C:WINDOWSsystem32SETUPAPI.dll 0x6eda0000 - 0x6edaa000 C:Program FilesJavajdk8jreinmanagement.dll 0x76ef0000 - 0x76f17000 C:WINDOWSsystem32DNSAPI.dll 0x76f80000 - 0x76f88000 C:WINDOWSSystem32winrnr.dll 0x76f90000 - 0x76f96000 C:WINDOWSsystem32 asadhlp.dll 0x6ee60000 - 0x6ee6f000 C:Program FilesJavajdk8jrein io.dll 0x66380000 - 0x664c4000 E:eclipse4EEworkspaceMongoLocSpeedEstimate.dll 0x68d60000 - 0x68e01000 C:WINDOWSsystem32dbghelp.dll 0x77bd0000 - 0x77bd8000 C:WINDOWSsystem32VERSION.dll VM Arguments: jvm_args: -Xms64m -Xmx128m -Xmn64m -Dfile.encoding=GBK java_command: tong.mongo.loction.MdbFindThread2 java_class_path (initial): E:eclipse4EEworkspaceMongoLocin;C:Program FilesJavajdk1.7.0_71libextmongo-java-driver-2.13.0-rc1.jar Launcher Type: SUN_STANDARD Environment Variables: JAVA_HOME=C:Program FilesJavajdk1.7.0_71 CLASSPATH=.;C:Program FilesJavajdk1.7.0_71libdt.jar;C:Program FilesJavajdk1.7.0_71lib ools.jar;C:Program FilesJavajre6lib t.jar;D:apache-tomcat-6.0.43libservlet-api.jar; PATH=C:/Program Files/Java/jdk1.7.0_71/jre/bin/client;C:/Program Files/Java/jdk1.7.0_71/jre/bin;C:/Program Files/Java/jdk1.7.0_71/jre/lib/i386;C:apache-maven-3.2.3in;C:Program FilesJavajdk1.7.0_71jrein;C:Program FilesJavajdk8in;C:WINDOWSsystem32;C:WINDOWS;C:WINDOWSSystem32Wbem;E:Matlab2013a untimewin32;E:Matlab2013ain;E: untimewin32;E:mysql 5.1in;C:Program FilesMicrosoft SQL Server100ToolsBinn;C:Program FilesMicrosoft SQL Server100DTSBinn;C:Program FilesMicrosoft SQL Server100ToolsBinnVSShellCommon7IDE;D:vs2008Common7IDEPrivateAssemblies;C:WINDOWSsystem32WindowsPowerShellv1.0;E:in;c:Program FilesCommon FilesUlead SystemsMPEG;C:Program FilesJavajdk1.7.0_71in;E:Androidandroid-sdk-windows ools;E:Androidandroid-sdk-windowsplatform-tools;C:Program Filesscalain;E:sbtin;D:apache-tomcat-6.0.43lib;D:apache-tomcat-6.0.43in;E:mongodb-win32-i386-2.0.7in;E:eclipse4EEeclipse; USERNAME=Administrator OS=Windows_NT PROCESSOR_IDENTIFIER=x86 Family 6 Model 23 Stepping 6, GenuineIntel --------------- S Y S T E M --------------- OS: Windows XP Build 2600 Service Pack 3 CPU:total 2 (2 cores per cpu, 1 threads per core) family 6 model 23 stepping 6, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, tsc Memory: 4k page, physical 2085780k(366936k free), swap 2977508k(896652k free) vm_info: Java HotSpot(TM) Client VM (25.11-b03) for windows-x86 JRE (1.8.0_11-b12), built on Jun 16 2014 18:18:58 by "java_re" with MS VC++ 10.0 (VS2010) time: Tue Apr 21 20:34:24 2015 elapsed time: 4 seconds
问题分析:
查阅网上资料,可能原因是内存泄露。
原因是调用c++库的内容有错误。
逐步定位,经检查,发现是代码中初始化数据的时候出现了问题
在这个过程中学习了JNI,JVM结构以及内存管理
参考资料:
http://www.oschina.net/question/248406_120277?sort=default&p=2#answers