代码之家  ›  专栏  ›  技术社区  ›  wishihadabettername

JVM消耗所有CPU,大多数线程被阻塞。JVM错误?

locking concurrency multithreading java
  •  10
  • wishihadabettername  · 技术社区  · 15 年前

    昨晚,一个服务器(JBase5.1Ga,Java(TM)SE运行时环境(构建1.60y20B02), Java热点(TM)64位服务器VM,运行在Linux VM上的VMware)突然开始使用100% CPU。这个应用程序是一个相当典型的运行Seam的J2EE业务应用程序,没有什么特别之处。当时负荷极低。

    我设法在进程终止和重新启动之前得到了一个线程转储(应用程序必须可用)。JSTACK在死锁检测中有一个异常;除此之外,大多数线程都处于阻塞状态。

    这是一个JVM bug吗?我以前没见过。重新启动之后一切都正常,就像在应用程序在该服务器上使用之后的几个月(定期更新)。

    谢谢你的建议。

    下面是经过轻微编辑的线程转储(以适应SO的后大小限制)。我删除了一些相同的线程(从jboss/seam内部的不同线程池中),只留下堆栈跟踪的顶行。

    编辑:全线程转储 在 http://pastie.org/1083984 

    [~]$ jstack -F 13553
Attaching to process ID 13553, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 11.3-b02
Deadlock Detection:

java.lang.NullPointerException
        at sun.jvm.hotspot.oops.InstanceKlass.computeSubtypeOf(InstanceKlass.java:426)
        at sun.jvm.hotspot.oops.Klass.isSubtypeOf(Klass.java:137)
        at sun.jvm.hotspot.oops.Oop.isA(Oop.java:92)
        at sun.jvm.hotspot.runtime.DeadlockDetector.print(DeadlockDetector.java:93)
        at sun.jvm.hotspot.runtime.DeadlockDetector.print(DeadlockDetector.java:39)
        at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:52)
        at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
        at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
        at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
        at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
        at sun.tools.jstack.JStack.main(JStack.java:84)
Can't print deadlocks:null
Thread 26208: (state = BLOCKED)


Thread 25250: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 25249: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 21240: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.apache.tomcat.util.net.JIoEndpoint$Worker.await() @bci=8, line=416 (Interpreted frame)

Thread 21140: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.apache.tomcat.util.net.JIoEndpoint$Worker.await() @bci=8, line=416 (Interpreted frame)

Thread 21139: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.apache.tomcat.util.net.JIoEndpoint$Worker.await() @bci=8, line=416 (Interpreted frame)

Thread 21138: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.apache.tomcat.util.net.JIoEndpoint$Worker.await() @bci=8, line=416 (Interpreted frame)

Thread 19380: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 19377: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 19361: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)

Thread 19343: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 19317: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 18995: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)

Thread 18986: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)

Thread 17659: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 17658: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 17653: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 17631: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 14977: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 14796: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 14609: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 14437: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 14248: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 14232: (state = BLOCKED)
 - java.util.Arrays.copyOf(char[], int) @bci=1, line=2882 (Compiled frame)
 - java.lang.AbstractStringBuilder.expandCapacity(int) @bci=33, line=100 (Compiled frame)

Thread 14040: (state = IN_NATIVE)
 - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame)
 - java.net.PlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=384 (Interpreted frame)

Thread 13858: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=116 (Compiled frame)

Thread 13815: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.apache.tomcat.util.net.JIoEndpoint$Worker.await() @bci=8, line=416 (Interpreted frame)

 Thread 13814: (state = IN_NATIVE)
 - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame)

Thread 13813: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.apache.tomcat.util.net.JIoEndpoint$Worker.await() @bci=8, line=416 (Interpreted frame)


Thread 13812: (state = BLOCKED)
Error occurred during stack walking:
sun.jvm.hotspot.utilities.AssertionFailure: range check
        at sun.jvm.hotspot.utilities.Assert.that(Assert.java:32)
        at sun.jvm.hotspot.runtime.RegisterMap.<init>(RegisterMap.java:109)
        at sun.jvm.hotspot.runtime.x86.X86RegisterMap.<init>(X86RegisterMap.java:39)
        at sun.jvm.hotspot.runtime.x86.X86RegisterMap.clone(X86RegisterMap.java:43)
        at sun.jvm.hotspot.runtime.VFrame.<init>(VFrame.java:37)
        at sun.jvm.hotspot.runtime.JavaVFrame.<init>(JavaVFrame.java:45)
        at sun.jvm.hotspot.runtime.CompiledVFrame.<init>(CompiledVFrame.java:43)
        at sun.jvm.hotspot.runtime.VFrame.newVFrame(VFrame.java:77)
        at sun.jvm.hotspot.runtime.VFrame.sender(VFrame.java:129)
        at sun.jvm.hotspot.runtime.VFrame.javaSender(VFrame.java:146)
        at sun.jvm.hotspot.runtime.JavaThread.getLastJavaVFrameDbg(JavaThread.java:231)
        at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:76)
        at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:45)
        at sun.jvm.hotspot.tools.JStack.run(JStack.java:60)
        at sun.jvm.hotspot.tools.Tool.start(Tool.java:221)
        at sun.jvm.hotspot.tools.JStack.main(JStack.java:86)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at sun.tools.jstack.JStack.runJStackTool(JStack.java:118)
        at sun.tools.jstack.JStack.main(JStack.java:84)


Thread 13811: (state = IN_NATIVE)
 - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame)

Thread 13810: (state = IN_NATIVE)
 - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame)

Thread 13809: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.jboss.jms.client.container.ClientConsumer.getMessage(long) @bci=71, line=866 (Interpreted frame)

Thread 13808: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.jboss.jms.client.container.ClientConsumer.getMessage(long) @bci=71, line=866 (Interpreted frame)

Thread 13807: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.jboss.bootstrap.AbstractServerImpl$LifeThread.run() @bci=11, line=866 (Interpreted frame)

Thread 13806: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 13805: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 13804: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.util.TimerThread.mainLoop() @bci=201, line=509 (Interpreted frame)

Thread 13803: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.util.TimerThread.mainLoop() @bci=201, line=509 (Interpreted frame)

Thread 13802: (state = BLOCKED)
 - java.util.Arrays.copyOfRange(char[], int, int) @bci=40, line=3209 (Compiled frame)
 - java.lang.String.<init>(char[], int, int) @bci=65, line=216 (Compiled frame)
 - java.lang.StringBuffer.toString() @bci=13, line=585 (Compiled frame)
 - org.apache.log4j.PatternLayout.format(org.apache.log4j.spi.LoggingEvent) @bci=68, line=506 (Compiled frame)

Thread 13784: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=116 (Compiled frame)
 - org.jboss.el.util.ReferenceCache$ReferenceQueueRunner.run() @bci=1, line=159 (Interpreted frame)

Thread 13753: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be imprecise)
 - org.jboss.ejb3.cache.simple.SimpleStatefulCache$SessionTimeoutTask.block() @bci=11, line=210 (Interpreted frame)

Thread 13752: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - org.jboss.resource.connectionmanager.IdleRemover$IdleRemoverRunnable.run() @bci=31, line=167 (Interpreted frame)

Thread 13749: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - org.jboss.remoting.transport.socket.ServerThread.run() @bci=209, line=284 (Interpreted frame)

Thread 13734: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.util.TimerThread.mainLoop() @bci=201, line=509 (Interpreted frame)

Thread 13701: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, line=158 (Interpreted frame)

Thread 13700: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be imprecise)
 - org.jboss.console.plugins.AOPLister$RefreshPoller.run() @bci=41, line=898 (Interpreted frame)

Thread 13699: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be imprecise)
 - org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run() @bci=22, line=1590 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=619 (Interpreted frame)


Thread 13698: (state = IN_NATIVE)
 - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Interpreted frame)

Thread 13678: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) @bci=20, line=198 (Interpreted frame)

Thread 13554: (state = BLOCKED)

Thread 13560: (state = BLOCKED)

Thread 13559: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=116 (Compiled frame)
 - java.lang.ref.ReferenceQueue.remove() @bci=2, line=132 (Compiled frame)
 - java.lang.ref.Finalizer$FinalizerThread.run() @bci=3, line=159 (Compiled frame)


Thread 13558: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Compiled frame)
 - java.lang.ref.Reference$ReferenceHandler.run() @bci=46, line=116 (Compiled frame)
    2 回复  |  直到 13 年前
        1
  •  15
  •   Tim Bender    15 年前

    一种可能是堆空间不足。如果堆空间用完,JVM将开始以惊人的速度消耗CPU进行疯狂的垃圾收集。

        2
  •  0
  •   Zan Lynx    15 年前

    另一种可能是磁盘或RAM错误。这些是非常罕见的,特别是如果您使用的是ECC RAM、SAS或SCSI,它们在数据传输中包含错误更正。但这可能发生!

    我读过IBM和Sun内核工程师为大型企业提供支持时看到的一些非常奇怪的错误,他们只能归咎于这样的事情。

    这对你没有多大帮助,除了让你知道,你可能永远不会发现它发生的真正原因,它可能不会再发生多年。:)

    推荐文章
    Bass  ·  @Volatile lateinit变量可以在Kotlin中使用DCL模式进行原子初始化吗?
    6 月前
    bepopov  ·  无法在继承的Thread类中将值设置为Inheritable ThreadLocal字段
    7 月前
    Starnec  ·  Anticapte FileHooser关闭以中止正在运行的任务
    7 月前
    user107586  ·  如何处理等待句柄不会导致无限循环?
    7 月前
    ron burgundy  ·  获取-释放语义是否跨线程传递?[副本]
    7 月前
    BenjiFB  ·  C#内存缓存:在一次操作中追加到列表?
    8 月前
    Adam Rutledge  ·  让父线程在初始化子线程pthread后等待,直到它从子线程接收到一些信号的最佳方法是什么?
    8 月前
    András Takács  ·  Python多线程问题
    12 月前
    András Takács  ·  Python多线程错误
    12 月前
    eden sson  ·  使用std::thread和native_handle()设置线程关联掩码
    1 年前
    关于   移动版
    代码之家 - 一站式码农服务社区
    沪ICP备11025650号