线上图片下载服务器平时运行正常,最近突然出现一种比较奇怪的现象,只接受请求,但却没有处理请求,最开始怀疑下载线程挂掉了,dump 项目线程后发现异常:
"pool-2-thread-1" prio=10 tid=0x00007f7fbc7c5000 nid=0x4e9 runnable [0x00007f800a05e000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:152)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
at sun.security.ssl.InputRecord.readV3Record(InputRecord.java:554)
at sun.security.ssl.InputRecord.read(InputRecord.java:509)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
- locked <0x00000006f886b898> (a java.lang.Object)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
- locked <0x00000006f886b948> (a java.lang.Object)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:275)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:254)
at org.apache.http.impl.conn.HttpClientConnectionOperator.connect(HttpClientConnectionOperator.java:117)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:314)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:363)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:219)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
所有下载线程都处于这种状态,项目中实际上时没有证书的,ssl验证肯定也不会过,java.net.SocketInputStream.socketRead0 基本上也不可能读取到数据,那就很可能是超时未设置或者设置之后未生效,后来发现,实际上是httpclient 4.3.6之前的一个bug,项目中用的是4.3.2,问题在httpclient/src/main/java/org/apache/http/conn/ssl/SSLConnectionSocketFactory.java:
参考链接:https://github.com/apache/httpcomponents-client/commit/d954cd287dfcdad8f153e61181e20d253175ca8c
因此将httpclient 版本更新至4.3.6之后就正常了。