最近修改同事代码时遇到一个问题，通过 httpclient 默认配置产生的 httpclient 如果不关闭，会导致连接无法释放，很快打满服务器连接（内嵌 Jetty 配置了 25 连接上限），主动关闭问题解决；后来优化为通过连接池生成 httpclient 后，如果关闭 httpclient 又会导致连接池关闭，后面新的 httpclient 也无法再请求，这里总结遇到的一些问题和疑问。
- 官网示例中的以下三个 close 分别释放了什么资源，是否可以省略，以及在什么时机调用，使用连接池时有区别么？
- 作为 RPC 通信客户端，如何复用 TCP 连接？
一、资源释放
CloseableHttpClient httpclient = HttpClients.createDefault();
HttpGet httpget = new HttpGet("http://localhost/");
CloseableHttpResponse response = httpclient.execute(httpget);
try {
HttpEntity entity = response.getEntity();
if (entity != null) {
InputStream instream = entity.getContent();
try {
// do something useful
} finally {
instream.close();
}
}
} finally {
response.close();
}
// httpclient.close();
首先需要了解默认配置 createDefault 和使用了 custom 连接池（文章最后的 HttpClientUtil）两种情况的区别，通过源码可以看到前者也创建了连接池，最大连接20个，单个 host最大2个，但是区别在于每次创建的 httpclient 都自己维护了自己的连接池，而 custom 连接池时所有 httpclient 共用同一个连接池，这是在 api 使用方面需要注意的地方，要避免每次请求新建连接池、关闭连接池，造成性能问题。
The difference between closing the content stream and closing the response is that the former will attempt to keep the underlying connection alive by consuming the entity content while the latter immediately shuts down and discards the connection.
第一个 close 是读取 http 正文的数据流，类似的还有响应写入流，都需要主动关闭，如果是使用 EntityUtils.toString(response.getEntity(), "UTF-8"); 的方式，其内部会进行关闭。如果还有要读/写的数据、或不主动关闭，相当于 http 请求事务未处理完成，这时通过其他方式关闭（第二个 close）相当于异常终止，会导致该连接无法被复用，对比下面两段日志。
第一个 close 未调用时，第二个 close 调用，连接无法被复用，kept alive 0。
o.a.http.impl.execchain.MainClientExec : Connection can be kept alive indefinitely
h.i.c.DefaultManagedHttpClientConnection : http-outgoing-0: Close connection
o.a.http.impl.execchain.MainClientExec : Connection discarded
h.i.c.PoolingHttpClientConnectionManager : Connection released: [id: 0][route: {}->http://127.0.0.1:8080][total kept alive: 0; route allocated: 0 of 2; total allocated: 0 of 20]
第一个 close 正常调用时，第二个 close 调用，连接可以被复用，kept alive 1。
o.a.http.impl.execchain.MainClientExec : Connection can be kept alive indefinitely
h.i.c.PoolingHttpClientConnectionManager : Connection [id: 0][route: {}->http://127.0.0.1:8080] can be kept alive indefinitely
h.i.c.DefaultManagedHttpClientConnection : http-outgoing-0: set socket timeout to 0
h.i.c.PoolingHttpClientConnectionManager : Connection released: [id: 0][route: {}->http://127.0.0.1:8080][total kept alive: 1; route allocated: 1 of 2; total allocated: 1 of 20]
第二个 close 是强行制止和释放连接到连接池，相当于对第一个 close 的保底操作（上面关闭了这个似乎没必要了？），结合上面引用的官方文档写到 immediately shuts down and discards the connection，这里如果判断需要 keep alive 实际也不会关闭 TCP 连接，因为通过 netstat 可以看到，第二段日志后在终端可以继续观察到连接：
# netstat -n | grep tcp4 | grep 8080
tcp4 0 0 127.0.0.1.8080 127.0.0.1.51003 ESTABLISHED
tcp4 0 0 127.0.0.1.51003 127.0.0.1.8080 ESTABLISHED
在 SOF 上可以搜到这段话，但是感觉和上面观察到的并不相符？
The underlying HTTP connection is still held by the response object to allow the response content to be streamed directly from the network socket. In order to ensure correct deallocation of system resources, the user MUST call CloseableHttpResponse#close() from a finally clause. Please note that if response content is not fully consumed the underlying connection cannot be safely re-used and will be shut down and discarded by the connection manager.
第三个 clsoe，也就是 httpclient.close 会彻底关闭连接池，以及其中所有连接，一般情况下，只有在关闭应用时调用以释放资源。
二、连接复用
根据 http 协议 1.1 版本，各个 web 服务器都默认支持 keepalive，因此当 http 请求正常完成后，服务器不会主动关闭 tcp（直到空闲超时或数量达到上限），使连接会保留一段时间，前面我们也知道 httpclient 在判断可以 keepalive 后，即使调用了 close 也不会关闭 tcp 连接（可以认为 release 到连接池）。为了管理这些保留的连接，以及方便 api 调用，一般设置一个全局的连接池，并基于该连接池提供 httpclient 实例，这样就不需要考虑维护 httpclient 实例生命周期，随用随取（方便状态管理？），此外考虑到 http 的单路性，一个请求响应完成结束后，该连接才可以再次复用，因此连接池的最大连接数决定了并发处理量，该配置也是一种保护机制，超出上限的请求会被阻塞，也可以配合熔断组件使用，当服务方慢、或不健康时熔断降级。
最后还有一个问题，观察到 keepalive 的 tcp 连接过一段时间后会变成如下状态：
# netstat -n | grep tcp4 | grep 8080
tcp4 0 0 127.0.0.1.8080 127.0.0.1.51866 FIN_WAIT_2
tcp4 0 0 127.0.0.1.51866 127.0.0.1.8080 CLOSE_WAIT
可以看出服务器经过一段时间，认为该连接空闲，因此主动关闭，收到对方响应后进入 FIN_WAIT_2 状态（等待对方也发起关闭），而客户端进入 CLOSE_WAIT 状态后却不再发起自己这一方的关闭请求，这时双方处于半关闭。官方文档解释如下：
One of the major shortcomings of the classic blocking I/O model is that the network socket can react to I/O events only when blocked in an I/O operation. When a connection is released back to the manager, it can be kept alive however it is unable to monitor the status of the socket and react to any I/O events. If the connection gets closed on the server side, the client side connection is unable to detect the change in the connection state (and react appropriately by closing the socket on its end).
这需要有定期主动做一些检测和关闭动作，从这个角度考虑，默认配置产生的 HttpClient 没有这一功能，不应该用于生产环境，下面这个监控线程可以完成该工作，包含它的完整的 HttpUtil 从文章最后连接获取。
public static class IdleConnectionMonitorThread extends Thread {
private final HttpClientConnectionManager connMgr;
private volatile boolean shutdown;
public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {
super();
this.connMgr = connMgr;
}
@Override
public void run() {
try {
while (!shutdown) {
synchronized (this) {
wait(30 * 1000);
// Close expired connections
connMgr.closeExpiredConnections();
// Optionally, close connections
// that have been idle longer than 30 sec
connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
}
}
} catch (InterruptedException ex) {
// terminate
}
}
最后展示一个完整的示例，首先多线程发起两个请求，看到创建两个连接，30秒之后再发起一个请求，可以复用之前其中一个连接，另一个连接因空闲被关闭，随后最后等待 2 分钟后再发起一个请求，由于之前连接已过期失效，重新创建连接。
-
并发两个请求
16:54:44.504 [ Thread-4] : Connection request: [route: {}->http://127.0.0.1:8080][total kept alive: 0; route allocated: 0 of 150; total allocated: 0 of 150] 16:54:44.504 [ Thread-5] : Connection request: [route: {}->http://127.0.0.1:8080][total kept alive: 0; route allocated: 0 of 150; total allocated: 0 of 150] 16:54:44.515 [ Thread-5] : Connection leased: [id: 1][route: {}->http://127.0.0.1:8080][total kept alive: 0; route allocated: 2 of 150; total allocated: 2 of 150] 16:54:44.515 [ Thread-4] : Connection leased: [id: 0][route: {}->http://127.0.0.1:8080][total kept alive: 0; route allocated: 2 of 150; total allocated: 2 of 150] 16:54:44.517 [ Thread-5] : Opening connection {}->http://127.0.0.1:8080 16:54:44.517 [ Thread-4] : Opening connection {}->http://127.0.0.1:8080 16:54:44.519 [ Thread-4] : Connecting to /127.0.0.1:8080 16:54:44.519 [ Thread-5] : Connecting to /127.0.0.1:8080 16:54:44.521 [ Thread-5] : Connection established 127.0.0.1:52421<->127.0.0.1:8080 16:54:44.521 [ Thread-4] : Connection established 127.0.0.1:52420<->127.0.0.1:8080 .... 16:54:49.486 [ main] : [leased: 2; pending: 0; available: 0; max: 150] 16:54:49.630 [ Thread-4] : Connection can be kept alive indefinitely 16:54:49.630 [ Thread-5] : Connection can be kept alive indefinitely 16:54:49.633 [ Thread-4] : Connection [id: 0][route: {}->http://127.0.0.1:8080] can be kept alive indefinitely 16:54:49.633 [ Thread-5] : Connection [id: 1][route: {}->http://127.0.0.1:8080] can be kept alive indefinitely 16:54:49.633 [ Thread-4] : http-outgoing-0: set socket timeout to 0 16:54:49.633 [ Thread-5] : http-outgoing-1: set socket timeout to 0 16:54:49.633 [ Thread-4] : Connection released: [id: 0][route: {}->http://127.0.0.1:8080][total kept alive: 1; route allocated: 2 of 150; total allocated: 2 of 150] 16:54:49.633 [ Thread-5] : Connection released: [id: 1][route: {}->http://127.0.0.1:8080][total kept alive: 2; route allocated: 2 of 150; total allocated: 2 of 150] 16:54:54.488 [ main] : [leased: 0; pending: 0; available: 2; max: 150]#netstat -n | grep tcp4 | grep 8080 tcp4 0 0 127.0.0.1.8080 127.0.0.1.52421 ESTABLISHED tcp4 0 0 127.0.0.1.8080 127.0.0.1.52420 ESTABLISHED tcp4 0 0 127.0.0.1.52421 127.0.0.1.8080 ESTABLISHED tcp4 0 0 127.0.0.1.52420 127.0.0.1.8080 ESTABLISHED -
下一个请求
16:55:14.489 [ Thread-6] : Connection request: [route: {}->http://127.0.0.1:8080][total kept alive: 2; route allocated: 2 of 150; total allocated: 2 of 150] 16:55:14.491 [ Thread-6] : http-outgoing-1 << "[read] I/O error: Read timed out" 16:55:14.491 [ Thread-6] : Connection leased: [id: 1][route: {}->http://127.0.0.1:8080][total kept alive: 1; route allocated: 2 of 150; total allocated: 2 of 150] 16:55:14.491 [ Thread-6] : http-outgoing-1: set socket timeout to 0 16:55:14.492 [ Thread-6] : http-outgoing-1: set socket timeout to 8000 ..... 16:55:19.501 [ main] : [leased: 1; pending: 0; available: 1; max: 150] 16:55:19.504 [ Thread-6] : Connection can be kept alive indefinitely 16:55:19.504 [ Thread-6] : Connection [id: 1][route: {}->http://127.0.0.1:8080] can be kept alive indefinitely 16:55:19.505 [ Thread-6] : http-outgoing-1: set socket timeout to 0 16:55:19.505 [ Thread-6] : Connection released: [id: 1][route: {}->http://127.0.0.1:8080][total kept alive: 2; route allocated: 2 of 150; total allocated: 2 of 150] 16:55:24.504 [ main] : [leased: 0; pending: 0; available: 2; max: 150]#netstat -n | grep tcp4 | grep 8080 tcp4 0 0 127.0.0.1.8080 127.0.0.1.52421 ESTABLISHED tcp4 0 0 127.0.0.1.8080 127.0.0.1.52420 ESTABLISHED tcp4 0 0 127.0.0.1.52421 127.0.0.1.8080 ESTABLISHED tcp4 0 0 127.0.0.1.52420 127.0.0.1.8080 ESTABLISHED复用了上面的连接，下面是随后逐步超时的日志。
16:55:39.513 [ main] : [leased: 0; pending: 0; available: 2; max: 150] 16:55:44.491 [ Thread-8] : Closing expired connections 16:55:44.492 [ Thread-8] : Closing connections idle longer than 30 SECONDS 16:55:44.492 [ Thread-8] : http-outgoing-0: Close connection 16:55:44.518 [ main] : [leased: 0; pending: 0; available: 1; max: 150] .... 16:56:09.535 [ main] : [leased: 0; pending: 0; available: 1; max: 150] 16:56:14.499 [ Thread-8] : Closing expired connections 16:56:14.499 [ Thread-8] : Closing connections idle longer than 30 SECONDS 16:56:14.499 [ Thread-8] : http-outgoing-1: Close connection 16:56:14.540 [ main] : [leased: 0; pending: 0; available: 0; max: 150]分别对应状态如下，可以看到复用了 52421，随后 52420 空闲超时被回收，以及最后 52421 也被回收。
#netstat -n | grep tcp4 | grep 8080 tcp4 0 0 127.0.0.1.8080 127.0.0.1.52421 ESTABLISHED tcp4 0 0 127.0.0.1.52421 127.0.0.1.8080 ESTABLISHED tcp4 0 0 127.0.0.1.52420 127.0.0.1.8080 TIME_WAIT ... #netstat -n | grep tcp4 | grep 8080 tcp4 0 0 127.0.0.1.52421 127.0.0.1.8080 TIME_WAIT -
最后一个请求后，日志省略，可以看到是新的连接 52443。
netstat -n | grep tcp4 | grep 8080 tcp4 0 0 127.0.0.1.8080 127.0.0.1.52443 ESTABLISHED tcp4 0 0 127.0.0.1.52443 127.0.0.1.8080 ESTABLISHED
文章所有演示用例和封装类链接：https://github.com/JeffreyPeng/http-client-case/
参考：
https://hc.apache.org/httpcomponents-client-4.5.x/tutorial/html/fundamentals.html
https://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html
https://www.baeldung.com/httpclient-connection-management