背景
今天下午遇到同事求助，说是服务端出现了好几个java.io.IOException: Broken pipe这样的异常，让我帮忙看一下，这个问题对于我们做服务端开发的技术人员是很容易遇到的，特此记录一下。
探究
问题堆栈
org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:356) at org.apache.catalina.connector.OutputBuffer.flushByteBuffer(OutputBuffer.java:825) at org.apache.catalina.connector.OutputBuffer.append(OutputBuffer.java:730) at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:391) at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:369) at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:96) at org.springframework.session.web.http.OnCommittedResponseWrapper$SaveContextServletOutputStream.write( OnCommittedResponseWrapper.java:620) at org.springframework.security.web.util.OnCommittedResponseWrapper$SaveContextServletOutputStream.write( OnCommittedResponseWrapper.java:639) at org.springframework.security.web.util.OnCommittedResponseWrapper$SaveContextServletOutputStream.write( OnCommittedResponseWrapper.java:639) at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2085) at com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeBytes(UTF8JsonGenerator.java:1173) at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeFieldName(UTF8JsonGenerator.java:256) at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:725) at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119) at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79) at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18) at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727) at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719) at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396) at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:913) at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.writeInternal( AbstractJackson2HttpMessageConverter.java:286) at org.springframework.http.converter.AbstractGenericHttpMessageConverter.write(AbstractGenericHttpMessageConverter.java:102) at org.springframework.web.servlet.mvc.method.annotation.AbstractMessageConverterMethodProcessor.writeWithMessageConverters( AbstractMessageConverterMethodProcessor.java:272) at org.springframework.web.servlet.mvc.method.annotation.RequestResponseBodyMethodProcessor.handleReturnValue( RequestResponseBodyMethodProcessor.java:180) at org.springframework.web.method.support.HandlerMethodReturnValueHandlerComposite.handleReturnValue( HandlerMethodReturnValueHandlerComposite.java:82) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle( ServletInvocableHandlerMethod.java:119) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod( RequestMappingHandlerAdapter.java:877) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal( RequestMappingHandlerAdapter.java:783) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:991) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:925) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:974) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:877) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:851) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
着重关注点
堆栈信息中有一行比较关键，在OutputBuffer :: realWriteBytes这个方法中出现了异常，在方法内部抛出了ClientAbortException：
at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:356)
OutputBuffer#realWriteBytes
/**
* Sends the buffer data to the client output, checking the
* state of Response and calling the right interceptors.
*
* @param buf the ByteBuffer to be written to the response
*
* @throws IOException An underlying IOException occurred
*/
public void realWriteBytes(ByteBuffer buf) throws IOException {
if (closed) {
return;
}
if (coyoteResponse == null) {
return;
}
// If we really have something to write
if (buf.remaining() > 0) {
// real write to the adapter
try {
coyoteResponse.doWrite(buf);
} catch (IOException e) {
// An IOException on a write is almost always due to
// the remote client aborting the request. Wrap this
// so that it can be handled better by the error dispatcher.
throw new ClientAbortException(e);
}
}
}
在上面的代码中，调用coyoteResponse.doWrite(buf);方法时，会抛出ClientAbortException异常
我们仔细阅读以下上述注释：
An IOException on a write is almost always due to the remote client aborting the request. Wrap this so that it can be handled better by the error dispatcher. 进行写操作时出现的IOException几乎总是由于远程客户端中止请求而导致的。将其包装起来，以便错误调度程序可以更好地处理它。
结论
有位大神将此问题对应的流程图画的很好，特摘录过来：秋夜无霜
如上图：
调用方通过restTemplate封装的http连接池，这时候连接设置connectTimeout=3000,readTimeout=3000，然后网络请求http请求+端口调用到服务方，然后调用一个api（uri），这时候响应的时候，然后Spring框架对响应进行JSON序列化，然后写入Socket，这时候调用方（客户端）突然Socket端中断连接，而Server端正在进行输出流写，导致如上异常问题。
然后针对当前业务场景，看了今天300个请求，有不到10个出现这种问题，超时大多3、4秒样子，我们通知调用方增大连接池的超时时间，就不再出现这个问题。
最近气温变化大，冷暖转换明显，由于没有及时增添衣物，注意保暖，翎野君感染风寒，现在正在调养恢复中，希望大家也多多注意，春捂秋冻。
参考文章