JAVA—HTTP客户端警告：Going to buffer response body of large or unknown size.

关注微信公众号：CodingTechWork，一起学习进步。

原编码问题

  在修复原有代码bug时，发现日志里经常抛出Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended，这种提示。查看了原有代码中的逻辑如下：

... ...
HttpClient httpclient = new HttpClient();
GetMethod getMethod = new GetMethod(url);
int statusCode = httpclient.executeMethod(getMethod);
String respContent = getMethod.getResponseBodyAsString();
... ...

  原因就在于使用了getResponseBodyAsString()方法

源码分析

getResponseBodyAsString()源码

	//getResponseBodyAsString()方法源码
    public String getResponseBodyAsString() throws IOException {
        byte[] rawdata = null;
        if (this.responseAvailable()) {
        	//调用了getResponseBody()，容易消耗内存
            rawdata = this.getResponseBody();
        }

        return rawdata != null ? EncodingUtil.getString(rawdata, this.getResponseCharSet()) : null;
    }
    //responseAvailable()方法源码
    private boolean responseAvailable() {
        return this.responseBody != null || this.responseStream != null;
    }
    
    //getResponseBody()方法源码
    public byte[] getResponseBody() throws IOException {
        if (this.responseBody == null) {
            InputStream instream = this.getResponseBodyAsStream();
            if (instream != null) {
                long contentLength = this.getResponseContentLength();
                if (contentLength > 2147483647L) {
                    throw new IOException("Content too large to be buffered: " + contentLength + " bytes");
                }

                int limit = this.getParams().getIntParameter("http.method.response.buffer.warnlimit", 1048576);
                if (contentLength == -1L || contentLength > (long)limit) {
                //这里是warn的原文
                    LOG.warn("Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.");
                }

                LOG.debug("Buffering response body");
                ByteArrayOutputStream outstream = new ByteArrayOutputStream(contentLength > 0L ? (int)contentLength : 4096);
                byte[] buffer = new byte[4096];

                int len;
                while((len = instream.read(buffer)) > 0) {
                    outstream.write(buffer, 0, len);
                }

                outstream.close();
                this.setResponseStream((InputStream)null);
                this.responseBody = outstream.toByteArray();
            }
        }

        return this.responseBody;
    }

  从源码中可以看出，warn的条件是(contentLength == -1L || contentLength > (long)limit)，如果http头没有指定contentLength或大于上限值（默认1M），就会抛异常。其实，如果返回的结果比较确定，对程序没有太大影响。而对于返回结果不确定时，源码也建议我们使用下面的getResponseBodyAsStream()方法。

getResponseBodyAsStream()源码

    public InputStream getResponseBodyAsStream() throws IOException {
        if (this.responseStream != null) {
            return this.responseStream;
        } else if (this.responseBody != null) {
            InputStream byteResponseStream = new ByteArrayInputStream(this.responseBody);
            LOG.debug("re-creating response stream from byte array");
            return byteResponseStream;
        } else {
            return null;
        }
    }

  从源码中可以看出，getResponseBodyAsStream()内部没有使用getResponseBody()方法，避免了内存耗尽问题，而是使用了InputStream流方式处理。

优化原编码

... ...
HttpClient httpclient = new HttpClient();
GetMethod getMethod = new GetMethod(url);
int statusCode = httpclient.executeMethod(getMethod);
//String respContent = getMethod.getResponseBodyAsString();
//使用getResponseBodyAsStream()
InputStream inputStream = getMethod.getResponseBodyAsStream();
BufferedReader br = new BufferedReader(new InputStreamReader(inputStream));
StringBuffer stringBuffer = new StringBuffer();
String str = "";
while ((str = br.readLine()) != null) {
       stringBuffer.append(str);
}
LOGGER.info("respContent: {}", stringBuffer.toString());
... ...

  测了一把，哦豁，不再报这个异常了。完美。