Can someone explain how to use the request.js pool hash?
The github notes say this about pools:
pool - A hash object containing the agents for these requests. If omitted this request will use the global pool which is set to node's default maxSockets.
pool.maxSockets - Integer containing the maximum amount of sockets in the pool.
I have this code for writing to a CouchDB instance (note the question marks). Basically, any user who connects to my Node server will write to the DB independent of each other:
1 var request = require('request'); 2 3 request({ 4 //pool:, // ?????????????????? 5 'pool.maxSockets' : 100, // ?????????????????? 6 'method' : 'PUT', 7 'timeout' : 4000, 8 'strictSSL' : true, 9 'auth' : { 10 'username' : myUsername, 11 'password' : myPassword 12 }, 13 'headers' : { 14 'Content-Type': 'application/json;charset=utf-8', 15 'Content-Length': myData.length 16 }, 17 'json' : myData, 18 'url': myURL 19 }, function (error, response, body){ 20 if (error == null) { 21 log('Success: ' + body); 22 } 23 else { 24 log('Error: ' + error); 25 } 26 });
What's best for high throughput/performance?
What are the drawbacks of a high 'maxSockets' number?
How do I create a separate pool to use instead of the global pool? Why do I only want to create a separate pool?
The pool option in request uses agent which is same as http.Agent
from standard http library. See the documentation for http.Agent and see the agent
options in http.request.
Usage
1 pool = new http.Agent(); //Your pool/agent 2 http.request({hostname:'localhost', port:80, path:'/', agent:pool}); 3 request({url:"http://www.google.com", pool:pool });
If you are curious to know what is that you can see it from console.
1 { domain: null, 2 _events: { free: [Function] }, 3 _maxListeners: 10, 4 options: {}, 5 requests: {}, 6 sockets: {}, 7 maxSockets: 5, 8 createConnection: [Function] }
The pool option in request uses agent which is same as Usage
If you are curious to know what is that you can see it from console.
The So different ways to use it :
Answering your questions in reverse. Pool is meant to keep certain number of sockets to be used by the program. Firstly the sockets are reused for different requests. So it reduces overhead of creating new sockets. Secondly it uses fewer sockets for requests, but consistently. It will not take up all sockets available. Thirdly it maintains queue of requests. So there is waiting time implied. Pool acts like both a cache and a throttle. The throttle effect will be more visible if you have more requests and lesser sockets. When using global pool it may limit functioning of two different clients, there are no guarantees on waiting time. Having separate pool for them will be fairer to both (think if one requests more than other). The maxSockets property gives maximum concurrency possible. It increases the overall throughput/performance. Drawback is throttle effect is reduced. You cannot control peak overhead. Setting it to large number, will be like no pooling at all. You would start getting errors like socket not available. It cannot be more than the allowed maximum limit set by the OS. So what is best for high throughput/performance? There is a physical limit in throughput. If you reach the limit, response time will increase with number of connections. You can keep increasing maxSockets till then, but after that increasing it will not help. |