source: trip-planner-front/node_modules/needle/README.md@ bdd6491

Last change on this file since bdd6491 was 6a3a178, checked in by Ema <ema_spirova@…>, 3 years ago

initial commit

  • Property mode set to 100644
File size: 23.5 KB
Line 
1Needle
2======
3
4[![NPM](https://nodei.co/npm/needle.png)](https://nodei.co/npm/needle/)
5
6The leanest and most handsome HTTP client in the Nodelands.
7
8```js
9var needle = require('needle');
10
11needle.get('http://www.google.com', function(error, response) {
12 if (!error && response.statusCode == 200)
13 console.log(response.body);
14});
15```
16
17Callbacks not floating your boat? Needle got your back.
18
19``` js
20var data = {
21 file: '/home/johnlennon/walrus.png',
22 content_type: 'image/png'
23};
24
25// the callback is optional, and needle returns a `readableStream` object
26// that triggers a 'done' event when the request/response process is complete.
27needle
28 .post('https://my.server.com/foo', data, { multipart: true })
29 .on('readable', function() { /* eat your chunks */ })
30 .on('done', function(err) {
31 console.log('Ready-o!');
32 })
33```
34
35From version 2.0.x up, Promises are also supported. Just call `needle()` directly and you'll get a native Promise object.
36
37```js
38needle('put', 'https://hacking.the.gibson/login', { password: 'god' }, { json: true })
39 .then(function(response) {
40 return doSomethingWith(response)
41 })
42 .catch(function(err) {
43 console.log('Call the locksmith!')
44 })
45```
46
47With only two real dependencies, Needle supports:
48
49 - HTTP/HTTPS requests, with the usual verbs you would expect
50 - All of Node's native TLS options, such as 'rejectUnauthorized' (see below)
51 - Basic & Digest authentication with auto-detection
52 - Multipart form-data (e.g. file uploads)
53 - HTTP Proxy forwarding, optionally with authentication
54 - Streaming gzip, deflate, and brotli decompression
55 - Automatic XML & JSON parsing
56 - 301/302/303 redirect following, with fine-grained tuning, and
57 - Streaming non-UTF-8 charset decoding, via `iconv-lite`
58
59And yes, Mr. Wayne, it does come in black.
60
61This makes Needle an ideal alternative for performing quick HTTP requests in Node, either for API interaction, downloading or uploading streams of data, and so on. If you need OAuth, AWS support or anything fancier, you should check out mikeal's request module.
62
63Install
64-------
65
66```
67$ npm install needle
68```
69
70Usage
71-----
72
73```js
74// using promises
75needle('get', 'https://server.com/posts/123')
76 .then(function(resp) {
77 // ...
78 })
79 .catch(function(err) {
80 // ...
81 });
82
83// with callback
84needle.get('ifconfig.me/all.json', function(error, response, body) {
85 if (error) throw error;
86
87 // body is an alias for `response.body`,
88 // that in this case holds a JSON-decoded object.
89 console.log(body.ip_addr);
90});
91
92// no callback, using streams
93needle.get('https://google.com/images/logo.png')
94 .pipe(fs.createWriteStream('logo.png'))
95 .on('done', function(err) {
96 console.log('Pipe finished!');
97 });
98```
99
100As you can see, you can use Needle with Promises or without them. When using Promises or when a callback is passed, the response's body will be buffered and written to `response.body`, and the callback will be fired when all of the data has been collected and processed (e.g. decompressed, decoded and/or parsed).
101
102When no callback is passed, however, the buffering logic will be skipped but the response stream will still go through Needle's processing pipeline, so you get all the benefits of post-processing while keeping the streamishness we all love from Node.
103
104Response pipeline
105-----------------
106
107Depending on the response's Content-Type, Needle will either attempt to parse JSON or XML streams, or, if a text response was received, will ensure that the final encoding you get is UTF-8.
108
109You can also request a gzip/deflated/brotli response, which, if sent by the server, will be processed before parsing or decoding is performed. (Note: brotli is only supported on Node 10.16.0 or above, and will not be requested or processed on earlier versions.)
110
111```js
112needle.get('http://stackoverflow.com/feeds', { compressed: true }, function(err, resp) {
113 console.log(resp.body); // this little guy won't be a Gzipped binary blob
114 // but a nice object containing all the latest entries
115});
116```
117
118Or in anti-callback mode, using a few other options:
119
120```js
121var options = {
122 compressed : true, // sets 'Accept-Encoding' to 'gzip, deflate, br'
123 follow_max : 5, // follow up to five redirects
124 rejectUnauthorized : true // verify SSL certificate
125}
126
127var stream = needle.get('https://backend.server.com/everything.html', options);
128
129// read the chunks from the 'readable' event, so the stream gets consumed.
130stream.on('readable', function() {
131 while (data = this.read()) {
132 console.log(data.toString());
133 }
134})
135
136stream.on('done', function(err) {
137 // if our request had an error, our 'done' event will tell us.
138 if (!err) console.log('Great success!');
139})
140```
141
142API
143---
144
145### needle(method, url[, data][, options][, callback]) `(> 2.0.x)`
146
147Calling `needle()` directly returns a Promise. Besides `method` and `url`, all parameters are optional, although when sending a `post`, `put` or `patch` request you will get an error if `data` is not present.
148
149```js
150needle('get', 'http://some.url.com')
151 .then(function(resp) { console.log(resp.body) })
152 .catch(function(err) { console.error(err) })
153```
154
155Except from the above, all of Needle's request methods return a Readable stream, and both `options` and `callback` are optional. If passed, the callback will return three arguments: `error`, `response` and `body`, which is basically an alias for `response.body`.
156
157### needle.head(url[, options][, callback])
158
159```js
160needle.head('https://my.backend.server.com', {
161 open_timeout: 5000 // if we're not able to open a connection in 5 seconds, boom.
162}, function(err, resp) {
163 if (err)
164 console.log('Shoot! Something is wrong: ' + err.message)
165 else
166 console.log('Yup, still alive.')
167})
168```
169
170### needle.get(url[, options][, callback])
171
172```js
173needle.get('google.com/search?q=syd+barrett', function(err, resp) {
174 // if no http:// is found, Needle will automagically prepend it.
175});
176```
177
178### needle.post(url, data[, options][, callback])
179
180```js
181var options = {
182 headers: { 'X-Custom-Header': 'Bumbaway atuna' }
183}
184
185needle.post('https://my.app.com/endpoint', 'foo=bar', options, function(err, resp) {
186 // you can pass params as a string or as an object.
187});
188```
189
190### needle.put(url, data[, options][, callback])
191
192```js
193var nested = {
194 params: {
195 are: {
196 also: 'supported'
197 }
198 }
199}
200
201needle.put('https://api.app.com/v2', nested, function(err, resp) {
202 console.log('Got ' + resp.bytes + ' bytes.') // another nice treat from this handsome fella.
203});
204```
205
206### needle.patch(url, data[, options][, callback])
207
208Same behaviour as PUT.
209
210### needle.delete(url, data[, options][, callback])
211
212```js
213var options = {
214 username: 'fidelio',
215 password: 'x'
216}
217
218needle.delete('https://api.app.com/messages/123', null, options, function(err, resp) {
219 // in this case, data may be null, but you need to explicity pass it.
220});
221```
222
223### needle.request(method, url, data[, options][, callback])
224
225Generic request. This not only allows for flexibility, but also lets you perform a GET request with data, in which case will be appended to the request as a query string, unless you pass a `json: true` option (read below).
226
227```js
228var params = {
229 q : 'a very smart query',
230 page : 2
231}
232
233needle.request('get', 'forum.com/search', params, function(err, resp) {
234 if (!err && resp.statusCode == 200)
235 console.log(resp.body); // here you go, mister.
236});
237```
238
239Now, if you set pass `json: true` among the options, Needle won't set your params as a querystring but instead send a JSON representation of your data through the request's body, as well as set the `Content-Type` and `Accept` headers to `application/json`.
240
241```js
242needle.request('get', 'forum.com/search', params, { json: true }, function(err, resp) {
243 if (resp.statusCode == 200) console.log('It worked!');
244});
245```
246
247Events
248------
249
250The [Readable stream](https://nodejs.org/api/stream.html#stream_class_stream_readable) object returned by the above request methods emits the following events, in addition to the regular ones (e.g. `end`, `close`, `data`, `pipe`, `readable`).
251
252### Event: `'response'`
253
254 - `response <http.IncomingMessage>`
255
256Emitted when the underlying [http.ClientRequest](https://nodejs.org/api/http.html#http_class_http_clientrequest) emits a response event. This is after the connection is established and the header received, but before any of it is processed (e.g. authorization required or redirect to be followed). No data has been consumed at this point.
257
258### Event: `'redirect'`
259
260 - `location <String>`
261
262Indicates that the a redirect is being followed. This means that the response code was a redirect (`301`, `302`, `303`, `307`) and the given [redirect options](#redirect-options) allowed following the URL received in the `Location` header.
263
264### Event: `'header'`
265
266 - `statusCode <Integer>`
267 - `headers <Object>`
268
269Triggered after the header has been processed, and just before the data is to be consumed. This implies that no redirect was followed and/or authentication header was received. In other words, we got a "valid" response.
270
271### Event: `'done'` (previously 'end')
272
273 - `exception <Error>` (optional)
274
275Emitted when the request/response process has finished, either because all data was consumed or an error ocurred somewhere in between. Unlike a regular stream's `end` event, Needle's `done` will be fired either on success or on failure, which is why the first argument may be an Error object. In other words:
276
277```js
278var resp = needle.get('something.worthy/of/being/streamed/by/needle');
279resp.pipe(someWritableStream);
280
281resp.on('done', function(err) {
282 if (err) console.log('An error ocurred: ' + err.message);
283 else console.log('Great success!');
284})
285```
286
287### Event: `'err'`
288
289 - `exception <Error>`
290
291Emitted when an error ocurrs. This should only happen once in the lifecycle of a Needle request.
292
293### Event: `'timeout'`
294
295 - `type <String>`
296
297Emitted when an timeout error occurs. Type can be either 'open', 'response', or 'read'. This will called right before aborting the request, which will also trigger an `err` event, a described above, with an `ECONNRESET` (Socket hang up) exception.
298
299Request options
300---------------
301
302For information about options that've changed, there's always [the changelog](https://github.com/tomas/needle/releases).
303
304 - `agent` : Uses an [http.Agent](https://nodejs.org/api/http.html#http_class_http_agent) of your choice, instead of the global, default one. Useful for tweaking the behaviour at the connection level, such as when doing tunneling (see below for an example).
305 - `json` : When `true`, sets content type to `application/json` and sends request body as JSON string, instead of a query string.
306 - `open_timeout`: (or `timeout`) Returns error if connection takes longer than X milisecs to establish. Defaults to `10000` (10 secs). `0` means no timeout.
307 - `response_timeout`: Returns error if no response headers are received in X milisecs, counting from when the connection is opened. Defaults to `0` (no response timeout).
308 - `read_timeout`: Returns error if data transfer takes longer than X milisecs, once response headers are received. Defaults to `0` (no timeout).
309 - `follow_max` : (or `follow`) Number of redirects to follow. Defaults to `0`. See below for more redirect options.
310 - `multipart` : Enables multipart/form-data encoding. Defaults to `false`. Use it when uploading files.
311 - `proxy` : Forwards request through HTTP(s) proxy. Eg. `proxy: 'http://user:pass@proxy.server.com:3128'`. For more advanced proxying/tunneling use a custom `agent`, as described below.
312 - `headers` : Object containing custom HTTP headers for request. Overrides defaults described below.
313 - `auth` : Determines what to do with provided username/password. Options are `auto`, `digest` or `basic` (default). `auto` will detect the type of authentication depending on the response headers.
314 - `stream_length`: When sending streams, this lets you manually set the Content-Length header --if the stream's bytecount is known beforehand--, preventing ECONNRESET (socket hang up) errors on some servers that misbehave when receiving payloads of unknown size. Set it to `0` and Needle will get and set the stream's length for you, or leave unset for the default behaviour, which is no Content-Length header for stream payloads.
315 - `localAddress`: <string>, IP address. Passed to http/https request. Local interface from which the request should be emitted.
316 - `uri_modifier`: Anonymous function taking request (or redirect location if following redirects) URI as an argument and modifying it given logic. It has to return a valid URI string for successful request.
317
318Response options
319----------------
320
321 - `decode_response` : (or `decode`) Whether to decode the text responses to UTF-8, if Content-Type header shows a different charset. Defaults to `true`.
322 - `parse_response` : (or `parse`) Whether to parse XML or JSON response bodies automagically. Defaults to `true`. You can also set this to 'xml' or 'json' in which case Needle will *only* parse the response if the content type matches.
323 - `output` : Dump response output to file. This occurs after parsing and charset decoding is done.
324 - `parse_cookies` : Whether to parse response’s `Set-Cookie` header. Defaults to `true`. If parsed, response cookies will be available at `resp.cookies`.
325
326HTTP Header options
327-------------------
328
329These are basically shortcuts to the `headers` option described above.
330
331 - `cookies` : Builds and sets a Cookie header from a `{ key: 'value' }` object.
332 - `compressed`: If `true`, sets 'Accept-Encoding' header to 'gzip,deflate', and inflates content if zipped. Defaults to `false`.
333 - `username` : For HTTP basic auth.
334 - `password` : For HTTP basic auth. Requires username to be passed, but is optional.
335 - `accept` : Sets 'Accept' HTTP header. Defaults to `*/*`.
336 - `connection`: Sets 'Connection' HTTP header. Not set by default, unless running Node < 0.11.4 in which case it defaults to `close`. More info about this below.
337 - `user_agent`: Sets the 'User-Agent' HTTP header. Defaults to `Needle/{version} (Node.js {node_version})`.
338 - `content_type`: Sets the 'Content-Type' header. Unset by default, unless you're sending data in which case it's set accordingly to whatever is being sent (`application/x-www-form-urlencoded`, `application/json` or `multipart/form-data`). That is, of course, unless the option is passed, either here or through `options.headers`. You're the boss.
339
340Node.js TLS Options
341-------------------
342
343These options are passed directly to `https.request` if present. Taken from the [original documentation](http://nodejs.org/docs/latest/api/https.html):
344
345 - `pfx` : Certificate, Private key and CA certificates to use for SSL.
346 - `key` : Private key to use for SSL.
347 - `passphrase` : A string of passphrase for the private key or pfx.
348 - `cert` : Public x509 certificate to use.
349 - `ca` : An authority certificate or array of authority certificates to check the remote host against.
350 - `ciphers` : A string describing the ciphers to use or exclude.
351 - `rejectUnauthorized` : If true, the server certificate is verified against the list of supplied CAs. An 'error' event is emitted if verification fails. Verification happens at the connection level, before the HTTP request is sent.
352 - `secureProtocol` : The SSL method to use, e.g. SSLv3_method to force SSL version 3.
353 - `family` : IP address family to use when resolving host and hostname. Valid values are 4 or 6. When unspecified, both IP v4 and v6 will be used.
354
355Redirect options
356----------------
357
358These options only apply if the `follow_max` (or `follow`) option is higher than 0.
359
360 - `follow_set_cookies` : Sends the cookies received in the `set-cookie` header as part of the following request. `false` by default.
361 - `follow_set_referer` : Sets the 'Referer' header to the requested URI when following a redirect. `false` by default.
362 - `follow_keep_method` : If enabled, resends the request using the original verb instead of being rewritten to `get` with no data. `false` by default.
363 - `follow_if_same_host` : When true, Needle will only follow redirects that point to the same host as the original request. `false` by default.
364 - `follow_if_same_protocol` : When true, Needle will only follow redirects that point to the same protocol as the original request. `false` by default.
365 - `follow_if_same_location` : Unless true, Needle will not follow redirects that point to same location (as set in the response header) as the original request URL. `false` by default.
366
367Overriding Defaults
368-------------------
369
370Yes sir, we have it. Needle includes a `defaults()` method, that lets you override some of the defaults for all future requests. Like this:
371
372```js
373needle.defaults({
374 open_timeout: 60000,
375 user_agent: 'MyApp/1.2.3',
376 parse_response: false });
377```
378
379This will override Needle's default user agent and 10-second timeout, and disable response parsing, so you don't need to pass those options in every other request.
380
381More advanced Proxy support
382---------------------------
383
384Since you can pass a custom HTTPAgent to Needle you can do all sorts of neat stuff. For example, if you want to use the [`tunnel`](https://github.com/koichik/node-tunnel) module for HTTPS proxying, you can do this:
385
386```js
387var tunnel = require('tunnel');
388var myAgent = tunnel.httpOverHttp({
389 proxy: { host: 'localhost' }
390});
391
392needle.get('foobar.com', { agent: myAgent });
393```
394
395Otherwise, you can use the [`hpagent`](https://github.com/delvedor/hpagent) package, which keeps the internal sockets alive to be reused.
396
397```js
398const { HttpsProxyAgent } = require('hpagent');
399needle('get', 'https://localhost:9200', {
400 agent: new HttpsProxyAgent({
401 keepAlive: true,
402 keepAliveMsecs: 1000,
403 maxSockets: 256,
404 maxFreeSockets: 256,
405 scheduling: 'lifo',
406 proxy: 'https://localhost:8080'
407 })
408});
409```
410
411Regarding the 'Connection' header
412---------------------------------
413
414Unless you're running an old version of Node (< 0.11.4), by default Needle won't set the Connection header on requests, yielding Node's default behaviour of keeping the connection alive with the target server. This speeds up immensely the process of sending several requests to the same host.
415
416On older versions, however, this has the unwanted behaviour of preventing the runtime from exiting, either because of a bug or 'feature' that was changed on 0.11.4. To overcome this Needle does set the 'Connection' header to 'close' on those versions, however this also means that making new requests to the same host doesn't benefit from Keep-Alive.
417
418So if you're stuck on 0.10 or even lower and want full speed, you can simply set the Connection header to 'Keep-Alive' by using `{ connection: 'Keep-Alive' }`. Please note, though, that an event loop handler will prevent the runtime from exiting so you'll need to manually call `process.exit()` or the universe will collapse.
419
420Examples Galore
421---------------
422
423### HTTPS GET with Basic Auth
424
425```js
426needle.get('https://api.server.com', { username: 'you', password: 'secret' },
427 function(err, resp) {
428 // used HTTP auth
429});
430```
431
432Or use [RFC-1738](http://tools.ietf.org/html/rfc1738#section-3.1) basic auth URL syntax:
433
434```js
435needle.get('https://username:password@api.server.com', function(err, resp) {
436 // used HTTP auth from URL
437});
438```
439
440### Digest Auth
441
442```js
443needle.get('other.server.com', { username: 'you', password: 'secret', auth: 'digest' },
444 function(err, resp, body) {
445 // needle prepends 'http://' to your URL, if missing
446});
447```
448
449### Custom Accept header, deflate
450
451```js
452var options = {
453 compressed : true,
454 follow : 10,
455 accept : 'application/vnd.github.full+json'
456}
457
458needle.get('api.github.com/users/tomas', options, function(err, resp, body) {
459 // body will contain a JSON.parse(d) object
460 // if parsing fails, you'll simply get the original body
461});
462```
463
464### GET XML object
465
466```js
467needle.get('https://news.ycombinator.com/rss', function(err, resp, body) {
468 // you'll get a nice object containing the nodes in the RSS
469});
470```
471
472### GET binary, output to file
473
474```js
475needle.get('http://upload.server.com/tux.png', { output: '/tmp/tux.png' }, function(err, resp, body) {
476 // you can dump any response to a file, not only binaries.
477});
478```
479
480### GET through proxy
481
482```js
483needle.get('http://search.npmjs.org', { proxy: 'http://localhost:1234' }, function(err, resp, body) {
484 // request passed through proxy
485});
486```
487
488### GET a very large document in a stream (from 0.7+)
489
490```js
491var stream = needle.get('http://www.as35662.net/100.log');
492
493stream.on('readable', function() {
494 var chunk;
495 while (chunk = this.read()) {
496 console.log('got data: ', chunk);
497 }
498});
499```
500
501### GET JSON object in a stream (from 0.7+)
502
503```js
504var stream = needle.get('http://jsonplaceholder.typicode.com/db', { parse: true });
505
506stream.on('readable', function() {
507 var node;
508
509 // our stream will only emit a single JSON root node.
510 while (node = this.read()) {
511 console.log('got data: ', node);
512 }
513});
514```
515
516### GET JSONStream flexible parser with search query (from 0.7+)
517
518```js
519
520 // The 'data' element of this stream will be the string representation
521 // of the titles of all posts.
522
523needle.get('http://jsonplaceholder.typicode.com/db', { parse: true })
524 .pipe(new JSONStream.parse('posts.*.title'));
525 .on('data', function (obj) {
526 console.log('got post title: %s', obj);
527 });
528```
529
530### File upload using multipart, passing file path
531
532```js
533var data = {
534 foo: 'bar',
535 image: { file: '/home/tomas/linux.png', content_type: 'image/png' }
536}
537
538needle.post('http://my.other.app.com', data, { multipart: true }, function(err, resp, body) {
539 // needle will read the file and include it in the form-data as binary
540});
541```
542
543### Stream upload, PUT or POST
544
545``` js
546needle.put('https://api.app.com/v2', fs.createReadStream('myfile.txt'), function(err, resp, body) {
547 // stream content is uploaded verbatim
548});
549```
550
551### Multipart POST, passing data buffer
552
553```js
554var buffer = fs.readFileSync('/path/to/package.zip');
555
556var data = {
557 zip_file: {
558 buffer : buffer,
559 filename : 'mypackage.zip',
560 content_type : 'application/octet-stream'
561 }
562}
563
564needle.post('http://somewhere.com/over/the/rainbow', data, { multipart: true }, function(err, resp, body) {
565 // if you see, when using buffers we need to pass the filename for the multipart body.
566 // you can also pass a filename when using the file path method, in case you want to override
567 // the default filename to be received on the other end.
568});
569```
570
571### Multipart with custom Content-Type
572
573```js
574var data = {
575 token: 'verysecret',
576 payload: {
577 value: JSON.stringify({ title: 'test', version: 1 }),
578 content_type: 'application/json'
579 }
580}
581
582needle.post('http://test.com/', data, { timeout: 5000, multipart: true }, function(err, resp, body) {
583 // in this case, if the request takes more than 5 seconds
584 // the callback will return a [Socket closed] error
585});
586```
587
588For even more examples, check out the examples directory in the repo.
589
590### Testing
591
592To run tests, you need to generate a self-signed SSL certificate in the `test` directory. After cloning the repository, run the following commands:
593
594 $ mkdir -p test/keys
595 $ openssl genrsa -out test/keys/ssl.key 2048
596 $ openssl req -new -key test/keys/ssl.key -x509 -days 999 -out test/keys/ssl.cert
597
598Then you should be able to run `npm test` once you have the dependencies in place.
599
600> Note: Tests currently only work on linux-based environments that have `/proc/self/fd`. They *do not* work on MacOS environments.
601> You can use Docker to run tests by creating a container and mounting the needle project directory on `/app`
602> `docker create --name Needle -v /app -w /app -v /app/node_modules -i node:argon`
603
604Credits
605-------
606
607Written by Tomás Pollak, with the help of contributors.
608
609Copyright
610---------
611
612(c) Fork Ltd. Licensed under the MIT license.
Note: See TracBrowser for help on using the repository browser.