| 1 | # proxy-from-env
|
|---|
| 2 |
|
|---|
| 3 | 
|
|---|
| 4 | [](https://coveralls.io/github/Rob--W/proxy-from-env?branch=master)
|
|---|
| 5 |
|
|---|
| 6 | `proxy-from-env` is a Node.js package that exports a function (`getProxyForUrl`)
|
|---|
| 7 | that takes an input URL (a string, an instance of
|
|---|
| 8 | [`URL`](https://nodejs.org/docs/latest/api/url.html#the-whatwg-url-api),
|
|---|
| 9 | or [`url.parse`](https://nodejs.org/docs/latest/api/url.html#url_url_parsing)'s
|
|---|
| 10 | return value) and returns the desired proxy URL (also a string) based on
|
|---|
| 11 | standard proxy environment variables. If no proxy is set, an empty string is
|
|---|
| 12 | returned.
|
|---|
| 13 |
|
|---|
| 14 | If your application makes important (security) decisions based on the URL, be
|
|---|
| 15 | consistent in the mechanism to parse and validate URLs, as differences in URL
|
|---|
| 16 | parsing behavior can affect the outcome of proxy resolution.
|
|---|
| 17 | Strings are parsed with the standard `URL` API, as of `proxy-from-env@2.0.0`.
|
|---|
| 18 | Older versions relied on the (now deprecated) `url.parse` method instead.
|
|---|
| 19 |
|
|---|
| 20 | Invalid values in environment variables are not handled by the library
|
|---|
| 21 | ([#41](https://github.com/Rob--W/proxy-from-env/issues/41)).
|
|---|
| 22 |
|
|---|
| 23 | It is your responsibility to actually proxy the request using the given URL.
|
|---|
| 24 |
|
|---|
| 25 | Installation:
|
|---|
| 26 |
|
|---|
| 27 | ```sh
|
|---|
| 28 | npm install proxy-from-env
|
|---|
| 29 | ```
|
|---|
| 30 |
|
|---|
| 31 | ## Example
|
|---|
| 32 | This example shows how the data for a URL can be fetched via the
|
|---|
| 33 | [`http` module](https://nodejs.org/api/http.html), in a proxy-aware way.
|
|---|
| 34 |
|
|---|
| 35 | warning: this simple example works for http requests only. To support https,
|
|---|
| 36 | you must establish a proxy tunnel via the
|
|---|
| 37 | [http `connect` method](https://developer.mozilla.org/en-us/docs/web/http/reference/methods/connect).
|
|---|
| 38 |
|
|---|
| 39 | ```javascript
|
|---|
| 40 | import http from 'node:test';
|
|---|
| 41 | import { getProxyForUrl } from 'proxy-from-env';
|
|---|
| 42 | // ^ or: var getProxyForUrl = require('proxy-from-env').getProxyForUrl;
|
|---|
| 43 |
|
|---|
| 44 | var some_url = 'http://example.com/something';
|
|---|
| 45 |
|
|---|
| 46 | // // Example, if there is a proxy server at 10.0.0.1:1234, then setting the
|
|---|
| 47 | // // http_proxy environment variable causes the request to go through a proxy.
|
|---|
| 48 | // process.env.http_proxy = 'http://10.0.0.1:1234';
|
|---|
| 49 | //
|
|---|
| 50 | // // But if the host to be proxied is listed in NO_PROXY, then the request is
|
|---|
| 51 | // // not proxied (but a direct request is made).
|
|---|
| 52 | // process.env.no_proxy = 'example.com';
|
|---|
| 53 |
|
|---|
| 54 | var proxy_url = getProxyForUrl(some_url); // <-- Our magic.
|
|---|
| 55 | if (proxy_url) {
|
|---|
| 56 | // Should be proxied through proxy_url.
|
|---|
| 57 | var parsed_some_url = new URL(some_url);
|
|---|
| 58 | var parsed_proxy_url = new URL(proxy_url);
|
|---|
| 59 | // A HTTP proxy is quite simple. It is similar to a normal request, except the
|
|---|
| 60 | // path is an absolute URL, and the proxied URL's host is put in the header
|
|---|
| 61 | // instead of the server's actual host.
|
|---|
| 62 | httpOptions = {
|
|---|
| 63 | protocol: parsed_proxy_url.protocol,
|
|---|
| 64 | hostname: parsed_proxy_url.hostname,
|
|---|
| 65 | port: parsed_proxy_url.port,
|
|---|
| 66 | path: parsed_some_url.href,
|
|---|
| 67 | headers: {
|
|---|
| 68 | Host: parsed_some_url.host, // = host name + optional port.
|
|---|
| 69 | },
|
|---|
| 70 | };
|
|---|
| 71 | } else {
|
|---|
| 72 | // Direct request.
|
|---|
| 73 | httpOptions = some_url;
|
|---|
| 74 | }
|
|---|
| 75 | http.get(httpOptions, function(res) {
|
|---|
| 76 | var responses = [];
|
|---|
| 77 | res.on('data', function(chunk) { responses.push(chunk); });
|
|---|
| 78 | res.on('end', function() { console.log(responses.join('')); });
|
|---|
| 79 | });
|
|---|
| 80 | ```
|
|---|
| 81 |
|
|---|
| 82 | ### Full proxy support
|
|---|
| 83 | The simple example above works for http requests only. To support https, you
|
|---|
| 84 | must establish a proxy tunnel via the
|
|---|
| 85 | [http `connect` method](https://developer.mozilla.org/en-us/docs/web/http/reference/methods/connect).
|
|---|
| 86 |
|
|---|
| 87 | An example of that is shown in the
|
|---|
| 88 | [`https-proxy-agent` npm package](https://www.npmjs.com/package/https-proxy-agent).
|
|---|
| 89 | The [`proxy-agent` npm package](https://www.npmjs.com/package/proxy-agent)
|
|---|
| 90 | combines `https-proxy-agent` and `proxy-from-env` to offer a `http.Agent` that
|
|---|
| 91 | supports proxies from environment variables.
|
|---|
| 92 |
|
|---|
| 93 | ### Built-in proxy support
|
|---|
| 94 | Node.js is working on built-in support for proxy environment variables,
|
|---|
| 95 | currently behind `NODE_USE_ENV_PROXY=1` or `--use-env-proxy`. For details, see:
|
|---|
| 96 |
|
|---|
| 97 | - https://github.com/nodejs/node/issues/57872
|
|---|
| 98 | - https://nodejs.org/api/http.html#built-in-proxy-support
|
|---|
| 99 |
|
|---|
| 100 |
|
|---|
| 101 | ## Environment variables
|
|---|
| 102 | The environment variables can be specified in all lowercase or all uppercase,
|
|---|
| 103 | with lowercase taking precedence over the uppercase variant. A variable that is
|
|---|
| 104 | not set has the same meaning as a variable that is set but has no value.
|
|---|
| 105 |
|
|---|
| 106 | ### NO\_PROXY
|
|---|
| 107 |
|
|---|
| 108 | `NO_PROXY` is a list of host names (optionally with a port). If the input URL
|
|---|
| 109 | matches any of the entries in `NO_PROXY`, then the input URL should be fetched
|
|---|
| 110 | by a direct request (i.e. without a proxy).
|
|---|
| 111 |
|
|---|
| 112 | Matching follows the following rules:
|
|---|
| 113 |
|
|---|
| 114 | - `NO_PROXY=*` disables all proxies.
|
|---|
| 115 | - Space and commas may be used to separate the entries in the `NO_PROXY` list.
|
|---|
| 116 | - If `NO_PROXY` does not contain any entries, then proxies are never disabled.
|
|---|
| 117 | - If a port is added after the host name, then the ports must match. If the URL
|
|---|
| 118 | does not have an explicit port name, the protocol's default port is used.
|
|---|
| 119 | - Generally, the proxy is only disabled if the host name is an exact match for
|
|---|
| 120 | an entry in the `NO_PROXY` list. The only exceptions are entries that start
|
|---|
| 121 | with a dot or with a wildcard; then the proxy is disabled if the host name
|
|---|
| 122 | ends with the entry.
|
|---|
| 123 |
|
|---|
| 124 | See `test.js` for examples of what should match and what does not.
|
|---|
| 125 |
|
|---|
| 126 | ### \*\_PROXY
|
|---|
| 127 |
|
|---|
| 128 | The environment variable used for the proxy depends on the protocol of the URL.
|
|---|
| 129 | For example, `https://example.com` uses the "https" protocol, and therefore the
|
|---|
| 130 | proxy to be used is `HTTPS_PROXY` (_NOT_ `HTTP_PROXY`, which is _only_ used for
|
|---|
| 131 | http:-URLs).
|
|---|
| 132 |
|
|---|
| 133 | The library is not limited to http(s), other schemes such as
|
|---|
| 134 | `FTP_PROXY` (ftp:),
|
|---|
| 135 | `WSS_PROXY` (wss:),
|
|---|
| 136 | `WS_PROXY` (ws:)
|
|---|
| 137 | are also supported.
|
|---|
| 138 |
|
|---|
| 139 | If present, `ALL_PROXY` is used as fallback if there is no other match.
|
|---|
| 140 |
|
|---|
| 141 |
|
|---|
| 142 | ## External resources
|
|---|
| 143 | The exact way of parsing the environment variables is not codified in any
|
|---|
| 144 | standard. This library is designed to be compatible with formats as expected by
|
|---|
| 145 | existing software.
|
|---|
| 146 | The following resources were used to determine the desired behavior:
|
|---|
| 147 |
|
|---|
| 148 | - cURL:
|
|---|
| 149 | https://curl.haxx.se/docs/manpage.html#ENVIRONMENT
|
|---|
| 150 | https://github.com/curl/curl/blob/4af40b3646d3b09f68e419f7ca866ff395d1f897/lib/url.c#L4446-L4514
|
|---|
| 151 | https://github.com/curl/curl/blob/4af40b3646d3b09f68e419f7ca866ff395d1f897/lib/url.c#L4608-L4638
|
|---|
| 152 |
|
|---|
| 153 | - wget:
|
|---|
| 154 | https://www.gnu.org/software/wget/manual/wget.html#Proxies
|
|---|
| 155 | http://git.savannah.gnu.org/cgit/wget.git/tree/src/init.c?id=636a5f9a1c508aa39e35a3a8e9e54520a284d93d#n383
|
|---|
| 156 | http://git.savannah.gnu.org/cgit/wget.git/tree/src/retr.c?id=93c1517c4071c4288ba5a4b038e7634e4c6b5482#n1278
|
|---|
| 157 |
|
|---|
| 158 | - W3:
|
|---|
| 159 | https://www.w3.org/Daemon/User/Proxies/ProxyClients.html
|
|---|
| 160 |
|
|---|
| 161 | - Python's urllib:
|
|---|
| 162 | https://github.com/python/cpython/blob/936135bb97fe04223aa30ca6e98eac8f3ed6b349/Lib/urllib/request.py#L755-L782
|
|---|
| 163 | https://github.com/python/cpython/blob/936135bb97fe04223aa30ca6e98eac8f3ed6b349/Lib/urllib/request.py#L2444-L2479
|
|---|