source: trip-planner-front/node_modules/jest-worker/README.md@ bdd6491

Last change on this file since bdd6491 was 6a3a178, checked in by Ema <ema_spirova@…>, 3 years ago

initial commit

  • Property mode set to 100644
File size: 11.3 KB
Line 
1# jest-worker
2
3Module for executing heavy tasks under forked processes in parallel, by providing a `Promise` based interface, minimum overhead, and bound workers.
4
5The module works by providing an absolute path of the module to be loaded in all forked processes. Files relative to a node module are also accepted. All methods are exposed on the parent process as promises, so they can be `await`'ed. Child (worker) methods can either be synchronous or asynchronous.
6
7The module also implements support for bound workers. Binding a worker means that, based on certain parameters, the same task will always be executed by the same worker. The way bound workers work is by using the returned string of the `computeWorkerKey` method. If the string was used before for a task, the call will be queued to the related worker that processed the task earlier; if not, it will be executed by the first available worker, then sticked to the worker that executed it; so the next time it will be processed by the same worker. If you have no preference on the worker executing the task, but you have defined a `computeWorkerKey` method because you want _some_ of the tasks to be sticked, you can return `null` from it.
8
9The list of exposed methods can be explicitly provided via the `exposedMethods` option. If it is not provided, it will be obtained by requiring the child module into the main process, and analyzed via reflection. Check the "minimal example" section for a valid one.
10
11## Install
12
13```sh
14$ yarn add jest-worker
15```
16
17## Example
18
19This example covers the minimal usage:
20
21### File `parent.js`
22
23```javascript
24import {Worker as JestWorker} from 'jest-worker';
25
26async function main() {
27 const worker = new JestWorker(require.resolve('./Worker'));
28 const result = await worker.hello('Alice'); // "Hello, Alice"
29}
30
31main();
32```
33
34### File `worker.js`
35
36```javascript
37export function hello(param) {
38 return 'Hello, ' + param;
39}
40```
41
42## Experimental worker
43
44Node 10 shipped with [worker-threads](https://nodejs.org/api/worker_threads.html), a "threading API" that uses SharedArrayBuffers to communicate between the main process and its child threads. This experimental Node feature can significantly improve the communication time between parent and child processes in `jest-worker`.
45
46Since `worker_threads` are considered experimental in Node, you have to opt-in to this behavior by passing `enableWorkerThreads: true` when instantiating the worker. While the feature was unflagged in Node 11.7.0, you'll need to run the Node process with the `--experimental-worker` flag for Node 10.
47
48## API
49
50The `Worker` export is a constructor that is initialized by passing the worker path, plus an options object.
51
52### `workerPath: string` (required)
53
54Node module name or absolute path of the file to be loaded in the child processes. Use `require.resolve` to transform a relative path into an absolute one.
55
56### `options: Object` (optional)
57
58#### `exposedMethods: $ReadOnlyArray<string>` (optional)
59
60List of method names that can be called on the child processes from the parent process. You cannot expose any method named like a public `Worker` method, or starting with `_`. If you use method auto-discovery, then these methods will not be exposed, even if they exist.
61
62#### `numWorkers: number` (optional)
63
64Amount of workers to spawn. Defaults to the number of CPUs minus 1.
65
66#### `maxRetries: number` (optional)
67
68Maximum amount of times that a dead child can be re-spawned, per call. Defaults to `3`, pass `Infinity` to allow endless retries.
69
70#### `forkOptions: Object` (optional)
71
72Allow customizing all options passed to `childProcess.fork`. By default, some values are set (`cwd`, `env` and `execArgv`), but you can override them and customize the rest. For a list of valid values, check [the Node documentation](https://nodejs.org/api/child_process.html#child_process_child_process_fork_modulepath_args_options).
73
74#### `computeWorkerKey: (method: string, ...args: Array<any>) => ?string` (optional)
75
76Every time a method exposed via the API is called, `computeWorkerKey` is also called in order to bound the call to a worker. This is useful for workers that are able to cache the result or part of it. You bound calls to a worker by making `computeWorkerKey` return the same identifier for all different calls. If you do not want to bind the call to any worker, return `null`.
77
78The callback you provide is called with the method name, plus all the rest of the arguments of the call. Thus, you have full control to decide what to return. Check a practical example on bound workers under the "bound worker usage" section.
79
80By default, no process is bound to any worker.
81
82#### `setupArgs: Array<mixed>` (optional)
83
84The arguments that will be passed to the `setup` method during initialization.
85
86#### `WorkerPool: (workerPath: string, options?: WorkerPoolOptions) => WorkerPoolInterface` (optional)
87
88Provide a custom worker pool to be used for spawning child processes. By default, Jest will use a node thread pool if available and fall back to child process threads.
89
90#### `enableWorkerThreads: boolean` (optional)
91
92`jest-worker` will automatically detect if `worker_threads` are available, but will not use them unless passed `enableWorkerThreads: true`.
93
94### `workerSchedulingPolicy: 'round-robin' | 'in-order'` (optional)
95
96Specifies the policy how tasks are assigned to workers if multiple workers are _idle_:
97
98- `round-robin` (default): The task will be sequentially distributed onto the workers. The first task is assigned to the worker 1, the second to the worker 2, to ensure that the work is distributed across workers.
99- `in-order`: The task will be assigned to the first free worker starting with worker 1 and only assign the work to worker 2 if the worker 1 is busy.
100
101Tasks are always assigned to the first free worker as soon as tasks start to queue up. The scheduling policy does not define the task scheduling which is always first-in, first-out.
102
103### `taskQueue`: TaskQueue` (optional)
104
105The task queue defines in which order tasks (method calls) are processed by the workers. `jest-worker` ships with a `FifoQueue` and `PriorityQueue`:
106
107- `FifoQueue` (default): Processes the method calls (tasks) in the call order.
108- `PriorityQueue`: Processes the method calls by a computed priority in natural ordering (lower priorities first). Tasks with the same priority are processed in any order (FIFO not guaranteed). The constructor accepts a single argument, the function that is passed the name of the called function and the arguments and returns a numerical value for the priority: `new require('jest-worker').PriorityQueue((method, filename) => filename.length)`.
109
110## JestWorker
111
112### Methods
113
114The returned `JestWorker` instance has all the exposed methods, plus some additional ones to interact with the workers itself:
115
116#### `getStdout(): Readable`
117
118Returns a `ReadableStream` where the standard output of all workers is piped. Note that the `silent` option of the child workers must be set to `true` to make it work. This is the default set by `jest-worker`, but keep it in mind when overriding options through `forkOptions`.
119
120#### `getStderr(): Readable`
121
122Returns a `ReadableStream` where the standard error of all workers is piped. Note that the `silent` option of the child workers must be set to `true` to make it work. This is the default set by `jest-worker`, but keep it in mind when overriding options through `forkOptions`.
123
124#### `end()`
125
126Finishes the workers by killing all workers. No further calls can be done to the `Worker` instance.
127
128Returns a Promise that resolves with `{ forceExited: boolean }` once all workers are dead. If `forceExited` is `true`, at least one of the workers did not exit gracefully, which likely happened because it executed a leaky task that left handles open. This should be avoided, force exiting workers is a last resort to prevent creating lots of orphans.
129
130**Note:**
131
132`await`ing the `end()` Promise immediately after the workers are no longer needed before proceeding to do other useful things in your program may not be a good idea. If workers have to be force exited, `jest-worker` may go through multiple stages of force exiting (e.g. SIGTERM, later SIGKILL) and give the worker overall around 1 second time to exit on its own. During this time, your program will wait, even though it may not be necessary that all workers are dead before continuing execution.
133
134Consider deliberately leaving this Promise floating (unhandled resolution). After your program has done the rest of its work and is about to exit, the Node process will wait for the Promise to resolve after all workers are dead as the last event loop task. That way you parallelized computation time of your program and waiting time and you didn't delay the outputs of your program unnecessarily.
135
136### Worker IDs
137
138Each worker has a unique id (index that starts with `1`), which is available inside the worker as `process.env.JEST_WORKER_ID`.
139
140## Setting up and tearing down the child process
141
142The child process can define two special methods (both of them can be asynchronous):
143
144- `setup()`: If defined, it's executed before the first call to any method in the child.
145- `teardown()`: If defined, it's executed when the farm ends.
146
147# More examples
148
149## Standard usage
150
151This example covers the standard usage:
152
153### File `parent.js`
154
155```javascript
156import {Worker as JestWorker} from 'jest-worker';
157
158async function main() {
159 const myWorker = new JestWorker(require.resolve('./Worker'), {
160 exposedMethods: ['foo', 'bar', 'getWorkerId'],
161 numWorkers: 4,
162 });
163
164 console.log(await myWorker.foo('Alice')); // "Hello from foo: Alice"
165 console.log(await myWorker.bar('Bob')); // "Hello from bar: Bob"
166 console.log(await myWorker.getWorkerId()); // "3" -> this message has sent from the 3rd worker
167
168 const {forceExited} = await myWorker.end();
169 if (forceExited) {
170 console.error('Workers failed to exit gracefully');
171 }
172}
173
174main();
175```
176
177### File `worker.js`
178
179```javascript
180export function foo(param) {
181 return 'Hello from foo: ' + param;
182}
183
184export function bar(param) {
185 return 'Hello from bar: ' + param;
186}
187
188export function getWorkerId() {
189 return process.env.JEST_WORKER_ID;
190}
191```
192
193## Bound worker usage:
194
195This example covers the usage with a `computeWorkerKey` method:
196
197### File `parent.js`
198
199```javascript
200import {Worker as JestWorker} from 'jest-worker';
201
202async function main() {
203 const myWorker = new JestWorker(require.resolve('./Worker'), {
204 computeWorkerKey: (method, filename) => filename,
205 });
206
207 // Transform the given file, within the first available worker.
208 console.log(await myWorker.transform('/tmp/foo.js'));
209
210 // Wait a bit.
211 await sleep(10000);
212
213 // Transform the same file again. Will immediately return because the
214 // transformed file is cached in the worker, and `computeWorkerKey` ensures
215 // the same worker that processed the file the first time will process it now.
216 console.log(await myWorker.transform('/tmp/foo.js'));
217
218 const {forceExited} = await myWorker.end();
219 if (forceExited) {
220 console.error('Workers failed to exit gracefully');
221 }
222}
223
224main();
225```
226
227### File `worker.js`
228
229```javascript
230import babel from '@babel/core';
231
232const cache = Object.create(null);
233
234export function transform(filename) {
235 if (cache[filename]) {
236 return cache[filename];
237 }
238
239 // jest-worker can handle both immediate results and thenables. If a
240 // thenable is returned, it will be await'ed until it resolves.
241 return babel.transformFileAsync(filename).then(result => {
242 cache[filename] = result;
243
244 return result;
245 });
246}
247```
Note: See TracBrowser for help on using the repository browser.