[6a3a178] | 1 | ![Piscina Logo](https://avatars1.githubusercontent.com/u/65627548?s=200&v=4)
|
---|
| 2 |
|
---|
| 3 | # piscina - the node.js worker pool
|
---|
| 4 |
|
---|
| 5 | ![CI](https://github.com/jasnell/piscina/workflows/CI/badge.svg)
|
---|
| 6 |
|
---|
| 7 | * ✔ Fast communication between threads
|
---|
| 8 | * ✔ Covers both fixed-task and variable-task scenarios
|
---|
| 9 | * ✔ Supports flexible pool sizes
|
---|
| 10 | * ✔ Proper async tracking integration
|
---|
| 11 | * ✔ Tracking statistics for run and wait times
|
---|
| 12 | * ✔ Cancellation Support
|
---|
| 13 | * ✔ Supports enforcing memory resource limits
|
---|
| 14 | * ✔ Supports CommonJS, ESM, and TypeScript
|
---|
| 15 | * ✔ Custom task queues
|
---|
| 16 | * ✔ Optional CPU scheduling priorities on Linux
|
---|
| 17 |
|
---|
| 18 | Written in TypeScript.
|
---|
| 19 |
|
---|
| 20 | For Node.js 12.x and higher.
|
---|
| 21 |
|
---|
| 22 | [MIT Licensed][].
|
---|
| 23 |
|
---|
| 24 | ## Piscina API
|
---|
| 25 |
|
---|
| 26 | ### Example
|
---|
| 27 |
|
---|
| 28 | In `main.js`:
|
---|
| 29 |
|
---|
| 30 | ```js
|
---|
| 31 | const path = require('path');
|
---|
| 32 | const Piscina = require('piscina');
|
---|
| 33 |
|
---|
| 34 | const piscina = new Piscina({
|
---|
| 35 | filename: path.resolve(__dirname, 'worker.js')
|
---|
| 36 | });
|
---|
| 37 |
|
---|
| 38 | (async function() {
|
---|
| 39 | const result = await piscina.run({ a: 4, b: 6 });
|
---|
| 40 | console.log(result); // Prints 10
|
---|
| 41 | })();
|
---|
| 42 | ```
|
---|
| 43 |
|
---|
| 44 | In `worker.js`:
|
---|
| 45 |
|
---|
| 46 | ```js
|
---|
| 47 | module.exports = ({ a, b }) => {
|
---|
| 48 | return a + b;
|
---|
| 49 | };
|
---|
| 50 | ```
|
---|
| 51 |
|
---|
| 52 | The worker may also be an async function or may return a Promise:
|
---|
| 53 |
|
---|
| 54 | ```js
|
---|
| 55 | const { promisify } = require('util');
|
---|
| 56 |
|
---|
| 57 | // Awaitable timers are available in Node.js 15.x+
|
---|
| 58 | // For Node.js 12 and 14, use promisify(setTimeout)
|
---|
| 59 | const { setTimeout } = require('timers/promises');
|
---|
| 60 |
|
---|
| 61 | module.exports = async ({ a, b }) => {
|
---|
| 62 | // Fake some async activity
|
---|
| 63 | await setTimeout(100);
|
---|
| 64 | return a + b;
|
---|
| 65 | };
|
---|
| 66 | ```
|
---|
| 67 |
|
---|
| 68 | ESM is also supported for both Piscina and workers:
|
---|
| 69 |
|
---|
| 70 | ```js
|
---|
| 71 | import { Piscina } from 'piscina';
|
---|
| 72 |
|
---|
| 73 | const piscina = new Piscina({
|
---|
| 74 | // The URL must be a file:// URL
|
---|
| 75 | filename: new URL('./worker.mjs', import.meta.url).href
|
---|
| 76 | });
|
---|
| 77 |
|
---|
| 78 | const result = await piscina.run({ a: 4, b: 6 });
|
---|
| 79 | console.log(result); // Prints 10
|
---|
| 80 | ```
|
---|
| 81 |
|
---|
| 82 | In `worker.mjs`:
|
---|
| 83 |
|
---|
| 84 | ```js
|
---|
| 85 | export default ({ a, b }) => {
|
---|
| 86 | return a + b;
|
---|
| 87 | };
|
---|
| 88 | ```
|
---|
| 89 |
|
---|
| 90 | ### Exporting multiple worker functions
|
---|
| 91 |
|
---|
| 92 | A single worker file may export multiple named handler functions.
|
---|
| 93 |
|
---|
| 94 | ```js
|
---|
| 95 | 'use strict';
|
---|
| 96 |
|
---|
| 97 | function add({ a, b }) { return a + b; }
|
---|
| 98 |
|
---|
| 99 | function multiply({ a, b }) { return a * b; }
|
---|
| 100 |
|
---|
| 101 | add.add = add;
|
---|
| 102 | add.multiply = multiply;
|
---|
| 103 |
|
---|
| 104 | module.exports = add;
|
---|
| 105 | ```
|
---|
| 106 |
|
---|
| 107 | The export to target can then be specified when the task is submitted:
|
---|
| 108 |
|
---|
| 109 | ```js
|
---|
| 110 | 'use strict';
|
---|
| 111 |
|
---|
| 112 | const Piscina = require('piscina');
|
---|
| 113 | const { resolve } = require('path');
|
---|
| 114 |
|
---|
| 115 | const piscina = new Piscina({
|
---|
| 116 | filename: resolve(__dirname, 'worker.js')
|
---|
| 117 | });
|
---|
| 118 |
|
---|
| 119 | (async function() {
|
---|
| 120 | const res = await Promise.all([
|
---|
| 121 | piscina.run({ a: 4, b: 6 }, { name: 'add' }),
|
---|
| 122 | piscina.run({ a: 4, b: 6 }, { name: 'multiply' })
|
---|
| 123 | ]);
|
---|
| 124 | })();
|
---|
| 125 | ```
|
---|
| 126 |
|
---|
| 127 | ### Cancelable Tasks
|
---|
| 128 |
|
---|
| 129 | Submitted tasks may be canceled using either an `AbortController` or
|
---|
| 130 | an `EventEmitter`:
|
---|
| 131 |
|
---|
| 132 | ```js
|
---|
| 133 | 'use strict';
|
---|
| 134 |
|
---|
| 135 | const Piscina = require('piscina');
|
---|
| 136 | const { AbortController } = require('abort-controller');
|
---|
| 137 | const { resolve } = require('path');
|
---|
| 138 |
|
---|
| 139 | const piscina = new Piscina({
|
---|
| 140 | filename: resolve(__dirname, 'worker.js')
|
---|
| 141 | });
|
---|
| 142 |
|
---|
| 143 | (async function() {
|
---|
| 144 | const abortController = new AbortController();
|
---|
| 145 | try {
|
---|
| 146 | const { signal } = abortController;
|
---|
| 147 | const task = piscina.run({ a: 4, b: 6 }, { signal });
|
---|
| 148 | abortController.abort();
|
---|
| 149 | await task;
|
---|
| 150 | } catch (err) {
|
---|
| 151 | console.log('The task was canceled');
|
---|
| 152 | }
|
---|
| 153 | })();
|
---|
| 154 | ```
|
---|
| 155 |
|
---|
| 156 | To use `AbortController`, you will need to `npm i abort-controller`
|
---|
| 157 | (or `yarn add abort-controller`).
|
---|
| 158 |
|
---|
| 159 | (In Node.js 15.0.0 or higher, there is a new built-in `AbortController`
|
---|
| 160 | implementation that can be used here as well.)
|
---|
| 161 |
|
---|
| 162 | Alternatively, any `EventEmitter` that emits an `'abort'` event
|
---|
| 163 | may be used as an abort controller:
|
---|
| 164 |
|
---|
| 165 | ```js
|
---|
| 166 | 'use strict';
|
---|
| 167 |
|
---|
| 168 | const Piscina = require('piscina');
|
---|
| 169 | const EventEmitter = require('events');
|
---|
| 170 | const { resolve } = require('path');
|
---|
| 171 |
|
---|
| 172 | const piscina = new Piscina({
|
---|
| 173 | filename: resolve(__dirname, 'worker.js')
|
---|
| 174 | });
|
---|
| 175 |
|
---|
| 176 | (async function() {
|
---|
| 177 | const ee = new EventEmitter();
|
---|
| 178 | try {
|
---|
| 179 | const task = piscina.run({ a: 4, b: 6 }, { signal: ee });
|
---|
| 180 | ee.emit('abort');
|
---|
| 181 | await task;
|
---|
| 182 | } catch (err) {
|
---|
| 183 | console.log('The task was canceled');
|
---|
| 184 | }
|
---|
| 185 | })();
|
---|
| 186 | ```
|
---|
| 187 |
|
---|
| 188 | ### Delaying Availability of Workers
|
---|
| 189 |
|
---|
| 190 | A worker thread will not be made available to process tasks until Piscina
|
---|
| 191 | determines that it is "ready". By default, a worker is ready as soon as
|
---|
| 192 | Piscina loads it and acquires a reference to the exported handler function.
|
---|
| 193 |
|
---|
| 194 | There may be times when the availability of a worker may need to be delayed
|
---|
| 195 | longer while the worker initializes any resources it may need to operate.
|
---|
| 196 | To support this case, the worker module may export a `Promise` that resolves
|
---|
| 197 | the handler function as opposed to exporting the function directly:
|
---|
| 198 |
|
---|
| 199 | ```js
|
---|
| 200 | async function initialize() {
|
---|
| 201 | await someAsyncInitializationActivity();
|
---|
| 202 | return ({ a, b }) => a + b;
|
---|
| 203 | }
|
---|
| 204 |
|
---|
| 205 | module.exports = initialize();
|
---|
| 206 | ```
|
---|
| 207 |
|
---|
| 208 | Piscina will await the resolution of the exported Promise before marking
|
---|
| 209 | the worker thread available.
|
---|
| 210 |
|
---|
| 211 | ### Backpressure
|
---|
| 212 |
|
---|
| 213 | When the `maxQueue` option is set, once the `Piscina` queue is full, no
|
---|
| 214 | additional tasks may be submitted until the queue size falls below the
|
---|
| 215 | limit. The `'drain'` event may be used to receive notification when the
|
---|
| 216 | queue is empty and all tasks have been submitted to workers for processing.
|
---|
| 217 |
|
---|
| 218 | Example: Using a Node.js stream to feed a Piscina worker pool:
|
---|
| 219 |
|
---|
| 220 | ```js
|
---|
| 221 | 'use strict';
|
---|
| 222 |
|
---|
| 223 | const { resolve } = require('path');
|
---|
| 224 | const Pool = require('../..');
|
---|
| 225 |
|
---|
| 226 | const pool = new Pool({
|
---|
| 227 | filename: resolve(__dirname, 'worker.js'),
|
---|
| 228 | maxQueue: 'auto'
|
---|
| 229 | });
|
---|
| 230 |
|
---|
| 231 | const stream = getStreamSomehow();
|
---|
| 232 | stream.setEncoding('utf8');
|
---|
| 233 |
|
---|
| 234 | pool.on('drain', () => {
|
---|
| 235 | if (stream.isPaused()) {
|
---|
| 236 | console.log('resuming...', counter, pool.queueSize);
|
---|
| 237 | stream.resume();
|
---|
| 238 | }
|
---|
| 239 | });
|
---|
| 240 |
|
---|
| 241 | stream
|
---|
| 242 | .on('data', (data) => {
|
---|
| 243 | pool.run(data);
|
---|
| 244 | if (pool.queueSize === pool.options.maxQueue) {
|
---|
| 245 | console.log('pausing...', counter, pool.queueSize);
|
---|
| 246 | stream.pause();
|
---|
| 247 | }
|
---|
| 248 | })
|
---|
| 249 | .on('error', console.error)
|
---|
| 250 | .on('end', () => {
|
---|
| 251 | console.log('done');
|
---|
| 252 | });
|
---|
| 253 | ```
|
---|
| 254 |
|
---|
| 255 | ### Additional Examples
|
---|
| 256 |
|
---|
| 257 | Additional examples can be found in the GitHub repo at
|
---|
| 258 | https://github.com/jasnell/piscina/tree/master/examples
|
---|
| 259 |
|
---|
| 260 | ## Class: `Piscina`
|
---|
| 261 |
|
---|
| 262 | Piscina works by creating a pool of Node.js Worker Threads to which
|
---|
| 263 | one or more tasks may be dispatched. Each worker thread executes a
|
---|
| 264 | single exported function defined in a separate file. Whenever a
|
---|
| 265 | task is dispatched to a worker, the worker invokes the exported
|
---|
| 266 | function and reports the return value back to Piscina when the
|
---|
| 267 | function completes.
|
---|
| 268 |
|
---|
| 269 | This class extends [`EventEmitter`][] from Node.js.
|
---|
| 270 |
|
---|
| 271 | ### Constructor: `new Piscina([options])`
|
---|
| 272 |
|
---|
| 273 | * The following optional configuration is supported:
|
---|
| 274 | * `filename`: (`string | null`) Provides the default source for the code that
|
---|
| 275 | runs the tasks on Worker threads. This should be an absolute path or an
|
---|
| 276 | absolute `file://` URL to a file that exports a JavaScript `function` or
|
---|
| 277 | `async function` as its default export or `module.exports`. [ES modules][]
|
---|
| 278 | are supported.
|
---|
| 279 | * `name`: (`string | null`) Provides the name of the default exported worker
|
---|
| 280 | function. The default is `'default'`, indicating the default export of the
|
---|
| 281 | worker module.
|
---|
| 282 | * `minThreads`: (`number`) Sets the minimum number of threads that are always
|
---|
| 283 | running for this thread pool. The default is based on the number of
|
---|
| 284 | available CPUs.
|
---|
| 285 | * `maxThreads`: (`number`) Sets the maximum number of threads that are
|
---|
| 286 | running for this thread pool. The default is based on the number of
|
---|
| 287 | available CPUs.
|
---|
| 288 | * `idleTimeout`: (`number`) A timeout in milliseconds that specifies how long
|
---|
| 289 | a `Worker` is allowed to be idle, i.e. not handling any tasks, before it is
|
---|
| 290 | shut down. By default, this is immediate. **Tip**: *The default `idleTimeout`
|
---|
| 291 | can lead to some performance loss in the application because of the overhead
|
---|
| 292 | involved with stopping and starting new worker threads. To improve performance,
|
---|
| 293 | try setting the `idleTimeout` explicitly.*
|
---|
| 294 | * `maxQueue`: (`number` | `string`) The maximum number of tasks that may be
|
---|
| 295 | scheduled to run, but not yet running due to lack of available threads, at
|
---|
| 296 | a given time. By default, there is no limit. The special value `'auto'`
|
---|
| 297 | may be used to have Piscina calculate the maximum as the square of `maxThreads`.
|
---|
| 298 | When `'auto'` is used, the calculated `maxQueue` value may be found by checking
|
---|
| 299 | the [`options.maxQueue`](#property-options-readonly) property.
|
---|
| 300 | * `concurrentTasksPerWorker`: (`number`) Specifies how many tasks can share
|
---|
| 301 | a single Worker thread simultaneously. The default is `1`. This generally
|
---|
| 302 | only makes sense to specify if there is some kind of asynchronous component
|
---|
| 303 | to the task. Keep in mind that Worker threads are generally not built for
|
---|
| 304 | handling I/O in parallel.
|
---|
| 305 | * `useAtomics`: (`boolean`) Use the [`Atomics`][] API for faster communication
|
---|
| 306 | between threads. This is on by default.
|
---|
| 307 | * `resourceLimits`: (`object`) See [Node.js new Worker options][]
|
---|
| 308 | * `maxOldGenerationSizeMb`: (`number`) The maximum size of each worker threads
|
---|
| 309 | main heap in MB.
|
---|
| 310 | * `maxYoungGenerationSizeMb`: (`number`) The maximum size of a heap space for
|
---|
| 311 | recently created objects.
|
---|
| 312 | * `codeRangeSizeMb`: (`number`) The size of a pre-allocated memory range used
|
---|
| 313 | for generated code.
|
---|
| 314 | * `stackSizeMb` : (`number`) The default maximum stack size for the thread.
|
---|
| 315 | Small values may lead to unusable Worker instances. Default: 4
|
---|
| 316 | * `env`: (`object`) If set, specifies the initial value of `process.env` inside
|
---|
| 317 | the worker threads. See [Node.js new Worker options][] for details.
|
---|
| 318 | * `argv`: (`any[]`) List of arguments that will be stringified and appended to
|
---|
| 319 | `process.argv` in the worker. See [Node.js new Worker options][] for details.
|
---|
| 320 | * `execArgv`: (`string[]`) List of Node.js CLI options passed to the worker.
|
---|
| 321 | See [Node.js new Worker options][] for details.
|
---|
| 322 | * `workerData`: (`any`) Any JavaScript value that can be cloned and made
|
---|
| 323 | available as `require('piscina').workerData`. See [Node.js new Worker options][]
|
---|
| 324 | for details. Unlike regular Node.js Worker Threads, `workerData` must not
|
---|
| 325 | specify any value requiring a `transferList`. This is because the `workerData`
|
---|
| 326 | will be cloned for each pooled worker.
|
---|
| 327 | * `taskQueue`: (`TaskQueue`) By default, Piscina uses a first-in-first-out
|
---|
| 328 | queue for submitted tasks. The `taskQueue` option can be used to provide an
|
---|
| 329 | alternative implementation. See [Custom Task Queues][] for additional detail.
|
---|
| 330 | * `niceIncrement`: (`number`) An optional value that decreases priority for
|
---|
| 331 | the individual threads, i.e. the higher the value, the lower the priority
|
---|
| 332 | of the Worker threads. This value is only used on Linux and requires the
|
---|
| 333 | optional [`nice-napi`][] module to be installed.
|
---|
| 334 | See [`nice(2)`][] for more details.
|
---|
| 335 | * `trackUnmanagedFds`: (`boolean`) An optional setting that, when `true`, will
|
---|
| 336 | cause Workers to track file descriptors managed using `fs.open()` and
|
---|
| 337 | `fs.close()`, and will close them automatically when the Worker exits.
|
---|
| 338 | Defaults to `true`. (This option is only supported on Node.js 12.19+ and
|
---|
| 339 | all Node.js versions higher than 14.6.0).
|
---|
| 340 |
|
---|
| 341 | Use caution when setting resource limits. Setting limits that are too low may
|
---|
| 342 | result in the `Piscina` worker threads being unusable.
|
---|
| 343 |
|
---|
| 344 | ### Method: `run(task[, options])`
|
---|
| 345 |
|
---|
| 346 | Schedules a task to be run on a Worker thread.
|
---|
| 347 |
|
---|
| 348 | * `task`: Any value. This will be passed to the function that is exported from
|
---|
| 349 | `filename`.
|
---|
| 350 | * `options`:
|
---|
| 351 | * `transferList`: An optional lists of objects that is passed to
|
---|
| 352 | [`postMessage()`] when posting `task` to the Worker, which are transferred
|
---|
| 353 | rather than cloned.
|
---|
| 354 | * `filename`: Optionally overrides the `filename` option passed to the
|
---|
| 355 | constructor for this task. If no `filename` was specified to the constructor,
|
---|
| 356 | this is mandatory.
|
---|
| 357 | * `name`: Optionally overrides the exported worker function used for the task.
|
---|
| 358 | * `abortSignal`: An [`AbortSignal`][] instance. If passed, this can be used to
|
---|
| 359 | cancel a task. If the task is already running, the corresponding `Worker`
|
---|
| 360 | thread will be stopped.
|
---|
| 361 | (More generally, any `EventEmitter` or `EventTarget` that emits `'abort'`
|
---|
| 362 | events can be passed here.) Abortable tasks cannot share threads regardless
|
---|
| 363 | of the `concurrentTasksPerWorker` options.
|
---|
| 364 |
|
---|
| 365 | This returns a `Promise` for the return value of the (async) function call
|
---|
| 366 | made to the function exported from `filename`. If the (async) function throws
|
---|
| 367 | an error, the returned `Promise` will be rejected with that error.
|
---|
| 368 | If the task is aborted, the returned `Promise` is rejected with an error
|
---|
| 369 | as well.
|
---|
| 370 |
|
---|
| 371 | ### Method: `runTask(task[, transferList][, filename][, abortSignal])`
|
---|
| 372 |
|
---|
| 373 | **Deprecated** -- Use `run(task, options)` instead.
|
---|
| 374 |
|
---|
| 375 | Schedules a task to be run on a Worker thread.
|
---|
| 376 |
|
---|
| 377 | * `task`: Any value. This will be passed to the function that is exported from
|
---|
| 378 | `filename`.
|
---|
| 379 | * `transferList`: An optional lists of objects that is passed to
|
---|
| 380 | [`postMessage()`] when posting `task` to the Worker, which are transferred
|
---|
| 381 | rather than cloned.
|
---|
| 382 | * `filename`: Optionally overrides the `filename` option passed to the
|
---|
| 383 | constructor for this task. If no `filename` was specified to the constructor,
|
---|
| 384 | this is mandatory.
|
---|
| 385 | * `abortSignal`: An [`AbortSignal`][] instance. If passed, this can be used to
|
---|
| 386 | cancel a task. If the task is already running, the corresponding `Worker`
|
---|
| 387 | thread will be stopped.
|
---|
| 388 | (More generally, any `EventEmitter` or `EventTarget` that emits `'abort'`
|
---|
| 389 | events can be passed here.) Abortable tasks cannot share threads regardless
|
---|
| 390 | of the `concurrentTasksPerWorker` options.
|
---|
| 391 |
|
---|
| 392 | This returns a `Promise` for the return value of the (async) function call
|
---|
| 393 | made to the function exported from `filename`. If the (async) function throws
|
---|
| 394 | an error, the returned `Promise` will be rejected with that error.
|
---|
| 395 | If the task is aborted, the returned `Promise` is rejected with an error
|
---|
| 396 | as well.
|
---|
| 397 |
|
---|
| 398 | ### Method: `destroy()`
|
---|
| 399 |
|
---|
| 400 | Stops all Workers and rejects all `Promise`s for pending tasks.
|
---|
| 401 |
|
---|
| 402 | This returns a `Promise` that is fulfilled once all threads have stopped.
|
---|
| 403 |
|
---|
| 404 | ### Event: `'error'`
|
---|
| 405 |
|
---|
| 406 | An `'error'` event is emitted by instances of this class when:
|
---|
| 407 |
|
---|
| 408 | - Uncaught exceptions occur inside Worker threads that do not currently handle
|
---|
| 409 | tasks.
|
---|
| 410 | - Unexpected messages are sent from from Worker threads.
|
---|
| 411 |
|
---|
| 412 | All other errors are reported by rejecting the `Promise` returned from
|
---|
| 413 | `run()` or `runTask()`, including rejections reported by the handler function
|
---|
| 414 | itself.
|
---|
| 415 |
|
---|
| 416 | ### Event: `'drain'`
|
---|
| 417 |
|
---|
| 418 | A `'drain'` event is emitted whenever the `queueSize` reaches `0`.
|
---|
| 419 |
|
---|
| 420 | ### Property: `completed` (readonly)
|
---|
| 421 |
|
---|
| 422 | The current number of completed tasks.
|
---|
| 423 |
|
---|
| 424 | ### Property: `duration` (readonly)
|
---|
| 425 |
|
---|
| 426 | The length of time (in milliseconds) since this `Piscina` instance was
|
---|
| 427 | created.
|
---|
| 428 |
|
---|
| 429 | ### Property: `options` (readonly)
|
---|
| 430 |
|
---|
| 431 | A copy of the options that are currently being used by this instance. This
|
---|
| 432 | object has the same properties as the options object passed to the constructor.
|
---|
| 433 |
|
---|
| 434 | ### Property: `runTime` (readonly)
|
---|
| 435 |
|
---|
| 436 | A histogram summary object summarizing the collected run times of completed
|
---|
| 437 | tasks. All values are expressed in milliseconds.
|
---|
| 438 |
|
---|
| 439 | * `runTime.average` {`number`} The average run time of all tasks
|
---|
| 440 | * `runTime.mean` {`number`} The mean run time of all tasks
|
---|
| 441 | * `runTime.stddev` {`number`} The standard deviation of collected run times
|
---|
| 442 | * `runTime.min` {`number`} The fastest recorded run time
|
---|
| 443 | * `runTime.max` {`number`} The slowest recorded run time
|
---|
| 444 |
|
---|
| 445 | All properties following the pattern `p{N}` where N is a number (e.g. `p1`, `p99`)
|
---|
| 446 | represent the percentile distributions of run time observations. For example,
|
---|
| 447 | `p99` is the 99th percentile indicating that 99% of the observed run times were
|
---|
| 448 | faster or equal to the given value.
|
---|
| 449 |
|
---|
| 450 | ```js
|
---|
| 451 | {
|
---|
| 452 | average: 1880.25,
|
---|
| 453 | mean: 1880.25,
|
---|
| 454 | stddev: 1.93,
|
---|
| 455 | min: 1877,
|
---|
| 456 | max: 1882.0190887451172,
|
---|
| 457 | p0_001: 1877,
|
---|
| 458 | p0_01: 1877,
|
---|
| 459 | p0_1: 1877,
|
---|
| 460 | p1: 1877,
|
---|
| 461 | p2_5: 1877,
|
---|
| 462 | p10: 1877,
|
---|
| 463 | p25: 1877,
|
---|
| 464 | p50: 1881,
|
---|
| 465 | p75: 1881,
|
---|
| 466 | p90: 1882,
|
---|
| 467 | p97_5: 1882,
|
---|
| 468 | p99: 1882,
|
---|
| 469 | p99_9: 1882,
|
---|
| 470 | p99_99: 1882,
|
---|
| 471 | p99_999: 1882
|
---|
| 472 | }
|
---|
| 473 | ```
|
---|
| 474 |
|
---|
| 475 | ### Property: `threads` (readonly)
|
---|
| 476 |
|
---|
| 477 | An Array of the `Worker` instances used by this pool.
|
---|
| 478 |
|
---|
| 479 | ### Property: `queueSize` (readonly)
|
---|
| 480 |
|
---|
| 481 | The current number of tasks waiting to be assigned to a Worker thread.
|
---|
| 482 |
|
---|
| 483 | ### Property: `utilization` (readonly)
|
---|
| 484 |
|
---|
| 485 | A point-in-time ratio comparing the approximate total mean run time
|
---|
| 486 | of completed tasks to the total runtime capacity of the pool.
|
---|
| 487 |
|
---|
| 488 | A pools runtime capacity is determined by multiplying the `duration`
|
---|
| 489 | by the `options.maxThread` count. This provides an absolute theoretical
|
---|
| 490 | maximum aggregate compute time that the pool would be capable of.
|
---|
| 491 |
|
---|
| 492 | The approximate total mean run time is determined by multiplying the
|
---|
| 493 | mean run time of all completed tasks by the total number of completed
|
---|
| 494 | tasks. This number represents the approximate amount of time the
|
---|
| 495 | pool as been actively processing tasks.
|
---|
| 496 |
|
---|
| 497 | The utilization is then calculated by dividing the approximate total
|
---|
| 498 | mean run time by the capacity, yielding a fraction between `0` and `1`.
|
---|
| 499 |
|
---|
| 500 | ### Property: `waitTime` (readonly)
|
---|
| 501 |
|
---|
| 502 | A histogram summary object summarizing the collected times tasks spent
|
---|
| 503 | waiting in the queue. All values are expressed in milliseconds.
|
---|
| 504 |
|
---|
| 505 | * `waitTime.average` {`number`} The average wait time of all tasks
|
---|
| 506 | * `waitTime.mean` {`number`} The mean wait time of all tasks
|
---|
| 507 | * `waitTime.stddev` {`number`} The standard deviation of collected wait times
|
---|
| 508 | * `waitTime.min` {`number`} The fastest recorded wait time
|
---|
| 509 | * `waitTime.max` {`number`} The longest recorded wait time
|
---|
| 510 |
|
---|
| 511 | All properties following the pattern `p{N}` where N is a number (e.g. `p1`, `p99`)
|
---|
| 512 | represent the percentile distributions of wait time observations. For example,
|
---|
| 513 | `p99` is the 99th percentile indicating that 99% of the observed wait times were
|
---|
| 514 | faster or equal to the given value.
|
---|
| 515 |
|
---|
| 516 | ```js
|
---|
| 517 | {
|
---|
| 518 | average: 1880.25,
|
---|
| 519 | mean: 1880.25,
|
---|
| 520 | stddev: 1.93,
|
---|
| 521 | min: 1877,
|
---|
| 522 | max: 1882.0190887451172,
|
---|
| 523 | p0_001: 1877,
|
---|
| 524 | p0_01: 1877,
|
---|
| 525 | p0_1: 1877,
|
---|
| 526 | p1: 1877,
|
---|
| 527 | p2_5: 1877,
|
---|
| 528 | p10: 1877,
|
---|
| 529 | p25: 1877,
|
---|
| 530 | p50: 1881,
|
---|
| 531 | p75: 1881,
|
---|
| 532 | p90: 1882,
|
---|
| 533 | p97_5: 1882,
|
---|
| 534 | p99: 1882,
|
---|
| 535 | p99_9: 1882,
|
---|
| 536 | p99_99: 1882,
|
---|
| 537 | p99_999: 1882
|
---|
| 538 | }
|
---|
| 539 | ```
|
---|
| 540 |
|
---|
| 541 | ### Static property: `isWorkerThread` (readonly)
|
---|
| 542 |
|
---|
| 543 | Is `true` if this code runs inside a `Piscina` threadpool as a Worker.
|
---|
| 544 |
|
---|
| 545 | ### Static property: `version` (readonly)
|
---|
| 546 |
|
---|
| 547 | Provides the current version of this library as a semver string.
|
---|
| 548 |
|
---|
| 549 | ### Static method: `move(value)`
|
---|
| 550 |
|
---|
| 551 | By default, any value returned by a worker function will be cloned when
|
---|
| 552 | returned back to the Piscina pool, even if that object is capable of
|
---|
| 553 | being transfered. The `Piscina.move()` method can be used to wrap and
|
---|
| 554 | mark transferable values such that they will by transfered rather than
|
---|
| 555 | cloned.
|
---|
| 556 |
|
---|
| 557 | The `value` may be any object supported by Node.js to be transferable
|
---|
| 558 | (e.g. `ArrayBuffer`, any `TypedArray`, or `MessagePort`), or any object
|
---|
| 559 | implementing the `Transferable` interface.
|
---|
| 560 |
|
---|
| 561 | ```js
|
---|
| 562 | const { move } = require('piscina');
|
---|
| 563 |
|
---|
| 564 | module.exports = () => {
|
---|
| 565 | return move(new ArrayBuffer(10));
|
---|
| 566 | }
|
---|
| 567 | ```
|
---|
| 568 |
|
---|
| 569 | The `move()` method will throw if the `value` is not transferable.
|
---|
| 570 |
|
---|
| 571 | The object returned by the `move()` method should not be set as a
|
---|
| 572 | nested value in an object. If it is used, the `move()` object itself
|
---|
| 573 | will be cloned as opposed to transfering the object it wraps.
|
---|
| 574 |
|
---|
| 575 | #### Interface: `Transferable`
|
---|
| 576 |
|
---|
| 577 | Objects may implement the `Transferable` interface to create their own
|
---|
| 578 | custom transferable objects. This is useful when an object being
|
---|
| 579 | passed into or from a worker contains a deeply nested transferable
|
---|
| 580 | object such as an `ArrayBuffer` or `MessagePort`.
|
---|
| 581 |
|
---|
| 582 | `Transferable` objects expose two properties inspected by Piscina
|
---|
| 583 | to determine how to transfer the object. These properties are
|
---|
| 584 | named using the special static `Piscina.transferableSymbol` and
|
---|
| 585 | `Piscina.valueSymbol` properties:
|
---|
| 586 |
|
---|
| 587 | * The `Piscina.transferableSymbol` property provides the object
|
---|
| 588 | (or objects) that are to be included in the `transferList`.
|
---|
| 589 |
|
---|
| 590 | * The `Piscina.valueSymbol` property provides a surrogate value
|
---|
| 591 | to transmit in place of the `Transferable` itself.
|
---|
| 592 |
|
---|
| 593 | Both properties are required.
|
---|
| 594 |
|
---|
| 595 | For example,
|
---|
| 596 |
|
---|
| 597 | ```js
|
---|
| 598 | const {
|
---|
| 599 | move,
|
---|
| 600 | transferableSymbol,
|
---|
| 601 | valueSymbol
|
---|
| 602 | } = require('piscina');
|
---|
| 603 |
|
---|
| 604 | module.exports = () => {
|
---|
| 605 | const obj = {
|
---|
| 606 | a: { b: new Uint8Array(5); },
|
---|
| 607 | c: { new Uint8Array(10); },
|
---|
| 608 |
|
---|
| 609 | get [transferableSymbol]() {
|
---|
| 610 | // Transfer the two underlying ArrayBuffers
|
---|
| 611 | return [this.a.b.buffer, this.c.buffer];
|
---|
| 612 | }
|
---|
| 613 |
|
---|
| 614 | get [valueSymbol]() {
|
---|
| 615 | return { a: { b: this.a.b }, c: this.c };
|
---|
| 616 | }
|
---|
| 617 | };
|
---|
| 618 | return move(obj);
|
---|
| 619 | };
|
---|
| 620 | ```
|
---|
| 621 |
|
---|
| 622 | ## Custom Task Queues
|
---|
| 623 |
|
---|
| 624 | By default, Piscina uses a simple array-based first-in-first-out (fifo)
|
---|
| 625 | task queue. When a new task is submitted and there are no available
|
---|
| 626 | workers, tasks are pushed on to the queue until a worker becomes
|
---|
| 627 | available.
|
---|
| 628 |
|
---|
| 629 | If the default fifo queue is not sufficient, user code may replace the
|
---|
| 630 | task queue implementation with a custom implementation using the
|
---|
| 631 | `taskQueue` option on the Piscina constructor.
|
---|
| 632 |
|
---|
| 633 | Custom task queue objects *must* implement the `TaskQueue` interface,
|
---|
| 634 | described below using TypeScript syntax:
|
---|
| 635 |
|
---|
| 636 | ```ts
|
---|
| 637 | interface Task {
|
---|
| 638 | readonly [Piscina.queueOptionsSymbol] : object | null;
|
---|
| 639 | }
|
---|
| 640 |
|
---|
| 641 | interface TaskQueue {
|
---|
| 642 | readonly size : number;
|
---|
| 643 | shift () : Task | null;
|
---|
| 644 | remove (task : Task) : void;
|
---|
| 645 | push (task : Task) : void;
|
---|
| 646 | }
|
---|
| 647 | ```
|
---|
| 648 |
|
---|
| 649 | An example of a custom task queue that uses a shuffled priority queue
|
---|
| 650 | is available in [`examples/task-queue`](./examples/task-queue/index.js);
|
---|
| 651 |
|
---|
| 652 | The special symbol `Piscina.queueOptionsSymbol` may be set as a property
|
---|
| 653 | on tasks submitted to `run()` or `runTask()` as a way of passing additional
|
---|
| 654 | options on to the custom `TaskQueue` implementation. (Note that because the
|
---|
| 655 | queue options are set as a property on the task, tasks with queue
|
---|
| 656 | options cannot be submitted as JavaScript primitives).
|
---|
| 657 |
|
---|
| 658 | ## Current Limitations (Things we're working on / would love help with)
|
---|
| 659 |
|
---|
| 660 | * Improved Documentation
|
---|
| 661 | * Benchmarks
|
---|
| 662 |
|
---|
| 663 | ## Performance Notes
|
---|
| 664 |
|
---|
| 665 | Workers are generally optimized for offloading synchronous,
|
---|
| 666 | compute-intensive operations off the main Node.js event loop thread.
|
---|
| 667 | While it is possible to perform asynchronous operations and I/O
|
---|
| 668 | within a Worker, the performance advantages of doing so will be
|
---|
| 669 | minimal.
|
---|
| 670 |
|
---|
| 671 | Specifically, it is worth noting that asynchronous operations
|
---|
| 672 | within Node.js, including I/O such as file system operations
|
---|
| 673 | or CPU-bound tasks such as crypto operations or compression
|
---|
| 674 | algorithms, are already performed in parallel by Node.js and
|
---|
| 675 | libuv on a per-process level. This means that there will be
|
---|
| 676 | little performance impact on moving such async operations into
|
---|
| 677 | a Piscina worker (see examples/scrypt for example).
|
---|
| 678 |
|
---|
| 679 | ### Queue Size
|
---|
| 680 |
|
---|
| 681 | Piscina provides the ability to configure the minimum and
|
---|
| 682 | maximum number of worker threads active in the pool, as well as
|
---|
| 683 | set limits on the number of tasks that may be queued up waiting
|
---|
| 684 | for a free worker. It is important to note that setting the
|
---|
| 685 | `maxQueue` size too high relative to the number of worker threads
|
---|
| 686 | can have a detrimental impact on performance and memory usage.
|
---|
| 687 | Setting the `maxQueue` size too small can also be problematic
|
---|
| 688 | as doing so could cause your worker threads to become idle and
|
---|
| 689 | be shutdown. Our testing has shown that a `maxQueue` size of
|
---|
| 690 | approximately the square of the maximum number of threads is
|
---|
| 691 | generally sufficient and performs well for many cases, but this
|
---|
| 692 | will vary significantly depending on your workload. It will be
|
---|
| 693 | important to test and benchmark your worker pools to ensure you've
|
---|
| 694 | effectively balanced queue wait times, memory usage, and worker
|
---|
| 695 | pool utilization.
|
---|
| 696 |
|
---|
| 697 | ### Queue Pressure and Idle Threads
|
---|
| 698 |
|
---|
| 699 | The thread pool maintained by Piscina has both a minimum and maximum
|
---|
| 700 | limit to the number of threads that may be created. When a Piscina
|
---|
| 701 | instance is created, it will spawn the minimum number of threads
|
---|
| 702 | immediately, then create additional threads as needed up to the
|
---|
| 703 | limit set by `maxThreads`. Whenever a worker completes a task, a
|
---|
| 704 | check is made to determine if there is additional work for it to
|
---|
| 705 | perform. If there is no additional work, the thread is marked idle.
|
---|
| 706 | By default, idle threads are shutdown immediately, with Piscina
|
---|
| 707 | ensuring that the pool always maintains at least the minimum.
|
---|
| 708 |
|
---|
| 709 | When a Piscina pool is processing a stream of tasks (for instance,
|
---|
| 710 | processing http server requests as in the React server-side
|
---|
| 711 | rendering example in examples/react-ssr), if the rate in which
|
---|
| 712 | new tasks are received and queued is not sufficient to keep workers
|
---|
| 713 | from going idle and terminating, the pool can experience a thrashing
|
---|
| 714 | effect -- excessively creating and terminating workers that will
|
---|
| 715 | cause a net performance loss. There are a couple of strategies to
|
---|
| 716 | avoid this churn:
|
---|
| 717 |
|
---|
| 718 | Strategy 1: Ensure that the queue rate of new tasks is sufficient to
|
---|
| 719 | keep workers from going idle. We refer to this as "queue pressure".
|
---|
| 720 | If the queue pressure is too low, workers will go idle and terminate.
|
---|
| 721 | If the queue pressure is too high, tasks will stack up, experience
|
---|
| 722 | increased wait latency, and consume additional memory.
|
---|
| 723 |
|
---|
| 724 | Strategy 2: Increase the `idleTimeout` configuration option. By
|
---|
| 725 | default, idle threads terminate immediately. The `idleTimeout` option
|
---|
| 726 | can be used to specify a longer period of time to wait for additional
|
---|
| 727 | tasks to be submitted before terminating the worker. If the queue
|
---|
| 728 | pressure is not maintained, this could result in workers sitting idle
|
---|
| 729 | but those will have less of a performance impact than the thrashing
|
---|
| 730 | that occurs when threads are repeatedly terminated and recreated.
|
---|
| 731 |
|
---|
| 732 | Strategy 3: Increase the `minThreads` configuration option. This has
|
---|
| 733 | the same basic effect as increasing the `idleTimeout`. If the queue
|
---|
| 734 | pressure is not high enough, workers may sit idle indefinitely but
|
---|
| 735 | there will be less of a performance hit.
|
---|
| 736 |
|
---|
| 737 | In applications using Piscina, it will be most effective to use a
|
---|
| 738 | combination of these three approaches and tune the various configuration
|
---|
| 739 | parameters to find the optimum combination both for the application
|
---|
| 740 | workload and the capabilities of the deployment environment. There
|
---|
| 741 | are no one set of options that are going to work best.
|
---|
| 742 |
|
---|
| 743 | ### Thread priority on Linux systems
|
---|
| 744 |
|
---|
| 745 | On Linux systems that support [`nice(2)`][], Piscina is capable of setting
|
---|
| 746 | the priority of every worker in the pool. To use this mechanism, an additional
|
---|
| 747 | optional native addon dependency (`nice-napi`, `npm i nice-napi`) is required.
|
---|
| 748 | Once [`nice-napi`][] is installed, creating a `Piscina` instance with the
|
---|
| 749 | `niceIncrement` configuration option will set the priority for the pool:
|
---|
| 750 |
|
---|
| 751 | ```js
|
---|
| 752 | const Piscina = require('piscina');
|
---|
| 753 | const pool = new Piscina({
|
---|
| 754 | worker: '/absolute/path/to/worker.js',
|
---|
| 755 | niceIncrement: 20
|
---|
| 756 | });
|
---|
| 757 | ```
|
---|
| 758 |
|
---|
| 759 | The higher the `niceIncrement`, the lower the CPU scheduling priority will be
|
---|
| 760 | for the pooled workers which will generally extend the execution time of
|
---|
| 761 | CPU-bound tasks but will help prevent those threads from stealing CPU time from
|
---|
| 762 | the main Node.js event loop thread. Whether this is a good thing or not depends
|
---|
| 763 | entirely on your application and will require careful profiling to get correct.
|
---|
| 764 |
|
---|
| 765 | The key metrics to pay attention to when tuning the `niceIncrement` are the
|
---|
| 766 | sampled run times of the tasks in the worker pool (using the [`runTime`][]
|
---|
| 767 | property) and the [delay of the Node.js main thread event loop][].
|
---|
| 768 |
|
---|
| 769 | ### Multiple Thread Pools and Embedding Piscina as a Dependency
|
---|
| 770 |
|
---|
| 771 | Every `Piscina` instance creates a separate pool of threads and operates
|
---|
| 772 | without any awareness of the other. When multiple pools are created in a
|
---|
| 773 | single application the various threads may contend with one another, and
|
---|
| 774 | with the Node.js main event loop thread, and may cause an overall reduction
|
---|
| 775 | in system performance.
|
---|
| 776 |
|
---|
| 777 | Modules that embed Piscina as a dependency *should* make it clear via
|
---|
| 778 | documentation that threads are being used. It would be ideal if those
|
---|
| 779 | would make it possible for users to provide an existing `Piscina` instance
|
---|
| 780 | as a configuration option in lieu of always creating their own.
|
---|
| 781 |
|
---|
| 782 |
|
---|
| 783 | ## Release Notes
|
---|
| 784 |
|
---|
| 785 | ### 3.1.0
|
---|
| 786 |
|
---|
| 787 | * Deprecates `piscina.runTask()`; adds `piscina.run()` as an alternative.
|
---|
| 788 | https://github.com/piscinajs/piscina/commit/d7fa24d7515789001f7237ad6ae9ad42d582fc75
|
---|
| 789 | * Allows multiple exported handler functions from a single file.
|
---|
| 790 | https://github.com/piscinajs/piscina/commit/d7fa24d7515789001f7237ad6ae9ad42d582fc75
|
---|
| 791 |
|
---|
| 792 | ### 3.0.0
|
---|
| 793 |
|
---|
| 794 | * Drops Node.js 10.x support
|
---|
| 795 | * Updates minimum TypeScript target to ES2019
|
---|
| 796 |
|
---|
| 797 | ### 2.1.0
|
---|
| 798 |
|
---|
| 799 | * Adds name property to indicate `AbortError` when tasks are
|
---|
| 800 | canceled using an `AbortController` (or similar)
|
---|
| 801 | * More examples
|
---|
| 802 |
|
---|
| 803 | ### 2.0.0
|
---|
| 804 |
|
---|
| 805 | * Added unmanaged file descriptor tracking
|
---|
| 806 | * Updated dependencies
|
---|
| 807 |
|
---|
| 808 | ### 1.6.1
|
---|
| 809 |
|
---|
| 810 | * Bug fix: Reject if AbortSignal is already aborted
|
---|
| 811 | * Bug Fix: Use once listener for abort event
|
---|
| 812 |
|
---|
| 813 | ### 1.6.0
|
---|
| 814 |
|
---|
| 815 | * Add the `niceIncrement` configuration parameter.
|
---|
| 816 |
|
---|
| 817 | ### 1.5.1
|
---|
| 818 |
|
---|
| 819 | * Bug fixes around abortable task selection.
|
---|
| 820 |
|
---|
| 821 | ### 1.5.0
|
---|
| 822 |
|
---|
| 823 | * Added `Piscina.move()`
|
---|
| 824 | * Added Custom Task Queues
|
---|
| 825 | * Added utilization metric
|
---|
| 826 | * Wait for workers to be ready before considering them as candidates
|
---|
| 827 | * Additional examples
|
---|
| 828 |
|
---|
| 829 | ### 1.4.0
|
---|
| 830 |
|
---|
| 831 | * Added `maxQueue = 'auto'` to autocalculate the maximum queue size.
|
---|
| 832 | * Added more examples, including an example of implementing a worker
|
---|
| 833 | as a Node.js native addon.
|
---|
| 834 |
|
---|
| 835 | ### 1.3.0
|
---|
| 836 |
|
---|
| 837 | * Added the `'drain'` event
|
---|
| 838 |
|
---|
| 839 | ### 1.2.0
|
---|
| 840 |
|
---|
| 841 | * Added support for ESM and file:// URLs
|
---|
| 842 | * Added `env`, `argv`, `execArgv`, and `workerData` options
|
---|
| 843 | * More examples
|
---|
| 844 |
|
---|
| 845 | ### 1.1.0
|
---|
| 846 |
|
---|
| 847 | * Added support for Worker Thread `resourceLimits`
|
---|
| 848 |
|
---|
| 849 | ### 1.0.0
|
---|
| 850 |
|
---|
| 851 | * Initial release!
|
---|
| 852 |
|
---|
| 853 | ## The Team
|
---|
| 854 |
|
---|
| 855 | * James M Snell <jasnell@gmail.com>
|
---|
| 856 | * Anna Henningsen <anna@addaleax.net>
|
---|
| 857 | * Matteo Collina <matteo.collina@gmail.com>
|
---|
| 858 |
|
---|
| 859 | ## Acknowledgements
|
---|
| 860 |
|
---|
| 861 | Piscina development is sponsored by [NearForm Research][].
|
---|
| 862 |
|
---|
| 863 | [`Atomics`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Atomics
|
---|
| 864 | [`EventEmitter`]: https://nodejs.org/api/events.html
|
---|
| 865 | [`postMessage`]: https://nodejs.org/api/worker_threads.html#worker_threads_port_postmessage_value_transferlist
|
---|
| 866 | [`examples/task-queue`]: https://github.com/jasnell/piscina/blob/master/examples/task-queue/index.js
|
---|
| 867 | [`nice(2)`]: https://linux.die.net/man/2/nice
|
---|
| 868 | [`nice-napi`]: https://npmjs.org/package/nice-napi
|
---|
| 869 | [`runTime`]: #property-runtime-readonly
|
---|
| 870 | [Custom Task Queues]: #custom_task_queues
|
---|
| 871 | [ES modules]: https://nodejs.org/api/esm.html
|
---|
| 872 | [Node.js new Worker options]: https://nodejs.org/api/worker_threads.html#worker_threads_new_worker_filename_options
|
---|
| 873 | [MIT Licensed]: LICENSE.md
|
---|
| 874 | [NearForm Research]: https://www.nearform.com/research/
|
---|
| 875 | [delay of the Node.js main thread event loop]: https://nodejs.org/dist/latest-v14.x/docs/api/perf_hooks.html#perf_hooks_perf_hooks_monitoreventloopdelay_options
|
---|