[6a3a178] | 1 | # Async-Limiter
|
---|
| 2 |
|
---|
| 3 | A module for limiting concurrent asynchronous actions in flight. Forked from [queue](https://github.com/jessetane/queue).
|
---|
| 4 |
|
---|
| 5 | [![npm](http://img.shields.io/npm/v/async-limiter.svg?style=flat-square)](http://www.npmjs.org/async-limiter)
|
---|
| 6 | [![tests](https://img.shields.io/travis/STRML/async-limiter.svg?style=flat-square&branch=master)](https://travis-ci.org/STRML/async-limiter)
|
---|
| 7 | [![coverage](https://img.shields.io/coveralls/STRML/async-limiter.svg?style=flat-square&branch=master)](https://coveralls.io/r/STRML/async-limiter)
|
---|
| 8 |
|
---|
| 9 | This module exports a class `Limiter` that implements some of the `Array` API.
|
---|
| 10 | Pass async functions (ones that accept a callback or return a promise) to an instance's additive array methods.
|
---|
| 11 |
|
---|
| 12 | ## Motivation
|
---|
| 13 |
|
---|
| 14 | Certain functions, like `zlib`, have [undesirable behavior](https://github.com/nodejs/node/issues/8871#issuecomment-250915913) when
|
---|
| 15 | run at infinite concurrency.
|
---|
| 16 |
|
---|
| 17 | In this case, it is actually faster, and takes far less memory, to limit concurrency.
|
---|
| 18 |
|
---|
| 19 | This module should do the absolute minimum work necessary to queue up functions. PRs are welcome that would
|
---|
| 20 | make this module faster or lighter, but new functionality is not desired.
|
---|
| 21 |
|
---|
| 22 | Style should confirm to nodejs/node style.
|
---|
| 23 |
|
---|
| 24 | ## Example
|
---|
| 25 |
|
---|
| 26 | ``` javascript
|
---|
| 27 | var Limiter = require('async-limiter')
|
---|
| 28 |
|
---|
| 29 | var t = new Limiter({concurrency: 2});
|
---|
| 30 | var results = []
|
---|
| 31 |
|
---|
| 32 | // add jobs using the familiar Array API
|
---|
| 33 | t.push(function (cb) {
|
---|
| 34 | results.push('two')
|
---|
| 35 | cb()
|
---|
| 36 | })
|
---|
| 37 |
|
---|
| 38 | t.push(
|
---|
| 39 | function (cb) {
|
---|
| 40 | results.push('four')
|
---|
| 41 | cb()
|
---|
| 42 | },
|
---|
| 43 | function (cb) {
|
---|
| 44 | results.push('five')
|
---|
| 45 | cb()
|
---|
| 46 | }
|
---|
| 47 | )
|
---|
| 48 |
|
---|
| 49 | t.unshift(function (cb) {
|
---|
| 50 | results.push('one')
|
---|
| 51 | cb()
|
---|
| 52 | })
|
---|
| 53 |
|
---|
| 54 | t.splice(2, 0, function (cb) {
|
---|
| 55 | results.push('three')
|
---|
| 56 | cb()
|
---|
| 57 | })
|
---|
| 58 |
|
---|
| 59 | // Jobs run automatically. If you want a callback when all are done,
|
---|
| 60 | // call 'onDone()'.
|
---|
| 61 | t.onDone(function () {
|
---|
| 62 | console.log('all done:', results)
|
---|
| 63 | })
|
---|
| 64 | ```
|
---|
| 65 |
|
---|
| 66 | ## Zlib Example
|
---|
| 67 |
|
---|
| 68 | ```js
|
---|
| 69 | const zlib = require('zlib');
|
---|
| 70 | const Limiter = require('async-limiter');
|
---|
| 71 |
|
---|
| 72 | const message = {some: "data"};
|
---|
| 73 | const payload = new Buffer(JSON.stringify(message));
|
---|
| 74 |
|
---|
| 75 | // Try with different concurrency values to see how this actually
|
---|
| 76 | // slows significantly with higher concurrency!
|
---|
| 77 | //
|
---|
| 78 | // 5: 1398.607ms
|
---|
| 79 | // 10: 1375.668ms
|
---|
| 80 | // Infinity: 4423.300ms
|
---|
| 81 | //
|
---|
| 82 | const t = new Limiter({concurrency: 5});
|
---|
| 83 | function deflate(payload, cb) {
|
---|
| 84 | t.push(function(done) {
|
---|
| 85 | zlib.deflate(payload, function(err, buffer) {
|
---|
| 86 | done();
|
---|
| 87 | cb(err, buffer);
|
---|
| 88 | });
|
---|
| 89 | });
|
---|
| 90 | }
|
---|
| 91 |
|
---|
| 92 | console.time('deflate');
|
---|
| 93 | for(let i = 0; i < 30000; ++i) {
|
---|
| 94 | deflate(payload, function (err, buffer) {});
|
---|
| 95 | }
|
---|
| 96 | t.onDone(function() {
|
---|
| 97 | console.timeEnd('deflate');
|
---|
| 98 | });
|
---|
| 99 | ```
|
---|
| 100 |
|
---|
| 101 | ## Install
|
---|
| 102 |
|
---|
| 103 | `npm install async-limiter`
|
---|
| 104 |
|
---|
| 105 | ## Test
|
---|
| 106 |
|
---|
| 107 | `npm test`
|
---|
| 108 |
|
---|
| 109 | ## API
|
---|
| 110 |
|
---|
| 111 | ### `var t = new Limiter([opts])`
|
---|
| 112 | Constructor. `opts` may contain inital values for:
|
---|
| 113 | * `t.concurrency`
|
---|
| 114 |
|
---|
| 115 | ## Instance methods
|
---|
| 116 |
|
---|
| 117 | ### `t.onDone(fn)`
|
---|
| 118 | `fn` will be called once and only once, when the queue is empty.
|
---|
| 119 |
|
---|
| 120 | ## Instance methods mixed in from `Array`
|
---|
| 121 | Mozilla has docs on how these methods work [here](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array).
|
---|
| 122 | ### `t.push(element1, ..., elementN)`
|
---|
| 123 | ### `t.unshift(element1, ..., elementN)`
|
---|
| 124 | ### `t.splice(index , howMany[, element1[, ...[, elementN]]])`
|
---|
| 125 |
|
---|
| 126 | ## Properties
|
---|
| 127 | ### `t.concurrency`
|
---|
| 128 | Max number of jobs the queue should process concurrently, defaults to `Infinity`.
|
---|
| 129 |
|
---|
| 130 | ### `t.length`
|
---|
| 131 | Jobs pending + jobs to process (readonly).
|
---|
| 132 |
|
---|