[6a3a178] | 1 | # minipass-json-stream
|
---|
| 2 |
|
---|
| 3 | Like [JSONStream](http://npm.im/JSONStream), but using Minipass streams
|
---|
| 4 |
|
---|
| 5 | ## install
|
---|
| 6 |
|
---|
| 7 | ```
|
---|
| 8 | npm install minipass-json-stream
|
---|
| 9 | ```
|
---|
| 10 |
|
---|
| 11 | ## example
|
---|
| 12 |
|
---|
| 13 | ```js
|
---|
| 14 |
|
---|
| 15 | const request = require('request')
|
---|
| 16 | const JSONStream = require('minipass-json-stream')
|
---|
| 17 | const es = require('event-stream')
|
---|
| 18 |
|
---|
| 19 | request({url: 'http://isaacs.couchone.com/registry/_all_docs'})
|
---|
| 20 | .pipe(JSONStream.parse('rows.*'))
|
---|
| 21 | .pipe(es.mapSync(function (data) {
|
---|
| 22 | console.error(data)
|
---|
| 23 | return data
|
---|
| 24 | }))
|
---|
| 25 | ```
|
---|
| 26 |
|
---|
| 27 | ## new JSONStream(options)
|
---|
| 28 |
|
---|
| 29 | Create a new stream. This is a [minipass](http://npm.im/minipass) stream
|
---|
| 30 | that is always set in `objectMode`. It emits objects parsed out of
|
---|
| 31 | string/buffer JSON input that match the supplied `path` option.
|
---|
| 32 |
|
---|
| 33 | ## JSONStream.parse(path)
|
---|
| 34 |
|
---|
| 35 | Return a new JSONStream object to stream values that match a path.
|
---|
| 36 |
|
---|
| 37 | (Equivalent to `new JSONStream({path})`.)
|
---|
| 38 |
|
---|
| 39 | ``` js
|
---|
| 40 | JSONStream.parse('rows.*.doc')
|
---|
| 41 | ```
|
---|
| 42 |
|
---|
| 43 | The `..` operator is the recursive descent operator from
|
---|
| 44 | [JSONPath](http://goessner.net/articles/JsonPath/), which will match a
|
---|
| 45 | child at any depth (see examples below).
|
---|
| 46 |
|
---|
| 47 | If your keys have keys that include `.` or `*` etc, use an array instead.
|
---|
| 48 | `['row', true, /^doc/]`.
|
---|
| 49 |
|
---|
| 50 | If you use an array, `RegExp`s, booleans, and/or functions. The `..`
|
---|
| 51 | operator is also available in array representation, using `{recurse:
|
---|
| 52 | true}`. any object that matches the path will be emitted as 'data' (and
|
---|
| 53 | `pipe`d down stream)
|
---|
| 54 |
|
---|
| 55 | If `path` is empty or null, no 'data' events are emitted.
|
---|
| 56 |
|
---|
| 57 | If you want to have keys emitted, you can prefix your `*` operator with
|
---|
| 58 | `$`: `obj.$*` - in this case the data passed to the stream is an object
|
---|
| 59 | with a `key` holding the key and a `value` property holding the data.
|
---|
| 60 |
|
---|
| 61 | ### Examples
|
---|
| 62 |
|
---|
| 63 | query a couchdb view:
|
---|
| 64 |
|
---|
| 65 | ``` bash
|
---|
| 66 | curl -sS localhost:5984/tests/_all_docs&include_docs=true
|
---|
| 67 | ```
|
---|
| 68 | you will get something like this:
|
---|
| 69 |
|
---|
| 70 | ``` js
|
---|
| 71 | {"total_rows":129,"offset":0,"rows":[
|
---|
| 72 | { "id":"change1_0.6995461115147918"
|
---|
| 73 | , "key":"change1_0.6995461115147918"
|
---|
| 74 | , "value":{"rev":"1-e240bae28c7bb3667f02760f6398d508"}
|
---|
| 75 | , "doc":{
|
---|
| 76 | "_id": "change1_0.6995461115147918"
|
---|
| 77 | , "_rev": "1-e240bae28c7bb3667f02760f6398d508","hello":1}
|
---|
| 78 | },
|
---|
| 79 | { "id":"change2_0.6995461115147918"
|
---|
| 80 | , "key":"change2_0.6995461115147918"
|
---|
| 81 | , "value":{"rev":"1-13677d36b98c0c075145bb8975105153"}
|
---|
| 82 | , "doc":{
|
---|
| 83 | "_id":"change2_0.6995461115147918"
|
---|
| 84 | , "_rev":"1-13677d36b98c0c075145bb8975105153"
|
---|
| 85 | , "hello":2
|
---|
| 86 | }
|
---|
| 87 | },
|
---|
| 88 | ]}
|
---|
| 89 | ```
|
---|
| 90 |
|
---|
| 91 | we are probably most interested in the `rows.*.doc`
|
---|
| 92 |
|
---|
| 93 | create a `JSONStream` that parses the documents from the feed like this:
|
---|
| 94 |
|
---|
| 95 | ``` js
|
---|
| 96 | var stream = JSONStream.parse(['rows', true, 'doc']) //rows, ANYTHING, doc
|
---|
| 97 |
|
---|
| 98 | stream.on('data', function(data) {
|
---|
| 99 | console.log('received:', data);
|
---|
| 100 | });
|
---|
| 101 |
|
---|
| 102 | //emits anything from _before_ the first match
|
---|
| 103 | stream.on('header', function (data) {
|
---|
| 104 | console.log('header:', data) // => {"total_rows":129,"offset":0}
|
---|
| 105 | })
|
---|
| 106 | ```
|
---|
| 107 |
|
---|
| 108 | awesome!
|
---|
| 109 |
|
---|
| 110 | In case you wanted the contents the doc emitted:
|
---|
| 111 |
|
---|
| 112 | ``` js
|
---|
| 113 | // equivalent to: 'rows.*.doc.$*'
|
---|
| 114 | var stream = JSONStream.parse([
|
---|
| 115 | 'rows',
|
---|
| 116 | true,
|
---|
| 117 | 'doc',
|
---|
| 118 | {emitKey: true}
|
---|
| 119 | ]) //rows, ANYTHING, doc, items in docs with keys
|
---|
| 120 |
|
---|
| 121 | stream.on('data', function(data) {
|
---|
| 122 | console.log('key:', data.key);
|
---|
| 123 | console.log('value:', data.value);
|
---|
| 124 | });
|
---|
| 125 | ```
|
---|
| 126 |
|
---|
| 127 | You can also emit the path:
|
---|
| 128 |
|
---|
| 129 | ``` js
|
---|
| 130 | var stream = JSONStream.parse([
|
---|
| 131 | 'rows',
|
---|
| 132 | true,
|
---|
| 133 | 'doc',
|
---|
| 134 | {emitPath: true}
|
---|
| 135 | ]) //rows, ANYTHING, doc, items in docs with keys
|
---|
| 136 |
|
---|
| 137 | stream.on('data', function(data) {
|
---|
| 138 | console.log('path:', data.path);
|
---|
| 139 | console.log('value:', data.value);
|
---|
| 140 | });
|
---|
| 141 | ```
|
---|
| 142 |
|
---|
| 143 | ### recursive patterns (..)
|
---|
| 144 |
|
---|
| 145 | `JSONStream.parse('docs..value')`
|
---|
| 146 | (or `JSONStream.parse(['docs', {recurse: true}, 'value'])` using an array)
|
---|
| 147 | will emit every `value` object that is a child, grand-child, etc. of the
|
---|
| 148 | `docs` object. In this example, it will match exactly 5 times at various depth
|
---|
| 149 | levels, emitting 0, 1, 2, 3 and 4 as results.
|
---|
| 150 |
|
---|
| 151 | ```js
|
---|
| 152 | {
|
---|
| 153 | "total": 5,
|
---|
| 154 | "docs": [
|
---|
| 155 | {
|
---|
| 156 | "key": {
|
---|
| 157 | "value": 0,
|
---|
| 158 | "some": "property"
|
---|
| 159 | }
|
---|
| 160 | },
|
---|
| 161 | {"value": 1},
|
---|
| 162 | {"value": 2},
|
---|
| 163 | {"blbl": [{}, {"a":0, "b":1, "value":3}, 10]},
|
---|
| 164 | {"value": 4}
|
---|
| 165 | ]
|
---|
| 166 | }
|
---|
| 167 | ```
|
---|
| 168 |
|
---|
| 169 | ## JSONStream.parse(pattern, map)
|
---|
| 170 |
|
---|
| 171 | (Equivalent to `new JSONStream({ pattern, map })`)
|
---|
| 172 |
|
---|
| 173 | provide a function that can be used to map or filter
|
---|
| 174 | the json output. `map` is passed the value at that node of the pattern,
|
---|
| 175 | if `map` return non-nullish (anything but `null` or `undefined`)
|
---|
| 176 | that value will be emitted in the stream. If it returns a nullish value,
|
---|
| 177 | nothing will be emitted.
|
---|
| 178 |
|
---|
| 179 | `JSONStream` also emits `'header'` and `'footer'` events,
|
---|
| 180 | the `'header'` event contains anything in the output that was before
|
---|
| 181 | the first match, and the `'footer'`, is anything after the last match.
|
---|
| 182 |
|
---|
| 183 | ## Acknowlegements
|
---|
| 184 |
|
---|
| 185 | This module is a fork of [JSONStream](http://npm.im/JSONStream) by Dominic
|
---|
| 186 | Tarr, modified and redistributed under the terms of the MIT license.
|
---|
| 187 |
|
---|
| 188 | this module depends on https://github.com/creationix/jsonparse
|
---|
| 189 | by Tim Caswell
|
---|