1 | # minipass-json-stream
|
---|
2 |
|
---|
3 | Like [JSONStream](http://npm.im/JSONStream), but using Minipass streams
|
---|
4 |
|
---|
5 | ## install
|
---|
6 |
|
---|
7 | ```
|
---|
8 | npm install minipass-json-stream
|
---|
9 | ```
|
---|
10 |
|
---|
11 | ## example
|
---|
12 |
|
---|
13 | ```js
|
---|
14 |
|
---|
15 | const request = require('request')
|
---|
16 | const JSONStream = require('minipass-json-stream')
|
---|
17 | const es = require('event-stream')
|
---|
18 |
|
---|
19 | request({url: 'http://isaacs.couchone.com/registry/_all_docs'})
|
---|
20 | .pipe(JSONStream.parse('rows.*'))
|
---|
21 | .pipe(es.mapSync(function (data) {
|
---|
22 | console.error(data)
|
---|
23 | return data
|
---|
24 | }))
|
---|
25 | ```
|
---|
26 |
|
---|
27 | ## new JSONStream(options)
|
---|
28 |
|
---|
29 | Create a new stream. This is a [minipass](http://npm.im/minipass) stream
|
---|
30 | that is always set in `objectMode`. It emits objects parsed out of
|
---|
31 | string/buffer JSON input that match the supplied `path` option.
|
---|
32 |
|
---|
33 | ## JSONStream.parse(path)
|
---|
34 |
|
---|
35 | Return a new JSONStream object to stream values that match a path.
|
---|
36 |
|
---|
37 | (Equivalent to `new JSONStream({path})`.)
|
---|
38 |
|
---|
39 | ``` js
|
---|
40 | JSONStream.parse('rows.*.doc')
|
---|
41 | ```
|
---|
42 |
|
---|
43 | The `..` operator is the recursive descent operator from
|
---|
44 | [JSONPath](http://goessner.net/articles/JsonPath/), which will match a
|
---|
45 | child at any depth (see examples below).
|
---|
46 |
|
---|
47 | If your keys have keys that include `.` or `*` etc, use an array instead.
|
---|
48 | `['row', true, /^doc/]`.
|
---|
49 |
|
---|
50 | If you use an array, `RegExp`s, booleans, and/or functions. The `..`
|
---|
51 | operator is also available in array representation, using `{recurse:
|
---|
52 | true}`. any object that matches the path will be emitted as 'data' (and
|
---|
53 | `pipe`d down stream)
|
---|
54 |
|
---|
55 | If `path` is empty or null, no 'data' events are emitted.
|
---|
56 |
|
---|
57 | If you want to have keys emitted, you can prefix your `*` operator with
|
---|
58 | `$`: `obj.$*` - in this case the data passed to the stream is an object
|
---|
59 | with a `key` holding the key and a `value` property holding the data.
|
---|
60 |
|
---|
61 | ### Examples
|
---|
62 |
|
---|
63 | query a couchdb view:
|
---|
64 |
|
---|
65 | ``` bash
|
---|
66 | curl -sS localhost:5984/tests/_all_docs&include_docs=true
|
---|
67 | ```
|
---|
68 | you will get something like this:
|
---|
69 |
|
---|
70 | ``` js
|
---|
71 | {"total_rows":129,"offset":0,"rows":[
|
---|
72 | { "id":"change1_0.6995461115147918"
|
---|
73 | , "key":"change1_0.6995461115147918"
|
---|
74 | , "value":{"rev":"1-e240bae28c7bb3667f02760f6398d508"}
|
---|
75 | , "doc":{
|
---|
76 | "_id": "change1_0.6995461115147918"
|
---|
77 | , "_rev": "1-e240bae28c7bb3667f02760f6398d508","hello":1}
|
---|
78 | },
|
---|
79 | { "id":"change2_0.6995461115147918"
|
---|
80 | , "key":"change2_0.6995461115147918"
|
---|
81 | , "value":{"rev":"1-13677d36b98c0c075145bb8975105153"}
|
---|
82 | , "doc":{
|
---|
83 | "_id":"change2_0.6995461115147918"
|
---|
84 | , "_rev":"1-13677d36b98c0c075145bb8975105153"
|
---|
85 | , "hello":2
|
---|
86 | }
|
---|
87 | },
|
---|
88 | ]}
|
---|
89 | ```
|
---|
90 |
|
---|
91 | we are probably most interested in the `rows.*.doc`
|
---|
92 |
|
---|
93 | create a `JSONStream` that parses the documents from the feed like this:
|
---|
94 |
|
---|
95 | ``` js
|
---|
96 | var stream = JSONStream.parse(['rows', true, 'doc']) //rows, ANYTHING, doc
|
---|
97 |
|
---|
98 | stream.on('data', function(data) {
|
---|
99 | console.log('received:', data);
|
---|
100 | });
|
---|
101 |
|
---|
102 | //emits anything from _before_ the first match
|
---|
103 | stream.on('header', function (data) {
|
---|
104 | console.log('header:', data) // => {"total_rows":129,"offset":0}
|
---|
105 | })
|
---|
106 | ```
|
---|
107 |
|
---|
108 | awesome!
|
---|
109 |
|
---|
110 | In case you wanted the contents the doc emitted:
|
---|
111 |
|
---|
112 | ``` js
|
---|
113 | // equivalent to: 'rows.*.doc.$*'
|
---|
114 | var stream = JSONStream.parse([
|
---|
115 | 'rows',
|
---|
116 | true,
|
---|
117 | 'doc',
|
---|
118 | {emitKey: true}
|
---|
119 | ]) //rows, ANYTHING, doc, items in docs with keys
|
---|
120 |
|
---|
121 | stream.on('data', function(data) {
|
---|
122 | console.log('key:', data.key);
|
---|
123 | console.log('value:', data.value);
|
---|
124 | });
|
---|
125 | ```
|
---|
126 |
|
---|
127 | You can also emit the path:
|
---|
128 |
|
---|
129 | ``` js
|
---|
130 | var stream = JSONStream.parse([
|
---|
131 | 'rows',
|
---|
132 | true,
|
---|
133 | 'doc',
|
---|
134 | {emitPath: true}
|
---|
135 | ]) //rows, ANYTHING, doc, items in docs with keys
|
---|
136 |
|
---|
137 | stream.on('data', function(data) {
|
---|
138 | console.log('path:', data.path);
|
---|
139 | console.log('value:', data.value);
|
---|
140 | });
|
---|
141 | ```
|
---|
142 |
|
---|
143 | ### recursive patterns (..)
|
---|
144 |
|
---|
145 | `JSONStream.parse('docs..value')`
|
---|
146 | (or `JSONStream.parse(['docs', {recurse: true}, 'value'])` using an array)
|
---|
147 | will emit every `value` object that is a child, grand-child, etc. of the
|
---|
148 | `docs` object. In this example, it will match exactly 5 times at various depth
|
---|
149 | levels, emitting 0, 1, 2, 3 and 4 as results.
|
---|
150 |
|
---|
151 | ```js
|
---|
152 | {
|
---|
153 | "total": 5,
|
---|
154 | "docs": [
|
---|
155 | {
|
---|
156 | "key": {
|
---|
157 | "value": 0,
|
---|
158 | "some": "property"
|
---|
159 | }
|
---|
160 | },
|
---|
161 | {"value": 1},
|
---|
162 | {"value": 2},
|
---|
163 | {"blbl": [{}, {"a":0, "b":1, "value":3}, 10]},
|
---|
164 | {"value": 4}
|
---|
165 | ]
|
---|
166 | }
|
---|
167 | ```
|
---|
168 |
|
---|
169 | ## JSONStream.parse(pattern, map)
|
---|
170 |
|
---|
171 | (Equivalent to `new JSONStream({ pattern, map })`)
|
---|
172 |
|
---|
173 | provide a function that can be used to map or filter
|
---|
174 | the json output. `map` is passed the value at that node of the pattern,
|
---|
175 | if `map` return non-nullish (anything but `null` or `undefined`)
|
---|
176 | that value will be emitted in the stream. If it returns a nullish value,
|
---|
177 | nothing will be emitted.
|
---|
178 |
|
---|
179 | `JSONStream` also emits `'header'` and `'footer'` events,
|
---|
180 | the `'header'` event contains anything in the output that was before
|
---|
181 | the first match, and the `'footer'`, is anything after the last match.
|
---|
182 |
|
---|
183 | ## Acknowlegements
|
---|
184 |
|
---|
185 | This module is a fork of [JSONStream](http://npm.im/JSONStream) by Dominic
|
---|
186 | Tarr, modified and redistributed under the terms of the MIT license.
|
---|
187 |
|
---|
188 | this module depends on https://github.com/creationix/jsonparse
|
---|
189 | by Tim Caswell
|
---|