| [62b2964] | 1 | # Split2(matcher, mapper, options)
|
|---|
| 2 |
|
|---|
| 3 | 
|
|---|
| 4 |
|
|---|
| 5 | Break up a stream and reassemble it so that each line is a chunk.
|
|---|
| 6 | `split2` is inspired by [@dominictarr](https://github.com/dominictarr) [`split`](https://github.com/dominictarr/split) module,
|
|---|
| 7 | and it is totally API compatible with it.
|
|---|
| 8 | However, it is based on Node.js core [`Transform`](https://nodejs.org/api/stream.html#stream_new_stream_transform_options).
|
|---|
| 9 |
|
|---|
| 10 | `matcher` may be a `String`, or a `RegExp`. Example, read every line in a file ...
|
|---|
| 11 |
|
|---|
| 12 | ``` js
|
|---|
| 13 | fs.createReadStream(file)
|
|---|
| 14 | .pipe(split2())
|
|---|
| 15 | .on('data', function (line) {
|
|---|
| 16 | //each chunk now is a separate line!
|
|---|
| 17 | })
|
|---|
| 18 |
|
|---|
| 19 | ```
|
|---|
| 20 |
|
|---|
| 21 | `split` takes the same arguments as `string.split` except it defaults to '/\r?\n/', and the optional `limit` paremeter is ignored.
|
|---|
| 22 | [String#split](https://developer.mozilla.org/en/JavaScript/Reference/Global_Objects/String/split)
|
|---|
| 23 |
|
|---|
| 24 | `split` takes an optional options object on it's third argument, which
|
|---|
| 25 | is directly passed as a
|
|---|
| 26 | [Transform](https://nodejs.org/api/stream.html#stream_new_stream_transform_options)
|
|---|
| 27 | option.
|
|---|
| 28 |
|
|---|
| 29 | Additionally, the `.maxLength` and `.skipOverflow` options are implemented, which set limits on the internal
|
|---|
| 30 | buffer size and the stream's behavior when the limit is exceeded. There is no limit unless `maxLength` is set. When
|
|---|
| 31 | the internal buffer size exceeds `maxLength`, the stream emits an error by default. You may also set `skipOverflow` to
|
|---|
| 32 | true to suppress the error and instead skip past any lines that cause the internal buffer to exceed `maxLength`.
|
|---|
| 33 |
|
|---|
| 34 | Calling `.destroy` will make the stream emit `close`. Use this to perform cleanup logic
|
|---|
| 35 |
|
|---|
| 36 | ``` js
|
|---|
| 37 | var splitFile = function(filename) {
|
|---|
| 38 | var file = fs.createReadStream(filename)
|
|---|
| 39 |
|
|---|
| 40 | return file
|
|---|
| 41 | .pipe(split2())
|
|---|
| 42 | .on('close', function() {
|
|---|
| 43 | // destroy the file stream in case the split stream was destroyed
|
|---|
| 44 | file.destroy()
|
|---|
| 45 | })
|
|---|
| 46 | }
|
|---|
| 47 |
|
|---|
| 48 | var stream = splitFile('my-file.txt')
|
|---|
| 49 |
|
|---|
| 50 | stream.destroy() // will destroy the input file stream
|
|---|
| 51 | ```
|
|---|
| 52 |
|
|---|
| 53 | # NDJ - Newline Delimited Json
|
|---|
| 54 |
|
|---|
| 55 | `split2` accepts a function which transforms each line.
|
|---|
| 56 |
|
|---|
| 57 | ``` js
|
|---|
| 58 | fs.createReadStream(file)
|
|---|
| 59 | .pipe(split2(JSON.parse))
|
|---|
| 60 | .on('data', function (obj) {
|
|---|
| 61 | //each chunk now is a js object
|
|---|
| 62 | })
|
|---|
| 63 | .on("error", function(error) {
|
|---|
| 64 | //handling parsing errors
|
|---|
| 65 | })
|
|---|
| 66 | ```
|
|---|
| 67 |
|
|---|
| 68 | However, in [@dominictarr](https://github.com/dominictarr) [`split`](https://github.com/dominictarr/split) the mapper
|
|---|
| 69 | is wrapped in a try-catch, while here it is not: if your parsing logic can throw, wrap it yourself. Otherwise, you can also use the stream error handling when mapper function throw.
|
|---|
| 70 |
|
|---|
| 71 | # License
|
|---|
| 72 |
|
|---|
| 73 | Copyright (c) 2014-2021, Matteo Collina <hello@matteocollina.com>
|
|---|
| 74 |
|
|---|
| 75 | Permission to use, copy, modify, and/or distribute this software for any
|
|---|
| 76 | purpose with or without fee is hereby granted, provided that the above
|
|---|
| 77 | copyright notice and this permission notice appear in all copies.
|
|---|
| 78 |
|
|---|
| 79 | THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
|
|---|
| 80 | WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
|
|---|
| 81 | MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
|
|---|
| 82 | ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
|
|---|
| 83 | WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
|
|---|
| 84 | ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR
|
|---|
| 85 | IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
|
|---|