source: trip-planner-front/node_modules/hdr-histogram-js/README.md@ 6a3a178

Last change on this file since 6a3a178 was 6a3a178, checked in by Ema <ema_spirova@…>, 3 years ago

initial commit

  • Property mode set to 100644
File size: 15.8 KB
Line 
1[![Build Status](https://travis-ci.org/HdrHistogram/HdrHistogramJS.svg?branch=master)](https://travis-ci.org/HdrHistogram/HdrHistogramJS)
2
3# HdrHistogramJS
4
5TypeScript port of HdrHistogram for NodeJS and web browsers.
6Since version 2, HdrHistogramJS comes in 2 flavors: the good old TypeScript implementation and a brand new WebAssembly implementation!
7This new WebAssembly implementation leverages on AssemblyScript to bring a significant performance boost. Since some caution must be taken using this WebAssembly implementation it is not enabled by default.
8Check out the [WebAssembly section](#boosting-performances-with-webassembly-since-hdrhistogramjs-v2) for more details on this topic.
9Most features from original Java HdrHistogram implementation are implemented, including the following ones:
10
11- regular latency recording
12- latency recording with coordinated omissions correction
13- resizable bucket based histograms
14- memory optimized packed histograms
15- add and substract histograms
16- encoding and decoding compressed histograms
17
18# Dataviz
19
20HdrHistogramJS allows to display histograms without server-side processing. Hence, within your browser, you can:
21
22- Display histograms with this slightly modified version of the [hdrhistogram plotter](https://hdrhistogram.github.io/HdrHistogramJSDemo/plotFiles.html). With this one you can use base64 v2 encoded histograms as inputs.
23- Analyze log files with this [log analyzer](https://hdrhistogram.github.io/HdrHistogramJSDemo/logparser.html), inspired from the original [java/swing based log analyzer](https://github.com/HdrHistogram/HistogramLogAnalyzer).
24
25# Getting started
26
27This library is packaged as a UMD module, hence you can use it directly
28from JavaScript within a browser. To do so, you can simply include HdrHistogramJS file from github's release page:
29
30```
31<script src="https://github.com/HdrHistogram/HdrHistogramJS/releases/download/v2.0.1/hdrhistogram.umd.js"></script>
32```
33
34Then you will have access to classes and functions of the APIs using "hdr" prefix.
35
36You can also use HdrHistogramJS as a commonjs NodeJS module.
37Using npm you can get HdrHIstogramJS with the following command:
38
39```
40 npm i hdr-histogram-js
41```
42
43Or if you like yarn better:
44
45```
46 yarn add hdr-histogram-js
47```
48
49Note for TypeScript developers: since HdrHistogramJS has been written in TypeScript, definition files are embedded, no additional task is needed to get them.
50
51# API
52
53The examples below use ES6 syntax. You can check out demo sources
54for examples on how to use HdrHistogram directly within a browser, you should
55not have any surprise though.
56
57## Instantiate an histogram
58
59HdrHistogramJS provides several histogram implementations. The simplest way to get a new histogram instance is to use the _build()_ function. Here is how to use it:
60
61```
62import * as hdr from "hdr-histogram-js"
63
64const histogram = hdr.build();
65```
66
67If you need more control on the memory footprint of the instantiated histogram, you can be more specific using and optionnal build request parameter:
68
69```
70import * as hdr from "hdr-histogram-js"
71
72const histogram
73 = hdr.build(
74 {
75 bitBucketSize: 32, // may be 8, 16, 32, 64 or 'packed'
76 autoResize: true, // default value is true
77 lowestDiscernibleValue: 1, // default value is also 1
78 highestTrackableValue: 2, // can increase up to Number.MAX_SAFE_INTEGER
79 numberOfSignificantValueDigits: 3 // Number between 1 and 5 (inclusive)
80 useWebAssembly: false // default value is false, see WebAssembly section for details
81 }
82 );
83
84```
85
86BitBucketSize 'packed' options is available since HdrHistogramJS v1.2 . Like the Java packed implementation, it has a very low memory footprint but it is way slower than regular bucket based implementation.
87
88## Record values and retrieve metrics
89
90Once you have an histogram instance, in order to record a value you just need
91to call method recordValue() as below:
92
93```
94import * as hdr from "hdr-histogram-js"
95const histogram = hdr.build();
96...
97const latency = 1234;
98histogram.recordValue(latency);
99```
100
101The number passed as a parameter is expected to be an integer. If it is not the case, the decimal part will be ignored.
102
103Once you have recorded some values, you can get min, max, median values and of course percentiles values as shown below:
104
105```
106import * as hdr from "hdr-histogram-js"
107
108const h = hdr.build();
109h.recordValue(123);
110h.recordValue(122);
111h.recordValue(1244);
112
113console.log(h.minNonZeroValue); // 122
114console.log(h.maxValue); // 1244
115console.log(h.mean); // 486.333...
116console.log(h.getValueAtPercentile(90)); // 1244 as well
117console.log(h.summary); // { "p50": 123, ... , max: 1244, totalCount: 3 }
118```
119
120If youn need a live example you can also take alook at this [simple ping demo](https://hdrhistogram.github.io/HdrHistogramJSDemo/ping-demo.html) or this [HdrHistogramJS on HdrHistogramJS demo](https://hdrhistogram.github.io/HdrHistogramJSDemo/hdr-on-hdr.html).
121
122You can also very easily generate a textual
123representation of an histogram:
124
125```
126console.log(`Statistics ${h}`);
127
128// output will be:
129//
130// Statistics Histogram 32b {
131// "p50": 123,
132// "p75": 1240,
133// "p90": 1240,
134// "p97_5": 1240,
135// "p99": 1240,
136// "p99_9": 1240,
137// "p99_99": 1240,
138// "p99_999": 1240,
139// "max": 1244,
140// "totalCount": 3
141// }
142
143```
144
145You can also general an exhaustive textual representation similar to the one generated by the original Java HdrHistogram implementation:
146
147```
148import * as hdr from "hdr-histogram-js"
149
150const histogram = hdr.build();
151histogram.recordValue(25);
152histogram.recordValue(50);
153histogram.recordValue(75);
154const output = histogram.outputPercentileDistribution();
155
156// output will be:
157//
158// Value Percentile TotalCount 1/(1-Percentile)
159//
160// 25.000 0.000000000000 1 1.00
161// ...
162// 75.000 0.700000000000 3 3.33
163// 75.000 1.000000000000 3
164//#[Mean = 50.000, StdDeviation = 20.412]
165//#[Max = 75.000, Total count = 3]
166//#[Buckets = 43, SubBuckets = 2048]
167
168```
169
170## Dealing with coordinated omissions
171
172If you are recording values at a fixed rate,
173you can correct coordinated omissions while recording values:
174
175```
176histogram.recordValueWithExpectedInterval(1234, 100);
177```
178
179If you prefer to apply correction afterward:
180
181```
182const correctedHistogram
183 = histogram.copyCorrectedForCoordinatedOmission(100);
184```
185
186## Boosting performances with WebAssembly
187
188Since version 2, HdrHistogramJS leverages on WebAssembly to speed up computations. Depending on the use case, the performance boost can be as high as twice as fast :)
189Everything has been done to make your life easier when using HdrHistogramJS WebAssembly implementation, but since the usage a little bit different WebAssembly is not enabled by default.
190To benefit from WebAssembly performance boost, there are three things to take care of:
191
192- Bootstrap the HdrHistogramJS WebAssembly module at application startup
193- Build a WebAssembly histogram setting the useWebAssembly flag to true
194- Explicitely ask to free up memory by calling _histogram.destroy()_ once an histogram is not needed anymore.
195
196Even if under the cover a WebAssembly histogram is very different from a regular JS based histogram, both provide exactly the same interface.
197The code fragment below shows how to instantiate a resizable 32 bits WebAssembly histogram:
198
199```
200import * as hdr from "hdr-histogram-js"
201
202// If you are on the browser side, you need to
203// load asynchronously HdrHistogramJS WASM module
204await hdr.initWebAssembly();
205
206// If you are on the server side, you can
207// load synchronously HdrHistogramJS WASM module
208hdr.initWebAssemblySync();
209
210const histogram = hdr.build({ useWebAssembly: true });
211
212// you can now use your histogram the same way you would do
213// with a regular "JS histogram"
214histogram.recordValue(42);
215console.log(histogram.outputPercentileDistribution());
216
217// free up memory once the histogram is not needed anymore,
218// otherwise WebAssembly memory footprint will keep growing
219// each time an histogram is created
220histogram.destroy();
221
222```
223
224Note: If you want to use this feature on the browser side, along with the UMD package, you need to add external dependency
225"pako". "pako" is mandatory to bootstrap the WASM module which is compressed to save some weight.
226
227## Encode & decode
228
229You can encode and decode base64 compressed histograms. Hence you can decode base64 compressed histograms produced by other implementations of HdrHistogram (Java, C#, Rust, ...).
230The code fragment below shows how to encode an histogram:
231
232```
233import * as hdr from "hdr-histogram-js"
234
235const histogram = hdr.build();
236histogram.recordvalue(42);
237const encodedString = hdr.encodeIntoBase64String(histogram);
238// gives something that looks like "HISTFAAAAB542pNpmSzMwMDAxAABzFCaEUoz2X+AMIKZAEARAtM="
239```
240
241Then to decode an histogram you can use this chunk of code:
242
243```
244import * as hdr from "hdr-histogram-js"
245
246const encodedString = "HISTFAAAAB542pNpmSzMwMDAxAABzFCaEUoz2X+AMIKZAEARAtM=";
247const histogram = hdr.decodeFromCompressedBase64(encodedString);
248```
249
250In the above code fragment, 'histogram' is a regular 32b bucket histogram. Other types of histograms can be specified using additionnal parameters. Below a code fragment where a WebAssembly packed histogram is used:
251
252```
253import * as hdr from "hdr-histogram-js"
254
255const encodedString = "HISTFAAAAB542pNpmSzMwMDAxAABzFCaEUoz2X+AMIKZAEARAtM=";
256const histogram = hdr.decodeFromCompressedBase64(encodedString, 'packed', true);
257```
258
259If you want to use this feature along with the UMD package, you need to add external dependency
260"pako". "pako" is used for zlib compression. Using npm you should get
261it as a transitive dependency, otherwise you need to add it in
262your html page.
263
264You can check out [this demo](https://hdrhistogram.github.io/HdrHistogramJSDemo/decoding-demo.html) or this [plotter on steroid](https://hdrhistogram.github.io/HdrHistogramJSDemo/plotFiles.html) to see this feature live!
265_Be aware that only latest V2 encoding has been implemented, please raise a github issue if you need to see other versions implemented_
266
267## Histogram logs
268
269HistogramLogWriter and HistogramLogReader classes have been migrated and the API is quite similar to the one you might have used with the Java version.
270Below a simple usage example of the HistogramLogWriter, where the log contents are appended to a string variable:
271
272```
273import * as hdr from "hdr-histogram-js"
274
275let buffer: string;
276const writer = new hdr.HistogramLogWriter(content => {
277 buffer += content;
278});
279const histogram = hdr.build();
280histogram.startTimeStampMsec = 1234001;
281histogram.endTimeStampMsec = 1235123;
282
283...
284
285histogram.recordValue(123000);
286
287writer.outputLogFormatVersion();
288writer.outputLegend();
289writer.outputIntervalHistogram(histogram);
290```
291
292As for the reading part, if you know a little bit the Java version, the following code fragment will sound familiar:
293
294```
295const reader = new hdr.HistogramLogReader(fileContent);
296let histogram;
297while ((histogram = reader.nextIntervalHistogram()) != null) {
298 // iterate on all histogram log lines
299 ...
300
301}
302```
303
304# Performance tips
305
306HdrHistogramJS stores values in memory buckets. Memory footprint of an histogram heavily depends on 3 things:
307
308- the bucket size. A bucket can take 8, 16, 32 or 64 bits of memory. 32 bits buckets is the default.
309- the precision of the histogram (i.e. the number of significant value digits). You can have up to 5 value digits, 3 value digits (default) should be enough for most use cases.
310- the allowed range of values. You can tunned this range with constructor/builder parameters _lowestDiscernibleValue_ and _highestTrackableValue_. If you are not sure of these values, the best option is to use flag _autoResize_, set to true by default.
311
312While tunning memory footprint, _estimatedFootprintInBytes_ histogram property can get quite useful since it gives you a clear indicator of the memory cost:
313
314```
315const histogram = hdr.build();
316console.log(histogram.estimatedFootprintInBytes);
317```
318
319If in your context saving memory is more important than saving CPU cycles, 'packed' bucket size is highly recommended. Available since HdrHistogramJS v1.2.0, this mode enables a very effective memory compression algorithm:
320
321```
322const histogram = hdr.build({ bitBucketSize: "packed" });
323console.log(histogram.estimatedFootprintInBytes);
324```
325
326Last but not least, unless you are targetting very old browsers or a very old NodeJS version, you should turn on WebAssembly mode. Available since HdrHistogramJS v2.0.0, this mode is often twice as fast as regular JS mode. Also all bucket size options are available with WebAssembly, including 'packed':
327
328```
329const histogram = hdr.build({ bitBucketSize: "packed", useWebAssembly: true });
330console.log(histogram.estimatedFootprintInBytes);
331```
332
333# Tree Shaking
334
335The above examples use a convenient 'barrel' index file. Using this barrel, you cannot leverage on the tree shaking features of your favorite bundler. Hence the size of your JavaScript bundle may increase significantly. If you need to optimize the size of your bundle, you can import HdrHistogram modules as shown in code fragment below:
336
337```
338import Int32Histogram from "hdr-histogram-js/dist/Int32Histogram"
339
340const histogram = new Int32Histogram(1, 2, 3);
341histogram.autoResize = true;
342
343histogram.recordValue(...);
344
345```
346
347# Migrating from v1 to v2
348
349For most users, migration from HdrHistogramJS v1 to v2 should be smooth. However since HdrHistogramJS v2 does not stick anymore with HdrHistogram Java API, you might run into some breaking changes.
350Prior to v2, _hdr.build()_ was returning an _AbstractHistogram_ instance. _AbstractHistogram_ does not exist anymore and has been replaced by the _Histogram_ interface. Most methods from _AbstractHistogram_ still exist in new interface _Histogram_, however in order to have a more JS idiomatic API, getter methods such as _getMean()_ or _getTotalCount()_ have been replaced by JS properties:
351
352```
353// HdrHistogramJS v1
354const histogram: AbstractHistogram = hdr.build();
355const statistics = {
356 count: histogram.getTotalCount(),
357 mean: histogram.getMean(),
358 p99: histogram.getValueAtPercentile(99),
359}
360
361// becomes with HdrHistogramJS v2
362const histogram: Histogram = hdr.build();
363const statistics = {
364 count: histogram.totalCount,
365 mean: histogram.mean,
366 p99: histogram.getValueAtPercentile(99),
367}
368```
369
370Module paths also change a little bit with v2. Hence if you were importing specific modules as described in the tree shaking section, you need to change a little bit your code as below:
371
372```
373// HdrHistogramJS v1
374import AbstractHistogram from "hdr-histogram-js/AbstractHistogram"
375import Int32Histogram from "hdr-histogram-js/Int32Histogram"
376
377// becomes with HdrHistogramJS v2
378import Histogram from "hdr-histogram-js/dist/Histogram"
379import Int32Histogram from "hdr-histogram-js/dist/Int32Histogram"
380```
381
382# Design & Limitations
383
384The code is almost a direct port of the Java version.
385Optimisation based on inheritance to avoid false sharing
386might not be relevant in JS, but I believe that keeping
387the same code structure might be handy to keep the code up to date
388with the Java version in the future.
389
390Main limitations comes from number support in JavaScript.
391There is no such thing as 64b integers in JavaScript. Everything is "number",
392and a number is safe as an integer up to 2^53.
393The most annoying issue encountered during the code migration,
394is that bit operations, heavily used within original HdrHistogram,
395only work on the first 32 bits. That means that the following JavaScript expression is evaluated as true:
396
397```
398Math.pow(2, 31) << 1 === 0 // sad but true
399```
400
401Anyway bit shift operations are not really optimized
402in most browser, so... everything related to bits have been
403converted to good old arithmetic expressions in the process
404of converting the Java code to TypeScript.
405With WebAssembly and AssemblyScript things are different. HdrHistogramsJS AssemblyScript source code is closer to the original Java code since all the limitations mentioned above does not apply to WebAssembly :)
Note: See TracBrowser for help on using the repository browser.