12 ms·

Big fan of HLL Apache foundation has a fantastic DataSketches library that includes HLL and many other powerful data analytics algorithms: https://datasketches

by tresil 4y ago

Big fan of HLL Apache foundation has a fantastic DataSketches library that includes HLL and many other powerful data analytics algorithms: https://datasketches.apache.org/ https://datasketches.apache.org/ Lee Rhodes has done an excellent introduction to this library - explaining some of the use cases, advantages, and things to be aware of when using these techniques: https://www.youtube.com/watch?v=nO9pauS-mGQ https://www.youtube.com/watch?v=nO9pauS-mGQ

twic 4y ago
On sketches, there is a genre of structure for estimating histogram-like statistics (median, 99th centile, etc) in fixed space, which i really like. Two examples: t-digest https://github.com/tdunning/t-digest https://github.com/tdunning/t-digest DDSketch https://github.com/DataDog/sketches-java https://github.com/DataDog/sketches-java
- arunk-s 4y ago
  In the same spirit there is also Circllhist from Circonus. https://arxiv.org/abs/2001.06561 https://arxiv.org/abs/2001.06561 There was a good discussion on approaches to store histogram here: https://news.ycombinator.com/item?id=31379383 https://news.ycombinator.com/item?id=31379383
- ranman 4y ago
  t-digest is used by AWS X-ray to render their charts