6 ms·
Would that kind of data not compress like crazy? Or would they need to keep all that data hot and fast.
by rickette 1y ago
Would that kind of data not compress like crazy? Or would they need to keep all that data hot and fast.
- guerby 1y agoFrom just a single exchange you can reach up to 1 million messages of order book change per second https://www.nasdaqtrader.com/snippets/inet2.html https://www.nasdaqtrader.com/snippets/inet2.html Message Volume 1,684,103,265 Messages per Second 1,134,640 Order Volume 871,875,595 Orders per Second 581,696 Share Volume 12,814,454,760 Executions per Second 193,350 Also if you look at equity derivative products which have parameters like type call/put, strike, maturity can be hundreds of financial products for one underlying stock. I worked in this sector and volume of data is a real challenge, no wonder you often get custom software to handle that :)
- rickette 1y agoThanks for the insight!
- pcthrowaway 1y agoHow do you propose lossless compression for all orderbook data? Of course if you are willing to lose granularity/information, it can be compressed a lot