15 ms·
Depends on the domain. There's a bunch of sciences using large datasets served up efficiently using static file formats, e.g., https://zarr.dev/ https://zarr.de
by lmc 3mo ago
Depends on the domain. There's a bunch of sciences using large datasets served up efficiently using static file formats, e.g., https://zarr.dev/ https://zarr.dev/ and https://parquet.apache.org/ https://parquet.apache.org/
- mmarian 3mo agoAdmit I'm not familiar with that domain. But don't people use managed services even there? In my job we host parquet files on S3.
- lmc 3mo agoTrue, we use S3 a lot too. But it's interesting to think of alternatives like this project, e.g., for when we don't have the setup for a full on block storage service.