7 ms·
On the subject of foreign data wrappers, there's an extension for storing columnar data in Postgres which is implemented via this concept: https://github.com/ci
by thinkpad20 6y ago
On the subject of foreign data wrappers, there's an extension for storing columnar data in Postgres which is implemented via this concept: https://github.com/citusdata/cstore_fdw https://github.com/citusdata/cstore_fdw
I haven't used it myself but it's pretty cool that it's out there.
- mildbyte 6y agoSplitgraph co-founder here. cstore_fdw is great (we even use it in Splitgraph to store data [0])! It's not as fast as purpose-built columnar stores like MonetDB. However, it plugs seamlessly into PostgreSQL and supports all types, even those added via extensions. Read performance for OLAP workloads like aggregations is better than PostgreSQL [1] and it has a much smaller IO load and disk footprint (you can get long runs of similar values in column-oriented storage, which compress better). As a nice bonus, you can simply swap cstore_fdw files in and out of the database without having to "load" them into PostgreSQL. We use this idea too to enable data sharing. [0] https://www.splitgraph.com/docs/concepts/objects https://www.splitgraph.com/docs/concepts/objects [1] https://tech.marksblogg.com/billion-nyc-taxi-rides-postgresql.html https://tech.marksblogg.com/billion-nyc-taxi-rides-postgresq...
- senderista 6y agoAlternative storage formats like columnar storage are better served by Postgres’s new custom access methods (introduced in PG12 I believe).