8 ms·

There is no spec. Personally I hope that the existing indexes (bloom filters, zone maps) get re-designed to fit into a paradigm where parquet itself has more fi

There is no spec. Personally I hope that the existing indexes (bloom filters, zone maps) get re-designed to fit into a paradigm where parquet itself has more first class support for multiple levels of indexes embedded in the file and conventions for how those common types. That is, start with Wild West and define specs as needed

alamb 1y ago
> That is, start with Wild West and define specs as needed Yes this is my personal hope as well -- if there are new index types that are widespread, they can be incorporated formally into the spec However, changing the spec is a non trivial process and requires significant consensus and engineering Thus the methods used in the blog can be used to use indexes prior to any spec change and potentially as a way to prototype / prove out new potential indexes (note I am an author)