4 ms·
What method/model are you using for sparse search?
by elpalek 1y ago
What method/model are you using for sparse search?
- GeorgeCurtis 1y agoWe're going to use BM25. Currently it is just dense search. Coming very soon
- elpalek 1y agohave you thought about SPALDE models? ex: https://arxiv.org/abs/2109.10086 https://arxiv.org/abs/2109.10086
- GeorgeCurtis 1y agoLooks really interesting, I'll have a proper read. What would be your reasoning to incorporate this if we already have vector functionality and semantic search?
- elpalek 1y agomy project deals w/ non-english text, bm25 performance is middeling. Language specific sparse model helps.
- xavcochran 1y agoWe will definitely look into it. The SPLADE models look promising!
- xavcochran 1y agoSPALDE*