5 ms·

I mean, it's pretty damn reliable now. Has been for simple high contrast Q&A for a while now. I've just been doing research and experiments for work related st

I mean, it's pretty damn reliable now. Has been for simple high contrast Q&A for a while now. I've just been doing research and experiments for work related stuff. Typically we've used plain embeddings for a lot of high contrast documents aka discrete facts. However I've been working with a >1000 page document of complex procedures with incredibly low contrast where embedding falls flat. There's top down/graph searching, bottom up/embedded; alts like colbert, reranking, reasoning, search agents and now (though seemingly quite new) specific search agent models. Ultimately I found that a reasoning enabled search agent doing a hybrid of bottom up (with reranking) followed by top down, gave the absolute best results. Paired with Luna for cheaper and faster tokens it benchmarks pretty well even for vague references to procedures. I would imagine that search specific models just coming out are even better and I'll have to evaluate using these but for now the above works well for us. Having an agent get vector search results to use as anchors and then being able to explore the sections and subsections above that, then eventually digesting as much as is relevant (big context, cheap tokens) is amazing.

mdp2021 1mo ago
Very interesting and very good (though, let us say, much more recent that ancient): only, it paradoxically does suggest that the computational cost (when compared either to a simple "search by vectors distance" or to the task failures that we can have witnessed by chief implementations*) strongly suggest the benefit of 1000x speed boosting, energy conservative Taalas board. *(Of course it has "always" worked well for «simple high contrast Q&A», ever since the base embeddings technology worked properly: that is almost by definition; it is on real world use cases, where the nuances of reality are present, that it failed miserably.)

mdp2021 1mo ago

Very interesting and very good (though, let us say, much more recent that ancient): only, it paradoxically does suggest that the computational cost (when compared either to a simple "search by vectors distance" or to the task failures that we can have witnessed by chief implementations*) strongly suggest the benefit of 1000x speed boosting, energy conservative Taalas board. *(Of course it has "always" worked well for «simple high contrast Q&A», ever since the base embeddings technology worked properly: that is almost by definition; it is on real world use cases, where the nuances of reality are present, that it failed miserably.)