9 ms·
Egress costs? I’m really surprised by this. Thanks for sharing.
by heywoods 1y ago
Egress costs? I’m really surprised by this. Thanks for sharing.
- qaq 1y agoSry maybe should've being more clear it was a sarcastic remark. The whole point of doing vector db search is to feed LLM with very targeted context so you can save $ on API calls to LLM.
- infecto 1y agoThat’s not the whole point it’s in the intersection of reducing tokens sent but also getting search both specific and generic enough to capture the correct context data.
- j45 1y agoIt's possible to create linking documents between the documents to help smooth out things in some cases.
- heywoods 1y agoNo worries. I should probably make sure I have at least a token understanding of the topic cloud based architecture before commenting next time haha.
- andreasgl 1y agoThey’re likely using an HNSW index, which typically requires a lot of memory for large data sets.