6 ms·
Vector DBs should never have existed in the first place. I feel sorry for the agent startups though.
by bluecrab 3y ago
Vector DBs should never have existed in the first place. I feel sorry for the agent startups though.
- m3kw9 3y agoHow does this absolve vectordbs
- danielbln 3y agoIt doesn't, but semantic search is a lot less relevant if you can squeeze 350 pages of text into the context.
- quinncom 3y agoOpenAI charges for all those input tokens. If an app requires squeezing 350 pages of content in every request is going to cost more. Vector DB still relevant for cost and speed.
- gk1 3y agoBesides the cost factor, stuffing the context window can actually make the results worse. https://www.pinecone.io/blog/why-use-retrieval-instead-of-larger-context/ https://www.pinecone.io/blog/why-use-retrieval-instead-of-la...
- dragonwriter 3y agoIf you are using OpenAI, the new Assistants API looks like itnwill handle internally what you used to handle externally with a vector DB for RAG (and for some things, GPT-4-Turbo’s 128k context window will make it unnecessary entirely.) There are some other uses for Vector DBs than RAG for LLMs, and there are reasons people might use non-OpenAI LLMs with RAG, so there is still a role for VectorDBs, but it shrunk a lot with this.
- oezi 3y agoOpenAI is still way too expensive to run a corporate knowledge base on top
- m3kw9 3y agoIt’s more reliable than chatpdfs that relies on vector search. With vector db all you are doing is doing a fuzzy search and then sending in that relevant portion near that text and send it to a LLM model as part of a prompt. It misses info.
- dragonwriter 3y agoI'd be very surprised if the Assistants API is not doing RAG with a vector DB behind the scenes with the supplied files.