5 ms·

Surprise, your data warehouse can RAG

18 points by ukd1 2y ago

bun_terminator 2y ago
I don't know what a RAG is (and apparently it's forbidden to explain). And at this point, I'm afraid to ask.
- fragmede 2y ago
  Have you tried asking Google LLM RAG or ChatGPT what RAG in the context of LLMs is?
- franciscassel 2y ago
  Taken from the same blog: "Roughly, RAG is runtime prompt engineering where you build a system to dynamically add relevant things to your prompt before you ask the agent for an answer."
- maciejgryka 2y ago
  > “Retrieval-Augmented Generation” is nothing more than a fancy way of saying “including helpful information in your LLM prompt.”
29athrowaway 2y ago
RAG is as valuable as the data you can retrieve. If the amount of data is small you don't need the flexibility of RAG. And if it is irrelevant it will stay irrelevant after found.
- maciejgryka 2y ago
  For sure, it's only worth doing if you actually have so much relevant data that it doesn't fit in the context! This is definitely the case for us for this problem, but it's not universal.
  - 29athrowaway 2y ago
    To me the RAG hype is just the sudden rediscovery of information retrieval by money hounds that did not care about AI/ML for decades and now are in panic mode due to FOMO.
    - maciejgryka 2y ago
      There's hype and FOMO for sure and you're right that there's lots to learn from information retrieval work. But why be dismissive of the whole thing? People learning from past research and applying it in new contexts seems like a good thing and there's people (like https://x.com/bclavie https://x.com/bclavie and https://x.com/jobergum https://x.com/jobergum), who are legit experts in information retrieval and are showing how to apply it to RAG properly.
      - 29athrowaway 2y ago
        There are knowledgeable people out there with excellent use-cases for RAG, to find data that is relevant. And they come from years of using the previous state of the art techniques in information retrieval. They have all my support and I am looking forward to read more about it. But the majority of talk about RAG is coming from hype driven individuals that produce low merit and uninformed opinions.
deleted 2y ago
[deleted]
maciejgryka 2y ago
This is one of the things we learned recently about building production workflows with LLMs. Happy to answer any questions/feedback here <3
- ukd1 2y ago
  For me it's nice to see the re-use of existing infra - big query. Personally, I was rooting for Postgres, but your logic of why not makes sense! Great post.
rodrigovicuna 2y ago
Here to learn more about this. Important for a startup I know.
djhn 2y ago
Reading this was very valuable. I really appreciate the Vespa mention and introduction to their Multi-Vector HNSW Indexing - I’ve recently thought a lot about how difficult chunking is and this seems like a promising avenue.

18 points by ukd1 2y ago

bun_terminator 2y ago

I don't know what a RAG is (and apparently it's forbidden to explain). And at this point, I'm afraid to ask.

fragmede 2y ago
Have you tried asking Google LLM RAG or ChatGPT what RAG in the context of LLMs is?
franciscassel 2y ago
Taken from the same blog: "Roughly, RAG is runtime prompt engineering where you build a system to dynamically add relevant things to your prompt before you ask the agent for an answer."
maciejgryka 2y ago
> “Retrieval-Augmented Generation” is nothing more than a fancy way of saying “including helpful information in your LLM prompt.”

29athrowaway 2y ago

RAG is as valuable as the data you can retrieve. If the amount of data is small you don't need the flexibility of RAG. And if it is irrelevant it will stay irrelevant after found.

maciejgryka 2y ago
For sure, it's only worth doing if you actually have so much relevant data that it doesn't fit in the context! This is definitely the case for us for this problem, but it's not universal.
- 29athrowaway 2y ago
  To me the RAG hype is just the sudden rediscovery of information retrieval by money hounds that did not care about AI/ML for decades and now are in panic mode due to FOMO.
  - maciejgryka 2y ago
    There's hype and FOMO for sure and you're right that there's lots to learn from information retrieval work. But why be dismissive of the whole thing? People learning from past research and applying it in new contexts seems like a good thing and there's people (like https://x.com/bclavie https://x.com/bclavie and https://x.com/jobergum https://x.com/jobergum), who are legit experts in information retrieval and are showing how to apply it to RAG properly.
    - 29athrowaway 2y ago
      There are knowledgeable people out there with excellent use-cases for RAG, to find data that is relevant. And they come from years of using the previous state of the art techniques in information retrieval. They have all my support and I am looking forward to read more about it. But the majority of talk about RAG is coming from hype driven individuals that produce low merit and uninformed opinions.

deleted 2y ago

[deleted]

maciejgryka 2y ago

This is one of the things we learned recently about building production workflows with LLMs. Happy to answer any questions/feedback here <3

ukd1 2y ago
For me it's nice to see the re-use of existing infra - big query. Personally, I was rooting for Postgres, but your logic of why not makes sense! Great post.

rodrigovicuna 2y ago

Here to learn more about this. Important for a startup I know.

djhn 2y ago

Reading this was very valuable. I really appreciate the Vespa mention and introduction to their Multi-Vector HNSW Indexing - I’ve recently thought a lot about how difficult chunking is and this seems like a promising avenue.