6 ms·

Isn't there retrieval degradation with such a large context size? I would still think that a RAG system on 128K is still better than No Rag + 1M context window,

by tk90 2y ago

Isn't there retrieval degradation with such a large context size? I would still think that a RAG system on 128K is still better than No Rag + 1M context window, no? (assuming text only)

afro88 2y ago
Not sure why you've been downvoted. Needle in a haystack testing exists for a reason
cathalc 2y ago
Absolutely. Gemini results tend to drop off after 128k tokens according to RULER: https://github.com/hsiehjackson/RULER https://github.com/hsiehjackson/RULER
- ayyndrew 2y ago
  If I'm reading the paper right, it says Gemini 1.5 is good up to 128k, and RULER doesn't test beyond 128k
  - cathalc 2y ago
    Hmm perhaps you are right.. There's no context on the page about what ">128k" means so possibly a bad assumption on my part!