Not an expert, but maybe using RAG/embeddings on the on-disk wikipedia would be better than finetuning on wikipedia? Most decent LLMs probably were already tra

Not an expert, but maybe using RAG/embeddings on the on-disk wikipedia would be better than finetuning on wikipedia? Most decent LLMs probably were already trained on wikipedia, that doesn't stop them from hallucinating when asked questions about it.