5 ms·
git or versioned path on cloud storage should work. Format is more important - I think hierarchical knowledge base is the best thing on the market atm. You keep
by sermakarevich 3mo ago
git or versioned path on cloud storage should work. Format is more important - I think hierarchical knowledge base is the best thing on the market atm. You keep main page with refs and short summaries to 5-10 topics. Model reads it and decides where to drop down next. You optimize the breadth and depth of topics for optimal performance. Dropping down is not a through away tokens - this helps model to understand wider context.
I use it in quite a few repoes:
-- https://github.com/sermakarevich/ai_knowledge_wiki https://github.com/sermakarevich/ai_knowledge_wiki Curated extraction of summaries from AI-related research papers, organized as a hierarchical wiki optimized for Obsidian and LLMs
-- https://github.com/sermakarevich/chunker/tree/master/output/functional_programming_in_scala https://github.com/sermakarevich/chunker/tree/master/output/... Chunker processes a document into a hierarchy of self-sufficient chunks and multi-level summaries, producing a set of linked markdown files that an AI model (or a human) can explore through progressive disclosure -- starting from a high-level overview and drilling into details on demand, without ever loading the entire document.
-- https://github.com/sermakarevich/kaggle_wiki https://github.com/sermakarevich/kaggle_wiki A structured knowledge base of Kaggle competitions — solutions, notebooks, and indexes — built for fast lookup and reuse.