viktor_von

7 ms·

1. 1.▲
   by viktor_von 6mo ago

   > yet somehow people married the acronym to one very particular implementation of the idea. Likely due to the rise in popularity of semantic search via LLM embeddings, which for some reason became the main selling point for RAG. Meanwhil
2. 2.▲
   by viktor_von 2y ago

   Ad-free and fast, Wikipedia is imo the best learning resource on-the-go, at home, or at work.
3. 3.▲
   by viktor_von 2y ago

   > The information cost of making the RNN state way bigger is high when done naively, but maybe someone can figure out a clever way to avoid storing full hidden states in memory during training or big improvements in hardware could make m
4. 4.▲
   by viktor_von 2y ago

   > It's astounding to me (and everyone else who's being honest) that LLMs can accomplish what they do when it's only linear "factors" (i.e. weights) that are all that's required to be adjusted during training
5. 5.▲
   by viktor_von 2y ago

   > I remember one of the initial transformer people saying in an interview that they didn't think this was the "one true architecture" but a lot of the performance came from people rallying around it and pushing in the one