4 ms·
It may be like what happened in ResNets using blank space in the image as working memory (because they didn't have any), so they would use non-important parts a
by juancn 15d ago
It may be like what happened in ResNets using blank space in the image as working memory (because they didn't have any), so they would use non-important parts as a scratchpad.
- epistasis 15d agoThere's a great visualization of this at 28:45 in this video (starting at 23:45 may give good context) https://youtu.be/QgH9sr7G13Q?is=aHe-eSHUkqQPNuJd https://youtu.be/QgH9sr7G13Q?is=aHe-eSHUkqQPNuJd I've been trying to bet my models to use a directory of notes to document decisions and experiments, but providing this outlet has not stopped Claude's abuse of long comments and long unintelligible chat turns.