6 ms·
> A modern sparse Transformer, for instance, is not "conscious," but it is an excellent engineering approximation of two core brain functions: the Global Worksp
by quantumgarbage 1y ago
> A modern sparse Transformer, for instance, is not "conscious," but it is an excellent engineering approximation of two core brain functions: the Global Workspace (via self-attention) and Dynamic Sparsity (via MoE).
Could you suggest some literature supporting this claim? Went through your blog post but couldn't find any.
- NetRunnerSu 1y agoSorry, I didn't have time to find the relevant references at the time, so I'm attaching some now https://www.frontiersin.org/journals/computational-neuroscience/articles/10.3389/fncom.2024.1352685/full https://www.frontiersin.org/journals/computational-neuroscie... https://arxiv.org/abs/2305.15775 https://arxiv.org/abs/2305.15775