7 ms·
what would it mean to plan ahead? decoding strategies like beam search are popular and effectively predict many words ahead
by gbasin 2y ago
what would it mean to plan ahead? decoding strategies like beam search are popular and effectively predict many words ahead
- jumpCastle 2y agoAlso the parameters are optimized also with loss of future tokens in the sequence.
- HarHarVeryFunny 2y agoThink before generating output - plan the entire sentence before you generate the first word(s) and maybe talk yourself into a corner. Tree-of-Thoughts (not Chain) is one way to provide something a bit similar - kind of like DeepBlue or AlphaGo generating possible branching future lines of play and picking the one with best outcomes. To be more brain-like you'd really want the system to generally be "looping" internally - a bit like our thalamo-cortical loop - and only start outputting when the thought had gelled.
- HarHarVeryFunny 2y agoIt's a shame HC doesn't use an LLM to upvote/downvote rather than people. Take the emotion out of technical discussions and rate based on factuality instead. I suppose whoever downvoted this either hasn't heard of tree-of-thoughts, or doesn't understand what it is and what problem it is addressing. Or, maybe they just didn't like that their "gotcha" question had a simple answer.
- astrange 2y agoLLMs have emotions ;) https://vgel.me/posts/representation-engineering/ https://vgel.me/posts/representation-engineering/