6 ms·
I wonder if combinators could be useful for neurosymbolic AI—either in the backward pass (e.g., training models on synthetic data) or the forward pass (e.g., it
by Yusefmosiah 2y ago
I wonder if combinators could be useful for neurosymbolic AI—either in the backward pass (e.g., training models on synthetic data) or the forward pass (e.g., iterative code generation with evolutionary algorithms). Combinators feel alien, making even Haskell or APL seem intuitive, but maybe that’s because they don’t align with human working memory. Language models, with their massive context windows, handle long-range dependencies in sequences well, even if their understanding is shallower in some ways. Could combinators, with their compositional and deductive nature, be a better fit for machines than humans? For example, instead of generating Python functions in an evolutionary approach[0], could we use combinators as the building blocks? They’re compact, formal, and inherently step-by-step, which might make them ideal for tasks requiring structured reasoning and generalization. What do you think?
[0]: https://jeremyberman.substack.com/p/how-i-got-a-record-536-on-arc-agi https://jeremyberman.substack.com/p/how-i-got-a-record-536-o...
- Vecr 2y agoThere's probably not enough training data. People use Python with LLMs because it's what works, not because it's the best language. Anything rarer starts having issues.
- deleted 2y ago[deleted]
- ocean_moist 2y agoYeah training data is the major issue. You can try programmatically brute-forcing/generating random well-formed (compilable) data and training on that. Then heuristically pick exogenous programs from the output of the trained model to train a new "generation". The issue is in the possibility of the algorithm generating "exogenous" program and "heuristically" picking them out.