6 ms·
The first AI winter was largely triggered by Minsky in a book he published in 1969, which mathematically proved that single-layer perceptrons couldn't solve non
by rahen 3mo ago
The first AI winter was largely triggered by Minsky in a book he published in 1969, which mathematically proved that single-layer perceptrons couldn't solve non-linear problems. Favorite quote: "Our intuitive judgment is that the extension [to multilayer systems] is sterile."
Yet we had the computational power to run backpropagation in the 1960s and small Transformers in the 1970s (I'm the author of both):
https://github.com/dbrll/Xortran https://github.com/dbrll/Xortran (backprop on IBM 1130, 60s)
https://github.com/dbrll/ATTN-11 https://github.com/dbrll/ATTN-11 (Transformer on PDP-11, 70s)
What was missing wasn't the raw processing power, but the ideas and algorithms themselves. Because funding and research were completely discouraged during the AI winter, neural networks research was left dormant and we lost two decades.
- WarmWash 3mo agoI wonder had we invented transformer architecture back in the 70's or 80's, if the pace of hardware innovation would have naturally slowed AI progression, and given humans decades to slowly adapt, rather than the current tidal wave (that seems to grow in size daily) bearing down on us.
- Schlagbohrer 3mo agoThis is a fascinating discussion, thank you both. I'll add this to my casual mental list of "Technologies that could have been developed generations earlier but through random chance, simply weren't."