17 ms·
Perhaps an analogy to Moore's law? Bitter lesson #1: don't waste time optimizing code when a faster processor is around the corner. What countered it: Moore's
by klipt 20d ago
Perhaps an analogy to Moore's law?
Bitter lesson #1: don't waste time optimizing code when a faster processor is around the corner.
What countered it: Moore's law stopped working.
Bitter lesson #2 similarly relies on scaling laws that might have diminishing returns wrt model runtime vs intelligence. Runtime matters for turnaround on the problem you're solving.
- wild_egg 20d agoMoore's Law has nothing to do with processors getting faster. Dennard scaling stopped working but Moore just slowed somewhat, not stopped.
- bsder 20d agoYou are technically correct. The best kind of correct. However, what most people think of as Moore's Law--CPU speed doubles every 18 months--broke somewhere between 90nm and 22nm. And even the actual Moore's Law--2x the transistors every 18 months--doesn't hold for all types of chips anymore. Memory only gained 2x density over 10 years.