5 ms·
We're running out of the ability to make transistors smaller and closer together so beyond some major breakthrough I wouldnt expect Moore's law to continue near
by aCoreyJ 2y ago
We're running out of the ability to make transistors smaller and closer together so beyond some major breakthrough I wouldnt expect Moore's law to continue nearly long enough to get to the point of running GPT4 on consumer hardware in the short term
- timschmidt 2y agoAh, but we've just begun stacking transistors in the third dimension.
- jazzyjackson 2y agoIt was never a solution, Moore's law has more than one dimension as well, not just density but heat dissipation. Can't cool down a transistor that's surrounded by transistors on all sides.
- ctrw 2y agoThat doesn't solve the problem, it just pushes is down the road a bit. The exponential growth is merely offset by a constant factor once. Unless we figure out how to push transistors in the 5th, 6th etc dimension with every new generation.
- moffkalast 2y agoWell consumer hardware can run something in the order of ~50B quantized at a "reasonable" price today, we'd need about 5 or 6 doublings to run something that would be GPT 4 tier at 1T+. So, it would need to continue for roughly a decade at least? Current models are horrendously inefficient though, so with architectural improvements we'll have something of that capability far sooner on weaker hardware.