7 ms·
Probably not. It's not clear how to scale other methods to make use of so much compute. They top out sooner, with fewer parameters and less compute. One way to
by gradys 5y ago
Probably not. It's not clear how to scale other methods to make use of so much compute. They top out sooner, with fewer parameters and less compute.
One way to look at why deep learning is having the impact it does is that unlike other ML methods, it's actually capable of making use of so much compute. It gives us modular ways to add more and more parameters and still fit them effectively.
- WanderPanda 5y agoWhat always astonishes me that deep learning seems to work on human timescales. For other problems (even if they are polynomial in complexity) we get into infeasible runtimes when we increase the problems complexity by 10x. With deep learning the fuzzy, approximative nature seems to help to grasp the gist of the 10x problem and somehow allows us to reach 95% of the solution in just e.g. 2x the runtime. Heuristics might play in the same league, but the development time kind of scales with the problem, while in deep learning I would put it in the linear or log basket.