6 ms·
Isn't specialized hardware also a big risk? GPUs are more amenable to any big changes that may happen in the next 5, 10 years of AI research. Maybe we won't eve
by amlib 2mo ago
Isn't specialized hardware also a big risk? GPUs are more amenable to any big changes that may happen in the next 5, 10 years of AI research. Maybe we won't even be talking about LLMs anymore. Maybe matrix multiplication won't even be the main primitive.
- moduspol 2mo agoIf matrix multiplication isn't the main primitive, I think we have a lot of pain coming our way.
- amlib 2mo agoMaybe that is far fetched, but I could see them specializing for some super high dimension multiplication and meanwhile 5 years later turns out "all you need" are 3x3 matrices and suddenly 90% of your specialized hardware is now dark silicon :)