foundval

7 ms·

1.
Merge, Don't Queue(inside.basepowercompany.com)
9 points by foundval 1mo ago|0 comments
2.
by foundval 2y ago
There is a lot out here. I gave a seminar about the overall approach recently, abstract: https://shorturl.at/E7TcA , recording: https://shorturl.at/zBcoL . This two-part AMA has a lot more detail if you'
3.
by foundval 2y ago
You can chat with these new models at ultra-low latency at groq.com. 8B and 70B API access is available at console.groq.com. 405B API access for select customers only – GA and 3rd party speed benchmarks soon. If you want to learn more, ther
4.
by foundval 2y ago
If you're interested in this sort of stuff, you might like this diff-based CLI tool I wrote: https://github.com/freuk/iter It runs on Groq (the company I work for), so it's super snappy.
5.
by foundval 3y ago
You've got good ideas. What I like to personally say is that Groq makes the "Copilot" metaphor real. A copilot is supposed to be fast enough to keep up with reality and react live :)
6.
by foundval 3y ago
(Groq Employee) As I'm sure you're aware, XTX takes its name from a particular linear algebra operation that happens to be used a lot in Finance. Groq happens to be excellent at doing huge linear algebra operations extremely fast.
7.
by foundval 3y ago
Sweet, thanks! It seems like this research ecosystem was incredibly rich, but Moore's law was in full swing, and statically known workloads weren't useful at the compute scale of back then. So these specialized approach never stoo
8.
by foundval 3y ago
(Groq Employee) Thanks for the feedback :) We're always improving that demo.
9.
by foundval 3y ago
(Groq Employee) Yes! Determinism + Simplicity are superpowers for ALU and interconnect utilization rates. This system is powered by 14nm chips, and even the interconnects aren't best in class. We're just that much better at squeez
10.
by foundval 3y ago
(Groq Employee) Agreed, one should care, and especially since this particular service is very differentiated by its speed and has no competitors. That being said, until there's another option at anywhere that speed.. That point is moot
11.
by foundval 3y ago
(Groq Employee) It's hard to discuss Tok/sec/$ outside of the context of a hardware sales engagement. This is because the relationship between Tok/s/u, Tok/s/system, Batching, and Pipelining is a complex o
12.
by foundval 3y ago
(Groq Employee) You're right - we are comparing to independently-clocked logic. I wonder whether async logic would be feasible for reconfigurable "Spatial Processor" type architectures [1]. As far as LPU architectures go, the