trsohmers

21 ms·

1.
by trsohmers 5mo ago
Software people, in my very direct experience, are terrible at hardware... While in jest, I do think most software engineer's understanding of hardware abstractions is pretty poor and does disservice to the hardware they run on. I know
2.
by trsohmers 6mo ago
It actually stands for "lizard brain"... it is (or at least was) an Infineon Aurix control and monitoring microcontroller, they may have changed to a newer one.
3.
by trsohmers 8mo ago
Feel free to ask me any questions! Website: https://positron.ai
4.
Positron's $230M Funding Led by Financial Trading Firms(eetimes.com)
1 points by trsohmers 8mo ago|1 comments
5.
The New Chips Designed to Solve AI's Energy Problem(wsj.com)
2 points by trsohmers 1y ago|0 comments
6.
by trsohmers 1y ago
Only with the oscillation overthruster flag enabled.
7.
by trsohmers 1y ago
I was put on it in 2015 after an acquaintance of mine that was previously on the list recommended me… I only heard from Forbes a few days before the list came out, they asked me for a photo and asked if I approved the 2 sentence blurb they
8.
by trsohmers 2y ago
Based on their S1 filing and public statements, the average cost per WSE system for their (~90% of their total revenue) largest customer is ~$1.36M, and I’ve heard “retail” pricing of $2.5M per system. They are also 15U and due to power and
9.
by trsohmers 2y ago
Do you think that the 16k GPUs get used once and then are thrown away? Llama 405B was trained over 56 days on the 16k GPUs; if I round that up to 60 days and assume the current mainstream hourly rate of $2/H100/hour from the Neocl
10.
by trsohmers 2y ago
+1 this commenter. I just visited the UK for the first time at the beginning of this month and had a fantastic ~3 hours at Bletchley Park, but felt I had to cram TNMOC and the amazing Colossus live demonstration (where I asked a million que
11.
An "Observatory" for a Shy Super AI?(substack.com)
3 points by trsohmers 2y ago|0 comments
12.
by trsohmers 2y ago
They meant that there is no support for Codestral Mamba for llama.cpp yet.
13.
by trsohmers 2y ago
We had a basic LLVM backend that supported a slightly modified clang frontend and a basic ABI. We tried to make it drastically easier for both the programmer and compiler to handle memory by having all memory (code+data) be part of a global
14.
by trsohmers 2y ago
Founder of REX Computing here; I highly recommend checking out my interview on the Microarch Club podcast linked elsewhere on the thread; will also answer questions on this thread if anyone has them.
15.
by trsohmers 2y ago
This is a lesson that like all good Hitchhikers, you should always carry a towel.
16.
by trsohmers 3y ago
Significantly more than that; MFN pricing for NVIDIA DGX H100 (which has been getting priority supply allocation, so many have been suckered into buying them in order to get fast delivery) is ~$309k, while a basically equivalent HGX H100 sy
17.
by trsohmers 3y ago
The quote from the linked press release is that they do training on TPUv4, while inference is running on GPUs. I have also heard this separately from people associated with Midjourney recently, and that they solely do training on TPUs.
18.
by trsohmers 3y ago
I’m right on the millenial/gen Z divide and an inner selfish purpose for me working on AI/ML is just to enable a creation of Jodorowsky’s 10 hour version of Dune with soundtrack by Pink Floyd.
19.
by trsohmers 3y ago
Long story, but technically REX is still around but has not been able to continue to develop due to lack of funding and my cofounder and I needing to pay bills. We produced initial test silicon, but due to us having very little money after
20.
by trsohmers 3y ago
I thought that was clear through my profile, but yes, Positron AI is focused on providing the best performance per dollar while providing the best quality of service and capabilities rather than just focusing on a single metric of speed. A
21.
by trsohmers 3y ago
Groq states in this article [0] that they used 576 chips to achieve these results, and continuing with your analysis, you also need to factor in that for each additional user you want to have requires a separate KV cache, which can add mult
22.
Lambda Raises $320M to Build a GPU Cloud for AI(lambdalabs.com)
3 points by trsohmers 3y ago|0 comments
23.
by trsohmers 3y ago
Yes; Mamba was a very easy match, with Hyena also being a good match, but could be greatly optimized with some minimal changes to the model architecture or hardware design.
24.
by trsohmers 3y ago
"The current round" of AI accelerators you are referring to are things that were designed 2015-2022; There are a number of startups (including my own) that are actually designing for the real bottlenecks that differentiate Transfo
25.
by trsohmers 3y ago
This article from less than a month ago says that it is on 576 chips https://www.nextplatform.com/2023/11/27/groq-says-it-can-dep...
26.
by trsohmers 3y ago
This article from less than a month ago says that it is on 576 chips https://www.nextplatform.com/2023/11/27/groq-says-it-can-dep...
27.
by trsohmers 3y ago
I think the more accurate thing was first hand experience with authoritarianism and totalitarianism. Stalin and Hitler were two sides of the same coin.
28.
by trsohmers 3y ago
They have announced the new Agilex 3 line, which should include some CPLD price point parts and be a real rebirth for ~$100/unit modern devices.
29.
by trsohmers 3y ago
The research is slightly misleading... the models they experimented all had an original pretrained context length significantly less than the fine tuned context length they tested for, e.g. they used MPT-30B-Instruct, which was pretrained f
30.
by trsohmers 3y ago
They have been doing less than ~5k a quarter for the past 2 years. Q3 last year was less than 1,500 sold.

More ›