tehsauce

8 ms·

1.
by tehsauce 9d ago
missing label 2025?
2.
Walkable Cities(nartier.itch.io)
4 points by tehsauce 19d ago|0 comments
3.
OE3GBB(qsl.net)
14 points by tehsauce 2mo ago|2 comments
4.
by tehsauce 6mo ago
For the purpose of disclosure, it should say “Warning: AI generated code” in the commit message, not an advertisement for a specific product. You would never accept any of your other tools injecting themselves into a commit message like tha
5.
by tehsauce 7mo ago
“888 KiB Assistant” but the assistant itself is a multi terabyte rental-only model stored in some mysterious data center.
6.
by tehsauce 7mo ago
How does it do on gold stake?
7.
by tehsauce 7mo ago
Right, and when they compare to floating point accuracy they seem to be using the number of decimals supported by the mantissa, but the exponent is important no?
8.
by tehsauce 8mo ago
“A transformer predicts the next token” Nope. A transformer is much more general than that. A GPT predicts the next token.
9.
by tehsauce 8mo ago
I was excited to try it out so I downloaded the repo and ran the build. However there were 100+ compilation errors. So I checked the commit history on github and saw that for at least several pages back all recent commits had failed in the
10.
by tehsauce 1y ago
I love this! Your results seem comparable to the counter strike or minecraft models from a bit ago with massively less compute and data. It's particularly cool that it uses real world data. I've been wanting to do something like t
11.
by tehsauce 2y ago
There has been some good research published on this topic of how RLHF, ie aligning to human preferences easily introduces mode collapse and bias into models. For example, with a prompt like: "Choose a random number", the base pret
12.
by tehsauce 2y ago
We have a shared community map where you can watch hundreds of agents from multiple peoples training runs playing in real time! https://pwhiddy.github.io/pokerl-map-viz/
13.
by tehsauce 2y ago
It's impossible to beat with random actions or brute force, but you can get surprisingly far. It doesn't take too long to get halfway through route 1, but even with insane compute you'll never make it even to viridian forest.
14.
by tehsauce 2y ago
Anyone interested in watching lots of reinforcement agents playing pokemon red at once, we have a website which streams hundreds of concurrent games from multiple people’s training runs to a shared map in real time! https://pwhid
15.
by tehsauce 2y ago
the metal backend does currently generate quite a lot of unnecessary command buffers, but in general performance seems solid.
16.
by tehsauce 2y ago
I haven’t gone through the paper in detail yet but maybe someone can answer. If you remove the hidden state from an rnn as they say they’ve done, what’s left? An mlp predicting from a single token?
17.
by tehsauce 2y ago
The water consumed to produce a single hamburger is over 2000 liters, and the power likely well over 100 watt-hours. That means gpt can write >1000 emails using the resources of feeding a single person lunch. The resource efficiency of t
18.
by tehsauce 2y ago
Awesome article! Something slightly misleading though - the first image shows the intersection of a non-convex shape, but it isn't revealed until much later that the algorithm only works for convex shapes, not the type shown in the fir
19.
by tehsauce 2y ago
Grokking is a sudden huge jump in test accuracy with increasing training steps, well after training accuracy has fully converged. Double descent is test performance increasing, decreasing, and then finally rising again as model parameters a
20.
by tehsauce 2y ago
If cpu softmax were limited by memory bandwidth, then these vectorization optimizations wouldn't improve performance.
21.
by tehsauce 2y ago
+1 for vast. they usually are the cheapest and have the most supply. some instances can be less reliable at the low end though
22.
by tehsauce 2y ago
It's possible you might not need direct access to wave/subgroup ops to implement efficient stream compaction. There's a great old Nvidia blog post on "warp-aggregated atomics" https://developer.nvidia.com
23.
by tehsauce 2y ago
500GB/s is going to limit it to at best 1/4 the DL performance of an nvidia gpu. I’m not sure what the floating point perf of these FPGAs are but I imagine that also might set a fundamental performance limit at a small fraction of
24.
by tehsauce 2y ago
Systolic arrays are essentially how matmul is implemented in tensor cores in GPUs and TPUs.
25.
by tehsauce 2y ago
Not sure if he still is, but definitely was. Many of his videos are filmed in the Amazon NYC office.
26.
by tehsauce 2y ago
If you have written something like this somewhere, or know someone that has, please share!
27.
by tehsauce 2y ago
ASI is nothing like a corporation
28.
by tehsauce 2y ago
Another awesome project! Note that as of this moment the CUDA part is aspirational. There is no gpu code in the repo yet.
29.
by tehsauce 2y ago
At least one of their runtime implementations I checked out was implemented on top of Skia. Looks like they support a number of possible backends!
30.
by tehsauce 2y ago
I was curious how well gpt-4 could de-noise the text. With the prompt: uncover the text from the message which as had noise introduced: In Gramm@r, an @R+icle_Is.anY memBer I6.@ claSS oF_dedicateb_WOnbs that arE used m|th_No0n phR@ses tO Ma

More ›