halflings

6 ms·

1.
Sampling from LLMs: Art and Science(kachkach.com)
1 points by halflings 5mo ago|0 comments
2.
by halflings 6mo ago
> Youtube charges $10 per month and doesn't produce a single video It is different from Netflix (that pays upfront for production costs), but there's of course a revenue share + the bulk of the revenue for creators is actually
3.
Generating one token at a time is a blessing in disguise(kachkach.com)
3 points by halflings 6mo ago|1 comments
4.
by halflings 6mo ago
LLMs generate their output one token at a time. The first thought when you learn this is that this is a huge performance bottleneck, as we are used to highly parallelized systems. However, a large part of what makes LLMs feel so magical com
5.
by halflings 8mo ago
The main thing I noticed in the video is that they have heavily sped up all the code generation sections... seems to be on 5x speed or more. (because people got used to how fast and good Sonnet, and especially Gemini 3.0 Flash, are)
6.
by halflings 8mo ago
Deploying from Antigravity is as easy as say connecting the Firebase MCP [1] and asking it "deploy my app to firebase". [1] https://firebase.google.com/docs/ai-assistance/mcp-server
7.
by halflings 9mo ago
+1, reading through the post, the PR updating the documentation... thanks for being transparent, but also don't be so hard on yourself! That was a very niche error, that you promptly corrected, no need to be so apologetic about it! And
8.
by halflings 9mo ago
"The models perform differently when called via the API vs in the Gemini UI." This shouldn't be surprised, e.g. the model != the product. The same way GPT4o behaves differently than the ChatGPT product when using GPT4o.
9.
by halflings 1y ago
I would also add that search has already moved elsewhere. Less and less people are using search engines to shop, ex:Amazon makes >$57B a year from search ads, but also look at Temu and Shein which are mostly glorified product search pl
10.
by halflings 1y ago
That's what the chart says yes. 14.1GB VRAM usage for the 27B model.
11.
by halflings 2y ago
> I cannot be the first person to think about such possibilities Differentiable Rendering [1] is the closest thing to what you are describing. And yes, people have been working on this for the same reason you outline, it is more data&#x2
12.
by halflings 2y ago
Training code is only useful to people in academia, and the closest thing to "code you can modify" are open weights. People are framing this as if it was an open-source hierarchy, with "actual" open-source requiring all
13.
by halflings 2y ago
> The world is teetering on the edge of world war The world probably has never been as peaceful as in the last 50 years or so. Same goes for access to drinkable water, food, decent shelter, gender equality, freedoms, technology, etc. But
14.
by halflings 2y ago
Thanks for putting this together! Will give it a watch now
15.
by halflings 2y ago
The # of chips is not the most important metric. Most important, even ignoring latency, is throughput (tokens) per $$$. And according to their own benchmark [1] (famous last words :)), they're quite cost efficient. [1] https:/&#x
16.
by halflings 2y ago
No HBM because they use tons of fast SRAM instead. Isn't that the main driver for performance here? (the way I understood it => it's still cost effective at scale due to throughput increase this brings)
17.
by halflings 2y ago
Agree re:hallucinations/safety issues, that was likely one of the main blockers. And here's the sad part: they had this back in 2019... see this paper released in Jan 2020: https://blog.research.google/2020/01
18.
by halflings 2y ago
This (innovator's dilemma / too afraid of disrupting your own ads business model) is the most common explanation folks are giving for this, but seems to be some sort of post-rationalization of why such a large company full of comp
19.
by halflings 2y ago
uv has been really awesome as a replacement for pip: https://github.com/astral-sh/uv So fast it finally made virtual environments usable for me. But it's not (yet) a full replacement for conda, e.g. it won't
20.
by halflings 3y ago
This looks pretty cool! What's the catch? e.g. why isn't this already implemented in accelerators, is it really just a forgotten algorithm, or this has some implications on the cost of building the accelerator or else?
21.
by halflings 3y ago
> Your analogy is the same as early Intel engineers completely unaware that those chips would bring on the ramifications of social media Exactly! As they should be. (for both Intel engineers developing chips, and physicists developing nu
22.
by halflings 3y ago
This is the definition of strawman. "Advocate for killing all humans" sounds like someone advocating for a genocide, but instead it's just the same transhumanist thinking (which Yudkowsky also believes in, FYI)
23.
by halflings 3y ago
Your comment (+ username) reads like what I would have written once upon a time when I was fully in the EA bubble. Truly no offense meant, as I was deeply into the EA movement myself, and still consider myself one (in the original "don
24.
by halflings 3y ago
The importance (e.g. attention) needs to be dynamic, e.g. one token will be important to some other tokens but not others. tf-idf and similar heuristics are what we were using before attention came along, e.g. tf-idf weighted bag-of-words r
25.
by halflings 3y ago
Yep that's pretty much it! That's what they call needle in a haystack. See: https://github.com/gkamradt/LLMTest_NeedleInAHaystack
26.
by halflings 3y ago
Not if you account for the increase in ad revenue from the company paying them.
27.
by halflings 3y ago
> The technology looks at hundreds of signals that could indicate a booking is higher risk for this type of incident, like the duration of the trip the guest is trying to book, how far the listing is from their location, the type of list
28.
by halflings 3y ago
> Nowhere it's touted as a rival to youtube in the popularity sense, just like you wouldn't call WordPress a rival to Twitter. ... the first image on the article linked here shows a monster called "Videorapter" with Y
29.
by halflings 3y ago
> Why not output MIDI instead, and let the artist manipulate that? This is not true at all, it all started with generating MIDI, and even the very link I shared is of a system that generates MIDI.
30.
by halflings 3y ago
The page fully explains what they mean by this, showing results on benchmarks etc.

More ›