cafkafk

12 ms·

1.
by cafkafk 1mo ago
Hey, author here, not sure how popular this article is gonna be with HN, but want it to get out there. This mostly just establishes some words that I'm using when discussing NixOS and the wider ecosystem. Formalizing them hopefully wil
2.
Terminology Beyond the NixOS Trademark(point.free)
3 points by cafkafk 1mo ago|2 comments
3.
by cafkafk 2mo ago
Author here. This came out of reading Denmark's national biological threat assessment (first update since 2020), which splits its AI finding in two: not much help for people without a professional background, meaningful help for people
4.
The AI bioweapon risk isn't jailbreaks(point.free)
3 points by cafkafk 2mo ago|1 comments
5.
by cafkafk 2mo ago
That "I actually don't have any personal criticisms of Jarred" made me do a spit take, because a majority of what I had just read was absolutely a personally targeted criticism of Jarred. Whether that's "okay"
6.
by cafkafk 3mo ago
Hi HN. Follow-up to the Xeon post from a couple of weeks ago. A lot of people came away from that one with a 25-flag command and no real idea which flags materially did anything, and the honest truth is neither did I fully. So I went back a
7.
by cafkafk 4mo ago
Loading will take some minutes, but at 96 you can squeeze the model in and have some headroom around like ~10 GB, although depending on the Xeon, you may have to downgrade to E4B instead. Should still work thou.
8.
by cafkafk 4mo ago
If you get the inference engine to route the heavy matrix math to the GPU and the speculative drafting to the CPU without choking on latency it's probably gonna be very fast. Would love to see the benchmarks if someone actually pulls s
9.
by cafkafk 4mo ago
That is a very fair point! I just ran a not very scientific benchmark with the system under load, and posted the raw logs in a sibling comment above, but the short answer is that it's hitting 11.94 tokens per second for generation - wh
10.
by cafkafk 4mo ago
Honestly, at this point you're probably looking at a smaller model, for the Gemma series I'd go with Gemma 4 E4B with drafters, but that's just a hunch from using it on my laptop (where I do have a RTX 4060 M and 96gb ram). S
11.
by cafkafk 4mo ago
> (purple on black is really hard to read) Noted, and agree (it looks like it has also already been clicked, which I dislike). I honestly I need to redo the themes. > You say it runs "at reading speed". Have you benchmarked
12.
by cafkafk 4mo ago
Hi HN. I wrote this post after getting frustrated by the lack of ways to run the new Gemma 4 Drafter models, and mainstream tools not prioritizing this, and hiding all the performance levers. I ended up getting a modern 26B MoE model (Gemma
13.
A 10 year old Xeon is all you need(point.free)
740 points by cafkafk 4mo ago|290 comments
14.
Farewell AWS(adventuresinoss.com)
2 points by cafkafk 4mo ago|0 comments
15.
by cafkafk 4mo ago
Often the problems for me come when: - It starts thinking for itself when I asked it to do something specific. - It reads its own wrong code comments and ignores my corrections. - Its knowledge cutoff means it thinks of solutions from 2024.
16.
by cafkafk 4mo ago
Not really, beyond using them as a downstream provider through openrouter at some point (IIRC). It worked? ¯\_(ツ)_/¯
17.
by cafkafk 4mo ago
Google Vertex (or Google AI Studio) might be a potential alternative, but the UX is a lot worse, and it can be hard to estimate cost since it's typically lagging. One of the advantages you'll miss when moving from openrouter is go
18.
A portentous reunion(bcantrill.dtrace.org)
147 points by cafkafk 4mo ago|45 comments
19.
by cafkafk 4mo ago
I think a lot of the problem with the current discourse is how black-and-white it is. Either you're a luddite or "ai pilled". In most cases, LLMs can get you 80-95% of the way, sometimes less, sometimes more. And heck, someti
20.
by cafkafk 4mo ago
I didn't realize the single spinning coin was actually a loading animation. Might make sense to add text indicating it is loading, or a loading bar. Without this, and considering the long load time I had, I imagine the bounce rate is g
21.
by cafkafk 4mo ago
Recommending Hetzner as an alternative here is a mistake. It just exposes you to a different problem. There is a reason for the term "Hetznered" existing. Hetzner can suddenly and permanently terminates your account. They do this
22.
by cafkafk 6mo ago
I get that everyone wants to be cynical about this, but you really can't deny that both the visualization and sheer scale of data is impressive. The way the "my life in weeks" is done is also very cool, I'll be stealing
23.
by cafkafk 7mo ago
I don't really believe in the specific numbers he gives, but I appreciate moving the conversation away from “should” and into the consequences — including those that arise from delays.
24.
by cafkafk 1y ago
VitVio | Senior Full-Stack Engineer (Elixir, NixOS, React) | Remote (EU Timezones) | Full-Time | vitvio.com We're building AI-powered systems to make hospital operating rooms safer and more efficient. We're looking for a seasoned
25.
by cafkafk 2y ago
fwiw we're glad we're scaring people like that away :p
26.
by cafkafk 2y ago
https://github.com/orgs/eza-community/discussions/679#discus...
27.
by cafkafk 2y ago
I was a former exa user, and the z is next to x on my keyboard, wasn't a huge hassle for me
28.
by cafkafk 2y ago
contributions welcome
29.
by cafkafk 2y ago
We do not