tarruda

7 ms·

1.
by tarruda 5d ago
The exception is if you use CoW filesystems. Btrfs has a workaround: it can create a cow-disabled subvolume for the swapfile. AFAIK you need a separate partition for Zfs.
2.
by tarruda 5d ago
Awesome!
3.
by tarruda 7d ago
Darn, for a moment I thought it would be possible to run in 128G T.T
4.
by tarruda 7d ago
200B of those 552B is PLE, which works more like a database that is read for each token, thus can be offloaded to a fast SSD.
5.
by tarruda 7d ago
> It also includes additional 196B Engram memory which you can put on an SSD. I think You can put Qwen 3.8 Flash Next engram on SSD, but prompt processing takes a good hit. On my mac studio, I get 300 pp and 33 tg with SSD offload, versu
6.
by tarruda 7d ago
I tried dwarfstar when llama.cpp DSV4 support was still very weak, and while it worked, I didn't see anything that would make me want to stick with it vs llama.cpp. llama.cpp is simply better with its awesome built-in webui, router and
7.
by tarruda 7d ago
Don't use my Vision-Exp GGUF though. As I said I built those GGUFs before llama.cpp supported, and they can't be loaded on current master (require my own branch). I already have new GGUFs but haven't uploaded yet. If you want
8.
by tarruda 8d ago
I would rather have just 3 levels: low, medium and high.
9.
by tarruda 8d ago
IQ3_XXS (~3.2 BPW). For me this is an option because my Mac studio is only used for serving LLMs, so I can afford to dedicate most of its RAM to this. I can run with 256k context and only uses ~117G, with the remaining (up to 125G which I c
10.
by tarruda 8d ago
Hopefully it will be open weights and have the same architecture and size as the current v4 flash vision, which is probably the best LLM that can be run on 128G devices.
11.
by tarruda 10d ago
I use my Mac as a headless server to run LLMs, and never even start a desktop session so it always stays at pre-login state without any applications running. Even so, I never saw it using less than 2G in htop when logged via SSH.
12.
by tarruda 10d ago
I'm certain the video is sped up. Still possible to script computer use, and I think this is what happens. One of the suggests this: https://x.com/DanielSMatthews/status/2096311244504572177 Could be hallucina
13.
by tarruda 10d ago
Interesting, thanks for sharing! A scripting API makes the problem much more approachable, but what about those videos where Astra is drawing people from a photo? Here's one using canva: https://x.com/iam_zachi/sta
14.
by tarruda 10d ago
Amazing project. My only blocker for installing Asahi on my Mac Studio (M1 ultra) is that llama.cpp performance is very bad when compared to using the Metal backend on the same hardware.
15.
by tarruda 10d ago
I haven't tried any of these demos, but I'm not surprised they stop impressing once you go deep. What got be mostly impressed were the demos of Astra doing computer use. At my job I do some RPA and can appreciate how challenging i
16.
by tarruda 10d ago
X.com is full of unverified posts, but here's a few: - https://x.com/MengTo/status/2096213835460084184?s=20 - https://x.com/birdabo/status/2096101381925114327?s=20 Seems really impr
17.
by tarruda 12d ago
> even need to understand that it's "bypassing" because it thinks it's just iterating towards its goal. Could they have added a "no internet access" goal constraint?
18.
by tarruda 12d ago
Even the behavior of agents searching for sandbox bypasses must have been in the training data, or at the very least, "suggested" in some way. To be this whole thing feels like a marketing play by OpenAI.
19.
by tarruda 22d ago
Also proud of having spent $2.5k on a used 128G M1 Ultra back in September 2024. Despite being outdated in terms of compute, it stills let me run very good recent models locally, with Deepseek V4 Flash 0731 being the greatest one right now,
20.
by tarruda 22d ago
> not that Apple prices weren’t insane before the ram/ssd shortages Funny that after the price started to increase last year (I think October/November), there was a window of a few months where Apple prices stayed the same as t
21.
by tarruda 23d ago
Can you share the source for the parameter count (125B A6B)? I didn't see it anywhere in the page.
22.
by tarruda 23d ago
https://www.youtube.com/watch?v=P4VBqTViEx4
23.
by tarruda 24d ago
> I’m struggling to understand how that was causing such a large difference in the output. It is incremental, the more a pattern appears in the context, the more likely it was to continue appearing in future turns. So the model was like
24.
by tarruda 25d ago
There's quite a few tangential features that must be implemented correctly or risk affecting the LLM output in significant ways. Parsing/encoding is one example: A couple of months ago I've debugged a reasoning loop bug in St
25.
by tarruda 1mo ago
> It can run 80-100t/s on a laptop That is a lot, what is your laptop hardware? One issue I have with Gemma is that they seem to use old architectures that rely on full attention, requiring a lot of RAM for context and quickly degra
26.
by tarruda 1mo ago
> It has all of the components of a pelican riding a bicycle, though not exactly arranged in the right order! Maybe a sign that they didn't have SVG pelicans in the dataset
27.
by tarruda 1mo ago
I don't have experience with opencode, so I couldn't tell you. Codex is really good in my experience, especially due to its native sandboxing. Deepseek seems really well versed in its tools, including update_plan and knowing when
28.
by tarruda 1mo ago
One of the best things about this version is that it is trained in the codex harness. It feels just as good as OpenAI models in using codex tools, but extremely cheap and with 1M context
29.
by tarruda 2mo ago
It is not that they don't release open weights, but some users report that they are significantly inferior to the closed versions.
30.
by tarruda 2mo ago
> You can't. Not if you're in the minority Is that a bad thing?

More ›