jakswa

6 ms·

1.
by jakswa 13d ago
dang only for certain nvidia GPUs, had my hopes up
2.
by jakswa 13d ago
don't see it in my AWS bedrock model list yet, but boy has bedrock mantle been annoying today with the errors/downtimes, with NO status page entries >_<
3.
by jakswa 19d ago
Oh. The UI screenshot on github is... not actually in the github repo? It's platform/hosted only? There's my first awkward discovery, but makes sense in retrospect.
4.
by jakswa 19d ago
I think I have to look at setting up one of these AI gateways for work, since AWS bedrock is such a PITA to hook a harness up to over IAM roles. Also I still can't believe bedrock hasn't released any open models in months (so ther
5.
by jakswa 27d ago
ended up disabling ornith 9B. Oddly Ling 3 Tiny is pretty dang capable if its thinking is unleashed (tons of output tokens, maybe 5X the tokens but it's so fast it's maybe only twice as slow as a smarter model). This is a really i
6.
by jakswa 28d ago
I had to go down to UD-Q3_K_XL for Qwen 3.8 27B to get it to fit in VRAM and be usable, but I worry I'm gutting its intelligence somewhat. I too am interested in faster + more-usable alternative that can exchange blows with the Q3-dumb
7.
by jakswa 28d ago
I'll be comparing the 9B vs Ling 3 Tiny (8B-A1B) as a scout model. Ling tiny is so fast but can be a little too dumb. Hope the 9B strikes a good middleground even if dense/slower.
8.
by jakswa 1mo ago
I've been waiting on this model to show up on the Deep SWE benchmark results and treat its absence/delay as an indication of how slow and unusable it is for good results. I bet it thinks to the moon on some of those complex challe
9.
by jakswa 1mo ago
Thanks for mentioning Ling 3 Tiny. This model has completely bypassed me and seems promising for how small it is.
10.
by jakswa 1mo ago
I'm waiting for this comparison too. I was impressed by a 1-shot GLM 5.3 did for me the other day.
11.
by jakswa 1mo ago
Gemma 4 (both E4B + 12B) performed really well as ears+brains. I mostly comment because I too am always scouting for a nice local all-in-one model.
12.
by jakswa 1mo ago
I went back to Glimmer 30b for my 20GB of VRAM. Just a better experience fit-wise and speed-wise and tone-/voice-wise.
13.
by jakswa 1mo ago
thank you I'm still on ie8
14.
by jakswa 1mo ago
I'm in the exact same boat with a 7900 XT and a good Glimmer 30B experience. I was really hoping qwen 3.8 would bring some memory/space efficiency savings along the lines of whatever is going on with Glimmer 30B. I have been surpr
15.
by jakswa 1mo ago
anecdotes: 35B-A3B does want more memory, bigger model. But if you get it running it will be faster and more enjoyable to use -- text will fly by -- due to only 3B params being active, in my experience at least.
16.
by jakswa 1mo ago
I dunno, I didn't read in-depth. Hopefully you don't gotta zoom in with human eyeballs.
17.
by jakswa 1mo ago
OMP changed the default compaction to images ! Kinda nuts to read about. Saves the generation cost of the traditional compaction step and writes the context as tiny text to an image, if I was following correctly.
18.
by jakswa 1mo ago
my whole world is shifting. have I been seeing _different pelicans_ from everyone else?!
19.
by jakswa 1mo ago
This pelican gave me a good laugh, because there's enough reasoning that the render is out of sight initially. The buildup!
20.
by jakswa 1mo ago
I'll back up your smaller claim, but be specific that it's UD-Q4_K_XL size: - muse glimmer: 15.9GB - qwen 3.6 27B: 17.6GB My video card is so close to its limit that these GB thresholds are mattering too much for me :D
21.
by jakswa 1mo ago
it's here! https://news.ycombinator.com/item?id=49245575
22.
by jakswa 1mo ago
I'm listening to pelican sounds on youtube while I wait for Simon.
23.
by jakswa 1mo ago
I like the tabletop RPG use case, and wanted to say: If your hardware likes it you should check out Gemma 4 for creative DMing use case. I found it to be much better at holding the plotlines and being creative on gaming turns. My experiment
24.
by jakswa 1mo ago
some support already merged, and I verified in a local build that it runs (cannot get MTP params working tho, about ~40 tok/s on my beefy 800GB/s 7900XT w/ 20GB VRAM). https://github.com/ggml-org/llama.cp
25.
by jakswa 1mo ago
Q3 results: unsloth/Muse-Glimmer-30B-GGUF:UD-Q3_K_XL gets down to 15.6GB VRAM and full context (131k) on the 4 parallel slots. Prompt/generation speeds about the same. Overall feeling like a nicer-fitting Qwen 3.6 27B, but want to
26.
by jakswa 1mo ago
Another candidate for the 7900XT (20GB VRAM) I got sitting around. I pulled latest llama.cpp (targeting vulkan during build) after seeing a muse PR merged a few hours ago, and unsloth/Muse-Glimmer-30B-GGUF:UD-Q4_K_XL runs on my 7900XT
27.
by jakswa 1mo ago
> Every source file is summarized once into a short description of what it does. I admit I haven't had a chance to read the whole README, but wanted to get down my hesitation after I got pretty far (as an interested user): I think i
28.
by jakswa 1mo ago
I remember trying out a zombie running app ~15yrs ago and am a fan of the concept. That was a podcast pretty much, audio-only storyline in your ears describing proximity/urgency/etc. I didn't grant this thing GPS access (work
29.
by jakswa 1mo ago
I can't begin to picture how much AI slop they are having to filter out. I wonder if they are going to entertain some UX to save people disappointment. "The project should be non-trivial. Don't post quickly-generated one-offs
30.
by jakswa 1mo ago
Hell yeah I got people to move in. Nice job getting it to walk me through those intro steps.

More ›