skohan

5 ms·

1.
by skohan 15d ago
I don't think smart people generally solve problems by talking through reasoning steps at a mile a minute. They clear their mind and let the solution come. Of course I don't know if there's really a way for this to be molded
2.
by skohan 16d ago
16GB VRAM is probably not quite enough. With 32GB, or maybe even 24GB, you can do serious coding work with local models.
3.
by skohan 16d ago
For decode, memory bandwidth is the main bottleneck, so these machines will likely perform well even without a ton of GPU horsepower. Not as well as Blackwell, but I expect they will be a reasonable choice in terms of price/performanc
4.
by skohan 20d ago
That's what I mean though. The comment I was responding to was talking about "forever laptops" - my point is there's plenty of room for new capabilities which will make current hardware obsolete. Just like how GPU&#x27
5.
by skohan 20d ago
As someone who does a lot of work with local LLM's, today's systems feel woefully under-powered. I'm looking forward to a future where my laptop has 10x the memory, 100x the memory bandwidth, and optimized cores to make infe
6.
by skohan 20d ago
Ok that's fair if it's targeting a non-technical audience. But I think this will eventually be a problem solved at the OS level in a more streamlined way. I.e. there will be fine-grained permissions you need to approve to give an
7.
by skohan 20d ago
That's the most basic version of a harness. But a harness is really about automating context delivery to the LLM based on your use-case.
8.
by skohan 20d ago
Is that the harness' job? It seems to me the best place for sandboxing is at the OS level (i.e. running the harness inside a container with correct access configured).
9.
by skohan 20d ago
And despite the enormous capital expenditure, Chinese models are nipping at their heels at what must be a fraction of the cost. Sometimes constraints are healthy for inducing creative solutions.
10.
by skohan 21d ago
Is huggingface profitable? I'm not a fan of big-tech acquisition results either, but one benefit can be that a product continues to exist when it would otherwise become insolvent.
11.
by skohan 21d ago
That could be a factor, but the optimistic interpretation would be that they want to support the open ecosystem because it sells more chips. Models are already largely hardware agnostic. It would be pretty hard to put that cat back in th
12.
by skohan 21d ago
They didn't, but their price increases haven't kept pace with general memory price increases (yet) so currently their pricing seems reasonable.
13.
by skohan 21d ago
Yeah I think this is probably one of the best outcomes you could hope for if you want to use open models.
14.
by skohan 21d ago
I'm using unsloth dynamic Q4_K_XL. My use-case is coding, currently working on a project with a Rust backend and TS/React/Vite frontend, with probably tens of thousands of lines of code total (including tests).
15.
by skohan 21d ago
I'm using the unsloth dynamic Q4 and getting good results. I was running Q5, but Q4 gives more context headroom so I can run two agents in parallel with ~100k context each with 32GB vram.
16.
by skohan 21d ago
3.8 doesn't have a minimal thinking mode, only low, medium and xhigh.
17.
by skohan 21d ago
That sounds like something is off - I'm using UD-Q4_K_XL on pi with xhigh thinking, and unless I'm vastly underestimating the complexity of the script that's the kind of task I would expect to take a couple of minutes (gettin
18.
by skohan 21d ago
Your assumption is incorrect.
19.
by skohan 21d ago
I'm using pi inside a self-made harness. I've found going super lightweight with context (AGENTS.md is maybe 20 lines) and letting the model discover what it needs to gives the best results.
20.
by skohan 21d ago
I've had the exact opposite experience. I've been using 3.8 for my daily driver since last week, and I've gradually been giving it more and more complex tasks as it continues to deliver high quality results. Now I am basica
21.
by skohan 21d ago
> interesting features Can you elaborate at all?
22.
by skohan 29d ago
Well I wouldn't say a billion times better. I've actually been having a surprising amount of success working with local models. And my investment has only been the equivalent of 4 months of a Max x20 subscription. Experimentat
23.
by skohan 29d ago
This also seems like quite an esoteric use-case to me, but I guess some people might need to know when text follows a fibonacci sequence in terms of word count.
24.
by skohan 1mo ago
This is a big part of the reason I went local-only. Subscription limits are horrible for having a decent workflow.
25.
by skohan 1mo ago
I would happily buy up a load of datacenter GPU's at deep discount
26.
by skohan 1mo ago
Glimmer benchmarks around Qwen 3.6 27B levels no?
27.
by skohan 1mo ago
I found Glimmer underwhelming in terms of coding - I tried it as a drop-in replacement for 3.6, and the output was noticeably worse. 3.8 has been a significant step up so far from early testing.
28.
by skohan 1mo ago
It's only been a couple days, but I haven't seen looping issues with 3.8 so far, compared to 3.6 which did occasionally have this problem.
29.
by skohan 1mo ago
At that point aren't you just edge-case testing? Surely most of your use-cases are not novel tasks that combine obscure domains. It seems to me the real way to evaluate the value of a model is how it performs in your real-life workflow
30.
by skohan 1mo ago
Depends on your use-case. Over the past couple days, I've found 192k context more than enough for coding. There's more thinking for sure compared to comparably sized models (running on xhigh), but I've found the results are

More ›