lambda

8 ms·

31.
by lambda 2mo ago
> I think that in 10-15 years, we are going to have consumer PCs (and phones!) running models doing pretty much anything that frontier models can do right now 10-15 years? The current rate is closer to 10-15 months. 15 months ago, the to
32.
by lambda 2mo ago
The article describes it. Both Codex and Claude Code have it, but they work slightly differently. Claude Code uses Haiku to read through the transcript and decide if the goal has been completed. If not, Haiku injects a prompt back to the ma
33.
by lambda 2mo ago
I've tried doing a loop of rending the SVG and then tweaking based on that, with local models (so, not nearly as strong). It wasn't very successful; it would mostly report that the image looked great and didn't need any tweak
34.
by lambda 2mo ago
You can always ask them to draw something else, as a way to avoid any possible pelican related data contamination; given how popular the pelican test is, I'm sure there's some pelican SVG drawing in the training sets of at least s
35.
by lambda 2mo ago
It's really frustrating, because now when I want to write something like a "not X but Y" or "you're absolutely right," I have to stop and decide if I want to self-censor to avoid sounding like a bot. Sometimes
36.
by lambda 2mo ago
It drives us crazy because everyone is using the same 2-3 different machines. So rather than each person having their own unique speaking style, the whole world (or, everyone that publishes direct LLM output) is now speaking in the same cou
37.
by lambda 2mo ago
He is saying that weaker models, as measured by a benchmark to distinguish "frontier" models, would be exempted. So an academic lab or startup that isn't yet producing frontier models would be exempted, but once it crossed so
38.
by lambda 2mo ago
Yeah, Kraken SDR removed some functionality due to these concerns, if I remember correctly. Odd, because export controls don't generally apply to published material (like open source software), but maybe they were worried that because
39.
by lambda 2mo ago
But there are already benchtop or handheld signal analyzer for that purpose. This seems more like a tool for checking across entire large assemblies like an entire building, car, aircraft, etc, for unknown sources. If you have an individual
40.
by lambda 2mo ago
I think that for a single device, this probably wouldn't help much over just having a more traditional signal analyzer, either benchtop or handheld. If you know what you're testing, just using a signal analyzer around it will give
41.
by lambda 2mo ago
This is really weird. Someone creating 4 new accounts just to call this project fraudulent because it can't statically analyze every property you'd like? Does this person have a personal grudge against the author, or something?
42.
by lambda 3mo ago
Tried it out. I'm compring against Qwen 3.5 122B-A10B, so a much larger model. It gets some correct, but Qwen 3.5 122B-A10B has done much better. Gemma 4 12B even hallucinated some species in trying to identify a plant, and the other g
43.
by lambda 3mo ago
I haven't run a proper eval, but I've been getting better luck with Qwen models than Gemma on plant and animal identification using vision. I do like Gemma for translation, however.
44.
by lambda 3mo ago
How could the harness fix this? It's the jinja template used by the inference engine to render the API requests into the raw text that gets tokenized and completed by the model. Unless you're using something like the raw completio
45.
by lambda 3mo ago
Much more complex than that. Even if it does give you a speedup at certain tasks, is it worth the cost and risks? You go faster, but now you have more code that you don't understand and so won't be as good at maintaining. There&#x
46.
by lambda 3mo ago
It means that even if it works for certain tasks, I think that the problems caused by use of LLMs outweigh their benefits. I think it's a bad idea to generate large piles of code that you don't understand, but due to competitive p
47.
by lambda 3mo ago
Huh? There is a Claude 4 Opus. It was released about a year ago. It is retired by now, in fact, just retired yesterday: https://platform.claude.com/docs/en/about-claude/model-depre... But it is still availabl
48.
by lambda 3mo ago
Not a harness issue. The harness (pi in my case) passes back the cot for all previous turns. The jinja template is what renders the openai-format request sent by the harness, into the actual string of text that will be tokenized and fed to
49.
by lambda 3mo ago
Right. Local models haven't quite hit that level yet. The biggest open models, which you need tens of thousands of dollars of hardware to run at reasonable speed, have pretty much hit that level of capability, but most models you can r
50.
by lambda 3mo ago
Which Opus? Anthropic has been releasing models named Opus since 2024 with Claude 3 Opus. Opus has gotten vastly more capable since then. Local model far surpass Opus 3. They even surpass Opus 4 on most benchmarks. Sure, if you compare to t
51.
by lambda 3mo ago
So, one of the ways that this problem manifests is that most local models aren't trained on preserving the full reasoning between turns. Every turn, they skip passing the reasoning trace from previous turns to the the LLM. So if on one
52.
by lambda 3mo ago
Yeah, definitely.
53.
by lambda 3mo ago
Yeah. I was pointing out that local 3b active models outperform frontier models from a year ago. Will this trend continue? Who knows. Both the frontier and local model will probably continue to get better. Which one will hit the top of the
54.
by lambda 3mo ago
OK, it looks like he did a browser OS test with both Claude 4 Opus and Qwen 3.6 35B-A3B. Claude 4 Opus: https://youtu.be/J7omabtqnBM?t=193 Qwen 3.6 35B A3B: https://youtu.be/gVU-DQeqkI0?t=215 Qwen 3.6 produ
55.
by lambda 3mo ago
Which Opus? They certainly outperform Claude 3 Opus. Anyhow, feel free to try them out head to head on OpenRouter. I'd love to see someone write up their results, of a modern local sized open source model vs. frontier models from ~a ye
56.
by lambda 3mo ago
That should do pretty well. Memory bandwidth is the biggest bottleneck for token generation, at 644 GB/s you should be able to do pretty well on a 9070, while prompt proessing is more compute bound and Nvidia tends to have the edge the
57.
by lambda 3mo ago
I use Vulkan mostly instead of ROCm. Vulkan is actually a bit faster, paradoxically. I do switch out and try them both out, and it's not a huge difference, but I've been mostly saying on Vulkan. The re-processing context every tur
58.
by lambda 3mo ago
If you believe the benchmarks, Qwen 3.6 35B-A3B already outperforms Claude 4 Opus. Now, there's a bit of a degree to which some of the open source models do some benchmaxxing, and bigger models with more params may always feel like the
59.
by lambda 3mo ago
The thing is, to do a proper fix it would really need all of the context (maybe the tool call that failed was for an edit to a file that was last touched way at the beginning of the context), so you'd need to either keep that smaller m
60.
by lambda 3mo ago
This is very similar to my setup. Pi in a container (I do let it have network access, just no access to creds or anything, only the one directory that I'm working on at the time and my ~/.pi directory), talking to llama.cpp in ano

More ›