t1amat

7 ms·

1.
by t1amat 6mo ago
Claude Code is closed source so this isn’t a concern they should have as Opus is great at Rust.
2.
by t1amat 7mo ago
Interesting idea, but I think it might have made more sense to use something like repomix to generate the source bundle and tiktoken’d that. Practically speaking you don’t send many source files in raw text form, either they have some sort
3.
by t1amat 7mo ago
When most people refer to “GLM” they refer to the mainline model. The difference in scale between GLM 5 and GLM 4.7 Flash is enormous: one runs on acceptably on a phone, the other on $100k+ hardware minimum. While GLM 4.7 Flash is a gift
4.
by t1amat 7mo ago
Perhaps the opposite: a language small enough that its entirety can easily be stuffed in context.
5.
by t1amat 8mo ago
Not a direct answer but it looks like v0.5 is a nanoGPT arch and v1 is a Phi 1.5 arch, which should be well supported by quanting utilities for any engine. They are small too and should be able to be done on a potato.
6.
by t1amat 9mo ago
With M2, yes - I’ve used it in Claude Code (e.g. native tool calling), Roo/Cline (e.g. custom tool parsing), etc. It’s quite good and for some time the best model to self-host. At 4bit it can fit on 2x RTX 6000 Pro (e.g. ~200GB VRAM)
7.
by t1amat 1y ago
You might have 1A rights as an American but it seems to me the manner in which this person protested would be grounds for termination in many jurisdictions.
8.
by t1amat 1y ago
This is the right take. You might be able to get decent (2-3x less than a GPU rig) token generation, which is adequate, but your prompt processing speeds are more like 50-100x slower. A hardware solution is needed to make long context act
9.
by t1amat 1y ago
The problem with OpenAI models is the lack of a Max-like subscription for a good agentic harness. Maybe OpenAI or Microsoft could fix this. I just went through the agony of provisioning my team with new Claude Code 5x subs 2 weeks ago afte
10.
by t1amat 1y ago
Is this actually true? Last I checked (a week ago?) Codex the agents were free at some tiers in a preview capacity (with future rate limits based on tier), but codex cli was not. With codex cli you can log in but the purpose of that is to
11.
by t1amat 1y ago
I doubt this is true anymore, if ever. Both require string escaping, which is the real hurdle. And they are heavily trained on JSON for tool calling.
12.
by t1amat 1y ago
With 32B active parameters it would be ridiculously slow at generation.
13.
by t1amat 1y ago
The user should be able to enable/disable tools or an entire tab’s toolset. Some keep open hundreds of tabs and that’s simply too many potential tools to expose. Deduping doesn’t make sense for the reasons you say, and that one logic
14.
by t1amat 1y ago
Your filter doesn’t seem to be working properly right now.
15.
by t1amat 1y ago
(Replying to both siblings questioning this) If the primary use case is input heavy, which is true of agentic tools, there’s a world where partial GPU offload with many channels of DDR5 system RAM leads to an overall better experience. A g
16.
by t1amat 1y ago
The UI is the product. If you just want the engine, use mlx-omni-server (for MLX) or llama-swap (for GGUF) and huggingface-cli (for model downloads).
17.
by t1amat 1y ago
I would recommend Qwen3 30B A3B for you. The MLX 4bit DWQ quants are fantastic.
18.
by t1amat 1y ago
Gemma3 models can follow instructions but were not trained to call tools, which is the backbone of MCP support. You would likely have a better experience with models from the Qwen3 family.
19.
by t1amat 1y ago
Not a lawyer but the timing of this seems poor when the govt is deciding on whether Google should spin off Chrome or not.
20.
by t1amat 1y ago
Llama is arguably the reason open weight LLM’s are a thing, with the leak of Llama 1 and subsequent release of Llama 2. Llama 3 was a huge push for quality, size, context length, and multi-modality. Llama 4 Maverick is clearly better than
21.
by t1amat 1y ago
But that’s just it: 300 good lines of reasonably complex working code in an hour vs o4-mini can churn out 600 lines of perfectly compilable code in less than 2 minutes, including the time it takes me to assemble the context with a tool such
22.
by t1amat 1y ago
Use a tool like repomix (npm), which has extensions in some editors (at least VSCode) that can quickly bundle source files into a machine readable format
23.
by t1amat 1y ago
This looks interesting, however Zod has become a standard of sorts and a lot of libraries I use expect, for example, a JSON schema defined as a Zod schema. I would need some sort of adapter to a Zod schema to make this work for me.
24.
by t1amat 1y ago
With Zod you can build a schema that would match an existing type. Typescript will complain if the schema you build does not match the type you are representing, which is helpful. From memory: import { z } from ‘zod’ type Message =
25.
by t1amat 1y ago
Regarding the repetition loops, I found that adding the end of turn token to the stop param was enough. Documentation mentions detecting this. But your point about quality stands. Separately, this model emits the docling XML format, not t
26.
by t1amat 1y ago
Goose is my favorite desktop experience with bring-your-own API key (most providers) and MCP support: https://github.com/block/goose The real win with this is CLI-driven agentic behavior like Claude Code without an API
27.
by t1amat 1y ago
Thank you, this was a great explanation!
28.
by t1amat 1y ago
I would like to see more knowledgeable people with experience talk about this. Is it just a matter of assembling Q/A pairs like: “What’s class X?”, “class X { … }” Do you really need to do this training on the base model instead, which
29.
by t1amat 2y ago
See QwQ-Max-Preview: https://qwenlm.github.io/blog/qwq-max-preview/
30.
by t1amat 2y ago
In fairness it’s become even more difficult now than ever before. * hardware spec * inference engine * specific model - differences to tokenizer will make models faster/slower with equivalent parameter count * quantization used - and y

More ›