musebox35

6 ms·

1.
by musebox35 20d ago
It was Nvidia’s competitors’ job to ensure this never happened but hardware companies rarely value the software stack as much as they should have. AMD screwed up several times to build a similar tool and Intel did not manage to create a pro
2.
by musebox35 23d ago
This is one question that is not just for kids but for all "gpu poor" adults as well. One concept that I found useful is "toy models": small compute models that replicate a certain, not very well explained phenomenon tha
3.
by musebox35 23d ago
A more apt analogy is the diminishing understanding of assembly language and machine instruction sets. I love programming and something got lost in both transitions, first to high level languages and now to coding agents. I am quite sure so
4.
by musebox35 27d ago
That also surprised me, the diffusion gemma is actually a bunch of architectural changes to make discrete diffusion practical and an sft + distill of the Gemma 4. One cool trick they do is to take advantage of the entropy of the estimated p
5.
by musebox35 1mo ago
This is one of the dilemmas that I am trying to wrap my head around. I love optimizing software pipelines, which often boils down to figuring out the operational constraints that the compiler and the generic libraries can’t assume. Then I e
6.
by musebox35 1mo ago
I would add that it is not just capacity, but also negotiation ability. With scale comes the ability to negotiate better prices than everyone else. Even if you can find capacity for your smallish user base, your inference cost can not match
7.
by musebox35 1mo ago
The biggest advantage of tpus is the high bandwidth fiber optic interconnect between them that allows distributed computing on pods with thousands of tpus and the co-design of cooling systems that go with their racks. I do not think that we
8.
by musebox35 2mo ago
Sadly no. Despite the name, the experts are not routed per concept or topic but per token. So for the same sentence you might activate multiple experts for different tokens. What it solves is the distributed training and inference problem.
9.
by musebox35 2mo ago
dev variants are usually cfg distilled which means that directly finetuning isn’t as effective. In the past, for the flux2 klein models,they released base versions that are not distilled. So it will probably be a while before you can fully
10.
by musebox35 3mo ago
Thanks for posting this. This is the tinkerer mentality. It is not for everyone, but certain things can only be learned in that way. It is the best antidote to AI paranoia. There is much that does not transfer between frontier models and lo
11.
by musebox35 3mo ago
Training isn’t a single homogeneous step. It starts with pretraining which requires bulk PB of data but you have less quality concerns here. You cover the whole data distribution. Later stages require less and less but increasingly higher q
12.
by musebox35 3mo ago
Thanks it looks promising :)
13.
by musebox35 3mo ago
I also have been using emacs for almost anything for the past 20 years. I had to switch to VSCode for coding over a remote ssh connection to cloud VMs. The client/server split of vscode felt superior over the ssh connection and the ema
14.
by musebox35 3mo ago
True enough, but that is true for all the products I buy. I do not expect to control every product I own. For some I prefer to have more control, for others I just need something that works out of the box. There is always an initial bias fo
15.
by musebox35 3mo ago
It is much more reasonable to do it in a visible / flagged way. At least you have visibility over the quality of service you get as a customer. Silent treatment is a breach of trust, what you buy changes depending on the context based
16.
by musebox35 3mo ago
I work on open source text-to-image finetuning of open source models like zimage/flux2 klein 4b and inference time latency optimization. The moment I read the silent treatment, I went ahead and cancelled my subscription too since I wou
17.
by musebox35 3mo ago
SFT + RL connection to model/hypothesis search is insightful. Brute force / scalable search is where Sutton's Bitter Lesson also points to. Once your search domain is small compared to your search budget, that makes a lot of
18.
by musebox35 3mo ago
I understand the skepticism. I am worried about the implications of AI as well. The deeper issue at stake is that the depth of scientific knowledge has been increasing for a very long time. Now you get to have a PhD in esoteric subproblems
19.
by musebox35 3mo ago
The most successful applications like coding are not the result of pure LLM/generative modeling. They come from closing the loop with an agentic harness. The generate-test-selectively refine loop is the core modality of scientific work
20.
by musebox35 3mo ago
I was about to post your last point / quote. Going multigpu is relatively not so though but once you go multi-node you have distributed storage/io/compute system which is highly non trivial. Add that the long training times n
21.
by musebox35 4mo ago
Not understanding the whole does not completely remove an ability to analyze. An interesting direction is individuality and having a notion of self. It is difficult to demarcate the individual for a model given how much the system prompt an
22.
by musebox35 4mo ago
I think the complexity issue in science and engineering has also been growing for some time beyond what can be analyzed/designed by a person or a group with conventional software and math. Wolfram argues that some processes are so comp
23.
by musebox35 4mo ago
The cutoff date is early 2025 so make sure to enable web search when experimenting. I was expecting something more recent, took a while to notice this.
24.
by musebox35 4mo ago
Thanks, that sounds like a good direction to try.
25.
by musebox35 4mo ago
Most of your analysis I can easily relate to except “There is evidence that the Chinese models are falling further behind, not gaining.” Where is that evidence? Deepseekv4 claims to be trailing front runners by six months. I read people agr
26.
by musebox35 4mo ago
I watched the last one S5:E17 What jobs are AI jobs and I think it gives the right framing to think about this. It is not prescriptive, it does not give a list which is smart. The job title might be the same but the actual role might have d
27.
by musebox35 4mo ago
I am exploring ways to document the design for the agent to read and update. What makes it difficult is the lack of structure. Spec writing is not my core skill. Schemas and APIs are easier, there are declarative ways to document them. Runt
28.
by musebox35 4mo ago
I totally agree. I loved coding because of its closed feedback loop. Since last November, I also delegated it mostly to agents. Now I concentrate more on the design part, which is not the same. However, you move with the times and hope some
29.
by musebox35 4mo ago
Could you briefly describe your workflow for doing that or give a pointer to a blog you wrote/like that aligns with the process? Thanks in any case, happy designing ;-)
30.
by musebox35 4mo ago
Please see Ben Evans’ podcast on a good take on this. Coding is just one of the task you do in your job, it is not the job or at least it probably is not. You do not get paid to code, you get paid to make a set of decisions that create valu

More ›