jtbaker

5 ms·

1.
by jtbaker 16d ago
Have you been using an MTP setup? I've been having pretty good luck with the Qwen models with the built in MTP heads via https://mtplx.com at around Q4. My main driver rig is an M5 Max MBP work got for me a few weeks ago. H
2.
by jtbaker 16d ago
having a 64GB mac mini m4 pro the last few years with some increasingly capable usefulness has kept me interested in this stuff in a way that using a paid platform wouldn't have. Similar to running K8s in a homelab, something about int
3.
by jtbaker 19d ago
Same! Recently ported some code from httpx -> niquests and the experience has been good, although admittedly the governance of the project seems a little unclear. I (ok, Qwen 3.7 27B) wrote some benchmarking code to compare throughput, a
4.
by jtbaker 22d ago
Sounds like Tahoe.
5.
by jtbaker 29d ago
Qwen3.8 27B doing a lot of lifting right now, and people seem to run it pretty well on 1-2x 3090 setups...
6.
by jtbaker 29d ago
You appear to be replying to a model.
7.
by jtbaker 1mo ago
> the only big gotcha I encountered was finding, somewhat counter-intuitively, that it's often best minimize partitioning. For parquet, I think with partitioning, it's really important to be mindful of the ordering of the data
8.
by jtbaker 1mo ago
For a schema-first (vs. code first) approach (which I think would be a sweet spot for agent driven dashboarding), I'd suggest looking at https://vega.github.io/vega-lite/ or https://vega.github.io/
9.
by jtbaker 1mo ago
found the twitter street team
10.
by jtbaker 1mo ago
I don't think DuckDB itself can coordinate work across multiple nodes. But you could put it behind an HTTP layer and scale horizontally based on resource utilization?
11.
by jtbaker 1mo ago
I've got a couple of different use cases: - ETL pipelines running on K8s nodes. Using their streaming processing engine means I can run smaller pods/nodes if needed, for datasets that may have required large dataframe-like transfo
12.
by jtbaker 1mo ago
DuckDB is one of the things I've been most excited about in a long time. Introduced it to projects at 3 companies since 2023, greatly lowering resource requirements and running it in a variety of environments. Just having the ability t
13.
by jtbaker 1mo ago
boot = strapped
14.
by jtbaker 2mo ago
Looks like a couple of people are already on it: https://github.com/antirez/ds4/issues/635
15.
by jtbaker 2mo ago
https://github.com/antirez/ds4 coming out at the same time I started a new job and they gave me an m5 max a few months ago was the lightbulb moment for me.
16.
by jtbaker 2mo ago
M5 max is 614 GB/s in the 40gpu variant
17.
by jtbaker 2mo ago
I think `smolvm` may fit the bill? I've been using it to run pi.dev in a sandbox. https://smolmachines.com/
18.
by jtbaker 2mo ago
https://www.utilitydive.com/news/virginia-legislature-passes...
19.
by jtbaker 2mo ago
Buf.build, never again. Had a proto evangelist get us ingratiated into their system a few years ago and then they introduced a bunch of rent seeking behavior just to be able to to the code generation, and getting it ripped out was a huge PI
20.
by jtbaker 2mo ago
273 GB/s. Same ballpark as M4 Pro and Strix Halo.
21.
by jtbaker 3mo ago
The RTX 5000 Pro 72GB seems like kind of a sleeper to me, and sips < 300W of power, approx 1/2 that of its big bro the RTX 6000. Kind of dream about installing it in a 10" rack, it seems like it might be able to work? @jeffgeer
22.
by jtbaker 3mo ago
Nope, have both these machines, can confirm the M5 max blows the M4 mini away. It does get hot, but I use it mostly with an external monitor and keyboard. Conceptually I like the headless model better with a workstation, but work was buying
23.
by jtbaker 3mo ago
I don’t think there’s a GPU component so it’s probably a much lower power profile. Also, per this cloudflare write up the Turin gen of the AMD Epycs is very efficient: https://blog.cloudflare.com/gen13-launch/
24.
by jtbaker 3mo ago
> Gas is the cheapest, fastest zero-to-production choice for onsite power generation, and has been for a long time. Last I heard the wait time for turbines was ~5 years at the moment. I'm sure MSFT has some inside baseball with Chev
25.
by jtbaker 3mo ago
I'm using the DS4 flash IQ2 2-bit quant, per Salvadore's recommendations for my hardware in the repo. I haven't messed with the cloud hosted variant. The only other paid API I have messed with is a $20 Anthropic sub, primaril
26.
by jtbaker 3mo ago
I'm on the verge of cancelling my anthropic $20 plan since it's come out. On an M5 Max 128GB, hooked up to the pi.dev harness, I get in the neighborhood of 400-450tps prefill and 30-35tps generation. It is imminently usable and at
27.
by jtbaker 3mo ago
> Trying to run them on a unified memory Mac > but still not quite in the realm of Sonnet or DeepSeek 4 Flash these are not mutually exclusive anymore. DS4 has set the bar for me these days. https://github.com/antirez&
28.
by jtbaker 4mo ago
Launching a new product in SvelteKit! Coming from Vue and it has been mostly great.
29.
by jtbaker 4mo ago
It's already greatly improved over previous generations due to M5s having tensor cores (higher compute capacity for matmul operations, the bottleneck for prefill).
30.
by jtbaker 4mo ago
Sounds like it means you don't have to wire up the RPC server yourself anymore? Just build a docker container that invokes this quack server command, expose it over the network and connect to it from remote clients using your own acces

More ›