msp26

9 ms·

1.
by msp26 5d ago
Got it, I'll test more deployment params when I benchmark models for vision tasks. Do you have any tips for Gemma 4 31B in particular? I quite like the model but I feel like I'm underutilizing my rented GPU hard due to skill issue
2.
by msp26 5d ago
Do you have any advice on this front? I use vllm for a project but only for text tasks at the moment.
3.
by msp26 9d ago
Relatable. Every time I finish working on something, I spot so many more new things that can be built.
4.
by msp26 26d ago
It's completely mental that HF ran into cyber safety blocks trying to use OpenAI models to help defend against the attack. They could only rely on a local hosted chinese model in the end.
5.
by msp26 1mo ago
Not sure how to fully fix this but I remember a session last week where I got so fed up mid way though reading a response that I used the following: "give me this again without jargon invented this session at high density and with a co
6.
by msp26 1mo ago
No the models are just ass at communication without being directed. Try asking them to make useful diagrams for some stuff in a codebase, out of the box without excessive hand holding they don't make good choices about what's wort
7.
by msp26 1mo ago
yep matches my experience completely But even fable has the annoying tendency to invent new jargon and produce an incomprehensible soup of text.
8.
by msp26 2mo ago
the hand drawn diagrams and highlights are charming
9.
by msp26 2mo ago
Asking fable to read it's own model card triggers this btw. Or asking if mitochondria is the powerhouse of the cell.
10.
by msp26 2mo ago
Last I heard, the feature was in beta so avoided it. But I'll definitely give it a go if it's mature now! I have been using the --watch flag to let my agent play with the notebook as I use it already.
11.
by msp26 2mo ago
I do that a decent chunk of the time yeah especially for learning. I also have a bunch of marimo notebooks that double as clis and they're lovely. But sometimes I want to do something too specific or high fidelity and it's just ea
12.
by msp26 2mo ago
I fucking love marimo for exploring data. However my use of it has decreased a little with how easily I can conjure disposable frontends with agents to explore one off things.
13.
by msp26 2mo ago
hello, please fix needing to reauth every day (sometimes with email verification). This started happening this month. It's tedious and makes switching very tempting. I'm using the VS Code extension over SSH.
14.
by msp26 2mo ago
Deeply unserious company, flip flopping on policy every week with ludicrous, sometimes invisible, guard rails on their top model. Imagine trying to make business decisions about AI use with this. I love Fable for many things including codin
15.
by msp26 3mo ago
> Summarized thinking provides the full intelligence benefits of extended thinking, while preventing misuse. > preventing misuse. Imagine not being able to read the tokens you are paying for.
16.
by msp26 3mo ago
Yep agreed completely. I couldn't imagine torturing myself with a small model for local coding. But Gemma 4 31B is so fucking good for a variety of language modelling tasks.
17.
by msp26 3mo ago
It triggered for me when I asked "Web search for your own model card (released today) and pick out your favourite highlights from the pdf"
18.
by msp26 3mo ago
>Pricing for both models is $10 per million input tokens and $50 per million output tokens.
19.
by msp26 4mo ago
hell will freeze over before anthropic release anything meaningful to the public
20.
by msp26 4mo ago
Interesting, I might try that, thanks!
21.
by msp26 4mo ago
Google is singlehandedly carrying western open source models. Gemma 4 31B is fantastic. However, it is a little painful to try to fit the best possible version into 24GB vram with vision + this drafter soon. My build doesn't support an
22.
by msp26 5mo ago
I like starting most of my projects on marimo notebooks now and slowly moving parts of it to the main codebase + db. By the end of it I might remove the notebook entirely but usually I keep it for some visualisation + running stuff as a cli
23.
by msp26 5mo ago
session usage limits this week feel like ass. Even when being careful to not break prefix caching.
24.
by msp26 5mo ago
Not necessarily with speculative decoding. Whitespace would be trivial to predict and they would petty much keep using the same amount of compute as before. I don't think that's their primary motive for doing this but it is a side
25.
by msp26 5mo ago
They don't have the compute to make Mythos generally available: that's all there is to it. The exclusivity is also nice from a marketing pov.
26.
by msp26 5mo ago
> First, Opus 4.7 uses an updated tokenizer that improves how the model processes text wow can I see it and run it locally please? Making API calls to check token counts is retarded.
27.
by msp26 6mo ago
> Data extraction tasks are amongst the easiest to evaluate because there’s a known “right” answer. Wrong. There can be a lot of subjectivity and pretending that some golden answer exists does more harm and narrows down the scope of what
28.
by msp26 6mo ago
Man the lowest end pricing has been thoroughly hiked. It was convenient while it lasted.
29.
by msp26 6mo ago
I got claude to reverse engineer the extension and compare to changedetection and here's what it came up with. Apologies for clanker slop but I think its in poor taste to not attribute the opensource tool that the service is built on (
30.
by msp26 6mo ago
see: https://news.ycombinator.com/item?id=47349069

More ›