schopra909

12 ms·

1.
Training Text-to-Image Models 3.6× Faster(linum.ai)
6 points by schopra909 5h ago|1 comments
2.
by schopra909 7d ago
The idea of getting a speed up in language models from using diffusion is compelling, but it just doesn’t seem like discrete diffusion models work as good at discrete non-diffusion. Which kind of makes sense, tokens don’t have implicit cont
3.
by schopra909 12d ago
Totally hear you on point a/b! I think ultimately the folks with the purses won’t care enough about a for it to be taken seriously, even if it’s an engineering bottleneck. B definitely has scope but still smaller than I’d expect. When
4.
by schopra909 12d ago
IMO this will be a blip. There’s a lot of talk in the wake of all the Uber handwringing about token spend. Legacy enterprises want to look innovative to Wall Street without spooking them, so it’s easy to hop on the narrative and “show” that
5.
by schopra909 14d ago
Totally, RLVR as a concept predates DeepSeek; but they proposed a version that was simple and scalable. Popularizing a specific version of a technique is exactly what I mean by iterations on a theme. It’s only 5% different from what others
6.
by schopra909 14d ago
Progress is iterative. Everyone is always riffing on other’s ideas and can execute on them given enough support (eg $$). The person to get to an idea first is just 5% away, so it’s possible to catch up. Moreover,I think it’s impossible to k
7.
by schopra909 21d ago
That might work! Off the dome, it’s not clear to me whether spatial/depth priors are better/worse than an LLM for this type of task. Only reason I can think why the LLM might still work better here is that it’s trained to solve a
8.
by schopra909 21d ago
Aah, for this we're just trying to filter not generate. When it comes to conditioning, you'll still need a model that understands text since the primary control is text. In the original Stable Diffusion, CLIP doubled as part of th
9.
by schopra909 21d ago
What would you have in mind for a modern model? Like Dino-V3 or something of that ilk? For the LAION classifier specifically, it's trained on-top of CLIP. The bottleneck for accuracy isn't the linear/non-linear readout, it&#x
10.
by schopra909 21d ago
Hi HN, one of the authors here. Lmk if you have any questions, and I'll try my best to answer them!
11.
Getting video models to learn better, faster(linum.ai)
36 points by schopra909 21d ago|11 comments
12.
by schopra909 21d ago
Can someone explain the intuition behind the en-gram idea? I know DeepSeek published a paper about it a few months ago and the Gemma models have a lightweight version of it; but it hasn’t clicked for me yet
13.
by schopra909 23d ago
Yep on iPhone I just use two dashes —- and it looks like an emdash
14.
by schopra909 23d ago
I feel this in my bones, as someone who has been using em dashes in their texts and writing before ChatGPT existed. In the past year, I’ve had 3 or 4 times when someone has “called me out” for using AI when I’m just an em dash organically.T
15.
by schopra909 23d ago
Yep checked 3.7
16.
by schopra909 23d ago
I think this is more “gray” than this. I feel like I can rip through ideas more quickly then ever before and as a result get a lot better at designing systems and (for my work) get a lot better at designing data/model experiments. But
17.
by schopra909 1mo ago
From our experiments it’s the best video captioning model in the world by a mile. This was not the case a year ago. When reasoning got introduced a year ago to GPT 5, on average the model performed worse than GPT4-o for short video clip cap
18.
by schopra909 1mo ago
I’m not entirely sure if local development will lead to Nvidia’s supremacy being challenged. I think a simple reason why it’s been hard to unseat in Nvidia is first mover advantage. A lot more water has flown through Nvidia pipes than TPUs
19.
by schopra909 1mo ago
100% agreed.
20.
by schopra909 5mo ago
Honestly never considered the forking use case; but it makes a ton of sense when explained Congrats on the launch. This is cool tech
21.
by schopra909 6mo ago
Really cool to see innovation in terms of quality of tiny models. Great work!
22.
by schopra909 6mo ago
Very cool work! We spend a lot of time thinking about "robust representations" in the video space. Are there any alternative ideas to JEPA right now, when it comes to speech encoding that couples meaning and sound? Curious to lear
23.
by schopra909 6mo ago
Honest question, why were folks posting AI generated comments in the first place? There's such a high inertia to comment. I only comment when I have something to contribute OR find something incredibly interesting. So I'm just baf
24.
We Built an $8/Month GPU-Cluster Monitor(linum.ai)
3 points by schopra909 7mo ago|0 comments
25.
by schopra909 7mo ago
It’s a great question. In terms of pre-training even if they were was enough data at that quality, storing it and either demuxing it into raw frames OR compressing it with a sufficiently powerful encoder likely would cost a lot of $. But th
26.
by schopra909 7mo ago
honestly, it's really hard to shorten the feedback loop in this space. For this, we really just did run one experiment at a time and visually inspect the results everywhere. when you're going 0 -> 1, you're looking for &qu
27.
by schopra909 7mo ago
Hadn’t seen that before! Seems very in line with what with the broader points about regularization. In table 4 they show faster convergence in 200 epochs when used alongside REPA. I’d be curious to see if it ended up beating REPA by itself
28.
by schopra909 7mo ago
yep, Apache 2.0! so anyone's welcome to download and hack away
29.
by schopra909 7mo ago
Hi HN, I’m one of the two authors of the post and the Linum v2 text-to-video model ( https://news.ycombinator.com/item?id=46721488 ). We're releasing our Image-Video VAE (open weights) and a deep dive on how we built it.
30.
Learnings from 4 months of Image-Video VAE experiments(linum.ai)
129 points by schopra909 7mo ago|16 comments

More ›