valine

5 ms·

1.
by valine 4mo ago
> The two modified satellites will test hardware planned for Starlink V3 and will attempt to scan Starship’s heat shield and transmit imagery down to operators to test methods of analyzing Starship’s heat shield readiness for return to l
2.
by valine 5mo ago
Apple silicon has been an unmitigated success so it makes sense they’d go with Ternus. On a related note Apple needs to add Ternus to their spell check dictionary
3.
by valine 6mo ago
Yup exactly, in principle it helps with both inference speed by reducing memory bandwidth usage and also reduces the memory footprint of your kvcache.
4.
by valine 6mo ago
So let’s start with a really simple decoder transformer with a single layer and single attention head, and train it to predict the next token in a sequence of text. To predict the next token you need a few things: a query for the very last
5.
by valine 8mo ago
The allegations of fraud made by the people invading my home and terrorizing my friends? I’ll take those with a grain of salt. And for the record, I’m not afraid of ICE, never said I was. ICE is racially profiling people, arresting them wit
6.
by valine 8mo ago
I lived my whole life in the twin cities and have a lot of friends, US citizens, who are too scared to go out to eat right now because of the ICE raids. If that wasn’t the point it is certainly the effect. I applaud Walz and Frey, and I wil
7.
by valine 9mo ago
> About the second point, I've been under the impression that because LLMs are trained on average code, they infer that the bugs and architectural flaws are desirable This is really only true about base models that haven’t undergone
8.
by valine 9mo ago
Humans don’t learn to write messy complex code. Messy, complex code is the default, writing clean code takes skill. You’re assuming the LLM produces extra complexity because it’s mimicking human code. I think it’s more likely that LLMs outp
9.
by valine 11mo ago
Image generation and image input are two totally different things. This is about feeding text into LLMs as images, it has nothing to do with image generation.
10.
by valine 1y ago
Here's my version, took about 5 minutes to create inside the ChatGPT web interface. https://valine.github.io/vibe-coded-ant-game/ I don't know if this game was vibe coded, but it certainly could have been. M
11.
by valine 1y ago
Both Claude and GPT5 can single shot this type of game. The score counter looks exactly like the type of thing Claude spits out.
12.
by valine 1y ago
Same. There are really only two features I care about in a phone: a high refresh rates and weight. At 165 grams the iPhone air is by far the lightest 120hz phone apple has ever made. Second place is the iPhone 15 Pro at 187 grams. Getting r
13.
by valine 1y ago
Embedding models are not always trained with the rest of the model. That’s the whole idea behind VLLMs. First layer embeddings are so interchangeable you can literally feed in the output of other models using linear projection layers. And l
14.
by valine 1y ago
>> For each LLM, we extract static, token-level embeddings from its input embedding layer (the ‘E‘matrix). This choice aligns our analysis with the context-free nature of stimuli typical in human categorization experiments, ensuring a
15.
by valine 1y ago
So instead of next token prediction its next event prediction. At some point this just loops around and we're back to teaching models to predict the next token in the sequence.
16.
by valine 1y ago
You’re thinking about this like the final layer of the model is all that exists. It’s highly likely reasoning is happening at a lower layer, in a different latent space that can’t natively be projected into logits.
17.
by valine 1y ago
My personal theory is that it’s an emergent property of many attention heads working together. If each attention head is a bird, reasoning would be the movement of the flock.
18.
by valine 1y ago
That’s true yeah. The model can do that because calculating latents is independent of next token prediction. You do a forward pass for each token in your sequence without the final projection to logits.
19.
by valine 1y ago
Attention computes a weighted average of all previous latents. So yes, it’s a new token as input to the forward pass, but after it feeds through an attention head it contains a little bit of every previous latent.
20.
by valine 1y ago
The dimensionality I suppose depends on the vocab size and your hidden dimension size, but that’s not really relevant. It’s a single linear projection to go from latents to logits. Reasoning is definitely not happening in the linear project
21.
by valine 1y ago
The lower dimensional logits are discarded, the original high dimensional latents are not. But yeah, the LLM doesn’t even know the sampler exists. I used the last layer as an example, but it’s likely that reasoning traces exist in the laten
22.
by valine 1y ago
I think it’s helpful to remember that language models are not producing tokens, they are producing a distribution of possible next tokens. Just because your sampler picks a sequence of tokens that contain incorrect reasoning doesn't me
23.
by valine 1y ago
The rendering is done with OpenGL, and for remote viewing I just render to an offscreen framebuffer and stream it back to the client with WebRTC. The code for that isn’t public yet, still needs some cleanup.
24.
by valine 1y ago
Thanks, I will do a deep writeup on that at some point.
25.
Show HN: Keep your PyTorch model in VRAM by hot swapping code(github.com)
77 points by valine 1y ago|7 comments
26.
by valine 2y ago
That will only have 128GB of unified memory
27.
by valine 2y ago
Probably better to be moving things from fast memory to faster memory than from slow disk to fast memory.
28.
by valine 2y ago
No one should be buying this for batch inference obviously. I remember right after OpenAI announced GPT3 I had a conversation with someone where we tried to predict how long it would be before GPT3 could run on a home desktop. This mac stud
29.
by valine 2y ago
With a mixture of experts model you only need to read a subset of the weights from memory to compute the output of each layer. The hidden dimensions are usually smaller as well so that reduces the size of the tensors you write to memory.
30.
by valine 2y ago
What would it cost to get 512GB of VRAM on an Nvidia card? That’s the real comparison.

More ›