salamo

8 ms·

1.
Show HN: Teaching a Neural Net to Fight(blog.lukesalamone.com)
1 points by salamo 20d ago|0 comments
2.
by salamo 28d ago
> We have paused some frontier RL training to ensure that we can meet the appropriate alignment, security and monitoring standards for the new level of capabilities in front of us. Model progress is now extremely rapid, and we always sai
3.
by salamo 29d ago
This matches my experience with Sol. Read and thought for a while, and edited files, tested, edited again, then ran out of budget in a relatively short time. But its solution was very good and was done quickly, so all things equal I prefer
4.
Optimal Ask(blog.lukesalamone.com)
1 points by salamo 1mo ago|0 comments
5.
by salamo 1mo ago
As someone who's done something similar ( https://blog.lukesalamone.com/posts/creating-tiny-semantic-s... ) the expensive part wasn't the training itself but the data curation and evaluation post-training. For
6.
by salamo 2mo ago
Possible reasons: - They might be dynamically adjusting these at inference time [1]. For example, start with a low temperature and generate samples with increasingly high temperatures until one of them passes some quality gate. - They don&#
7.
Keynesian Beauty Contest(en.wikipedia.org)
2 points by salamo 2mo ago|0 comments
8.
by salamo 2mo ago
Yeah, the model is small enough that inference is already basically instant for my usecase (only 6 transformer layers for the blog search).
9.
by salamo 2mo ago
Yeah, I also found that for ultra low footprint models ORT is a big portion of the total payload, because it contains logic for general ONNX graph operations. In my case I found that ORT alone was 3.4MB over the wire, so I swapped it out fo
10.
by salamo 2mo ago
On the one hand, compacting at 300k saves OpenAI 40%. That's great. On the other hand, $10 for 1M tokens still seems really high? It's not too hard to blow through that in an hour or two.
11.
by salamo 2mo ago
I come at it from a slightly different angle. I write technical blog posts with visualizations and live demos. That usually means embedding a bit of custom javascript in the page for the demo. Or shipping custom wasm to enable extreme seman
12.
by salamo 2mo ago
$1.7 billion is small potatoes. My bill is over $155 billion and growing. I'm worried if the trend continues I'll have depleted my rainy day fund.
13.
International Fixed Calendar(en.wikipedia.org)
4 points by salamo 3mo ago|0 comments
14.
Semantic Search in Under 3MB(blog.lukesalamone.com)
5 points by salamo 3mo ago|0 comments
15.
by salamo 3mo ago
An alternative algorithm which would probably converge faster than 100 questions would be something like Elo or Glicko 2. A word's "difficulty" would be some function of how rare it is. Once you have a reasonable estimate of
16.
by salamo 3mo ago
I’m working on an iOS app, One Million Checkmates [1]. It scratches an itch I had of chess puzzles for a long plane ride. This app has a functionally unlimited number of puzzles, all offline. There was a decent amount of work involved in ge
17.
PyTorch Landscape(pytorch.landscape2.io)
97 points by salamo 4mo ago|25 comments
18.
by salamo 5mo ago
> I'd really wouldn't on the machine. I'll second this. Much better to set up a second machine you can ssh/tailscale into. If a training run takes down your training machine, you don't want it to also take down
19.
Opponent Modeling Wins 2× Faster Than Stockfish(blog.lukesalamone.com)
2 points by salamo 7mo ago|0 comments
20.
by salamo 7mo ago
See, that's why I have to post these things. Someone will inevitably reply with something more clever.
21.
by salamo 7mo ago
You'll also have some fun pinning down the difference between an "inaccuracy", a "mistake", and a "blunder". These are meaningful delineations for humans but not for a chess algorithm. Objectively, any amo
22.
Graph Topology and Battle Royale Mechanics(blog.lukesalamone.com)
47 points by salamo 7mo ago|4 comments
23.
by salamo 1y ago
Really happy to see additional solutions for on-device ML. That said, I probably wouldn't use this unless mine was one of the specific use cases supported[0]. I have no idea how hard it would be to add a new model supporting arbitrary
24.
by salamo 2y ago
I mainly blog for myself in the future, but in a slightly different flavor than the author mentions. If there's a complicated ML concept that I'd really like to understand, explaining it to an audience (even if that audience is my
25.
Notes on DeepSeek R1(blog.lukesalamone.com)
2 points by salamo 2y ago|0 comments
26.
by salamo 2y ago
Ok, I emailed you.
27.
by salamo 2y ago
I live in the South Bay and would be willing to "host". But there's no way to announce a meetup or even reach out to some people.
28.
by salamo 2y ago
I believe that observation is borne out in the statistics too, but traditional chess training usually centers around finding the best, hard-to-find move in a position rather than avoiding blunders. I think it would be great if there was mor
29.
by salamo 2y ago
Sure, go ahead!
30.
by salamo 2y ago
I'm happy you enjoyed it! There are definitely a few rough edges, yes. Since the whole thing is executed in the browser (including the model) there aren't a ton of secrets for me to keep. Essentially it is expectation maximization

More ›