karpathy

6 ms·

1.
by karpathy 5mo ago
All possible 36 distinct level-2 eml functions of one variable (the first 18 of them with entirely Real outputs, the other 18 with "intermediate" complex-valued components): https://imgur.com/a/K7AoOFi
2.
by karpathy 6mo ago
The most recent round of autoresearch (round 2) which decreased "time to GPT-2" from 1.8 hours to 1.65 hours had some examples. I adjusted the program.md to "look at modded nanogpt project and draw inspirations from there for
3.
by karpathy 6mo ago
Wrong and short-sighted take given that the LLM explores serially learning along the way, and can tool use and change code arbitrarily. It seems to currently default to something resembling hyperparameter tuning in absence of more specific
4.
by karpathy 6mo ago
I was exploring how to parallelize autoresearch workers. The idea is to have a trusted pool of workers who can verify contributions from a much larger untrusted pool. It's backed bit a naked git repo and a sqlite with a simple go serve
5.
by karpathy 6mo ago
So the interesting part about this one is that when I had the model write up the results for that session: https://github.com/karpathy/autoresearch/discussions/32 Look at its comment about this "improvem
6.
by karpathy 6mo ago
So I think it works to just use GitHub CLI and Discussions, e.g. my agent just posted this one: https://github.com/karpathy/autoresearch/discussions/32 Other agents could be instructed to read Discussions and
7.
by karpathy 6mo ago
Cool idea!…
8.
by karpathy 6mo ago
this is very far from hyperparameter tuning in at least three important ways: - it can modify code arbitrarily, the notion of a "hyperparameter" dissolves - there is no need to run "sweeps" - this is the standard paralle
9.
by karpathy 9mo ago
came here to look exactly for this thank you!
10.
by karpathy 9mo ago
I agree with this fwiw, for many months I talked to people who never used o3 and didn’t know what it was because it sounded weird. Maybe it wasn’t obvious at the time but that was a good major point release to make then.
11.
by karpathy 9mo ago
You’re absolutely right! Jk jk, now that you pointed it out I can’t unsee it.
12.
by karpathy 9mo ago
Yeah, I made some edits to clarify.
13.
by karpathy 9mo ago
The CC point is more about the data and environmental and general configuration context, not compute and where it happens to run today. The cloud setups are clunky because of context and UIUX user in the loop considerations, not because of
14.
by karpathy 9mo ago
Yes I noticed a few of these around. The LLM is a little too willing to give out grades for comments that were good/bad in a bit more general sense, even if they weren't making strong predictions specifically. Another thing I noti
15.
by karpathy 9mo ago
Thank you
16.
by karpathy 11mo ago
It will work great with 40GB GPU, probably a bit less than twice slower. These are micro models of a few B param at most and fit easily during both training and inference.
17.
by karpathy 11mo ago
Still under development, remaining work includes tuning nanochat (current state being solid v0.1) and finalizing the in-between projects so that students can "unlock" all complexity that hides underneath: `torch.Tensor`, `torch.di
18.
by karpathy 11mo ago
Sorry I thought it would be clear and could have clarified that the code itself is just a joke illustrating the point, as an exaggeration. This was the thread if anyone is interested https://chatgpt.com/share/68e82db9-7
19.
by karpathy 1y ago
<3
20.
by karpathy 1y ago
I like that your post deliberately gets to the point first and then (optionally) expands later, I think it's a good and generally underutilized format. I often advise people to structure their emails in the same way, e.g. first just cu
21.
by karpathy 1y ago
Omg long post. TLDR from an LLM for anyone interested Speed your audio up 2–3× with ffmpeg before sending it to OpenAI’s gpt-4o-transcribe: the shorter file uses fewer input-tokens, cuts costs by roughly a third, and processes faster with l
22.
by karpathy 1y ago
Fun demo of an early idea was posted by Oriol just yesterday :) https://x.com/OriolVinyalsML/status/1935005985070084197
23.
by karpathy 1y ago
I kind of say it in words (agreeing with you) but I agree the versioning is a bit confusing analogy because it usually additionally implies some kind of improvement. When I’m just trying to distinguish them as very different software catego
24.
by karpathy 1y ago
Btw I notice many pretty bad errors in this transcription of the talk. The actual video will be up soon I hope.
25.
by karpathy 1y ago
I also find myself wanting to go back to RSS for the exact same reasons of 1st paragraph. You own your content and host it. Unfortunately all the RSS readers are too raw and I think one of them has to port over Twitter features, things like
26.
by karpathy 1y ago
What I never fully understood is that there is some implicit assumption about the dynamics of the system. So what that there are more microstates of some macrostate as far as counting is concerned? We also have to make assumptions about the
27.
by karpathy 2y ago
My YouTube videos fall into two tracks: 1. technical track (all the GPT repro series) 2. general audience track For (2), I had a 1hr video from 1 year ago, but I didn't actually expect that video to be some kind of authoritative introd
28.
by karpathy 2y ago
Thank you for sharing your concerns. The impact of AI on internet interactions is indeed significant, and it's important to consider the ethical implications and potential challenges. Responsible development and ethical guidelines are
29.
by karpathy 2y ago
This is the video of how the parent comment was made: https://operator.chatgpt.com/v/6792c91d5b048192b5dbad908de05... But there are many things missing from the video. - Operator couldn't recover after clicking on
30.
by karpathy 2y ago
The general sentiment about the OpenAI Operator launch on Hacker News is mixed. Some users express skepticism about its current capabilities, cost, and potential overreach, while others see promise in its ability to automate tasks and impro

More ›