programjames

7 ms·

121.
by programjames 10mo ago
A good start would be: 1. Robust to adversarial attacks (e.g. in classification models or LLM steering). 2. Solving ARC-AGI. Current models are optimized to solve the current problem they're presented, not really find the most general
122.
by programjames 10mo ago
Large latent flow models are unbiased. On the other hand, if you purely use policy optimization, RLHF will be biased towards short horizons. If you add in a value network, the value has some bias (e.g. MSE loss on the value --> Gaussian
123.
by programjames 10mo ago
Far too much marketing speech, far too little math or theory, and completely misses the mark on the 'next frontier'. Maybe four years ago, spatial reasoning was the problem to solve, but by 2022 it was solved. All that remained wa
124.
by programjames 11mo ago
This is only for review/position papers, though I agree that pretty much all ML papers for the past 20 years have been slop. I also consider the big names like, "Adam", "Attention", or "Diffusion" slop, be
125.
by programjames 11mo ago
No, they aren't. Locally, the person with the most esoteric knowledge is probably a weird nerd. It's mostly an accident that they chose to invest time in things typically associated with smarts. But globally, the best wizards got
126.
by programjames 11mo ago
Update 3: Okay, so my PixelCNN masking was wrong... which is why it went to the mean. The earth-mover did get better results than negative log-likelihood, but I found a better solution! The issue with negative log-likelihood was the neural
127.
by programjames 11mo ago
Update 2: After another 24 hours of training and around 100 epochs, we get down to 4.4 bits/dim and colors are starting to emerge[1]. However, an issue a friend brought up is that log-likelihood + beta distribution weights values near
128.
by programjames 11mo ago
The human traffic code is also written in blood. But humans are worse at applying the patch universally.
129.
by programjames 11mo ago
The model seems pretty shitty. Does it only look on a frame-by-frame basis? Literally one second of video context and it would never make that mistake.
130.
by programjames 11mo ago
What dates? 2000 August–2025 August gives 3.18%/year for the consumer index and 7.12%/year for egg prices. Even if you assume egg prices will come down to $3/dozen, it's still 6.24%/year.
131.
by programjames 11mo ago
Update 1: After ~12 hours of training and 45 epochs on CIFAR, I'm starting to see textures. https://imgur.com/MzKUKhH
132.
by programjames 11mo ago
Hmm, I think a mixture of beta distributions could work just as well as cateogrical here. I'm going to train it for PixelRNN, but it's going to take hours or days to train (it's a very inefficient and unparallelizable archite
133.
by programjames 11mo ago
I would prefer you refer to it as "courtesy" or "consideration" rather than "freedom".
134.
by programjames 11mo ago
I don't think schools have always been like that. I especially don't think schools have been like that for the top 10% of students until very recently (<20 years).
135.
by programjames 11mo ago
And paper money is a Chinese invention. Doesn't mean it's worthwhile to spend two weeks in an anthropology class talking about how much awesomer they are.
136.
by programjames 11mo ago
That isn't a comparison to the state of the art, just a naive quantum clock.
137.
by programjames 11mo ago
I believe the issue is measures of 'achievement' and 'excellence' typically use the 20th and 50th percentiles, respectively. For example, most education studies focus on what improves graduation rates, passing rates, or
138.
by programjames 1y ago
It's a little easier to see what's happening if you fully write out the central flow: -1/η * dw/dt = ∇L - ∇S * ⟨∇L, ∇S⟩/‖∇S‖² We're projecting the loss gradient onto the sharpness gradient, and subtrac
139.
by programjames 1y ago
I think the take is: If 100k people watch the episode, spending $200 more for higher quality subtitling comes out to... a whopping 0.2 cents per person (per episode). Let's just say that would cost an extra $1/month per person. Ar
140.
by programjames 1y ago
As someone in the Gen Z age bracket, I believe that they really do have a worse economy than their parents/grandparents due to no fault of their own, and that they're also much lazier than their parents/grandparents generatio
141.
by programjames 1y ago
I find it funny how people say GPT-5 "bombed". I noticed a significant improvement in maths and coding with GPT-5. To quantify were I've found the models useful: - GPT 3.5: Good for finding reference terms. I could not trust
142.
by programjames 1y ago
Shouldn't you expect Zoomers to be more than double millenials? They're 20 years younger, which means money is 4x cheaper.
143.
by programjames 1y ago
I'll just grant you that most societies are wholly unequipped to deal with long lifespans, and there will be tons of murder, exploitation, and suffering if we fixed our biology. First, how is that any different than the current situati
144.
by programjames 1y ago
I interpret "Computer Engineering" as electrical engineering for computers (e.g. the people making the new iPhone). I don't know if I mean "Computer Science" or "Software Engineering", because I was just u
145.
by programjames 1y ago
> “show me the connection” is a request for evidence. Evidence... yes, that narrowed it down a lot. You're looking for evidence! In a debate! I never could have guessed! Luckily for you, @TMWNN did, and tried to help twice, but clea
146.
by programjames 1y ago
EDIT: This rant isn't directed at you specifically. It's more just frustration at a phenomenon I see a lot on Reddit and (to a lesser extent) Hackernews. I'm fine if you feel personally attacked, I'm just putting this ed
147.
by programjames 1y ago
No? That was me. Also, it's very strange to say, "there's just no connection at all," instead of, "I don't think his lower offer letter was due to affirmative action, but probably due to skills in other areas (
148.
by programjames 1y ago
If you already believe he got in on merits, then obviously this is all a coincidence. But obviously the other guy you're arguing with believes he got in due to ideological fit, so it's no coincidence he espouses the ideology. You&
149.
by programjames 1y ago
Why does MIT get a 91.6/100 for the "International Student Ratio" while Peking University gets 37.3/100, when they have nearly identical undergraduate ratios? EDIT: Ah, it also includes graduate students.
150.
by programjames 1y ago
I think Harvard and MIT students mostly say, "Cambridge," rather than Boston. Then, when the inquirer frowns in confusion, they mumble, "Boston".

More ›