smodad

7 ms·

1.
by smodad 3y ago
Based on rumors, it seems that because the demand is so high and supply so short, that Nvidia is having to select who gets the cards. I bet they're likely thinking about who can do the most impactful work and OpenAI would definitely be
2.
by smodad 3y ago
For sure, I didn't mean to imply OpenAI's time-to-market was the main reason for my singularity comment. Say what you want about OpenAI, and trust me I think there's a lot of material there, but you can't say they don&#x
3.
by smodad 3y ago
So nutty. I didn't think they were going to drop such a huge bomb on the industry. We're firmly in "singularity" territory where the pace of acceleration is so fast, huge strides are being made every day / week, it
4.
by smodad 3y ago
Interesting paper. My confidence on the following idea isn't high, but I have an idea I don't think a lot of people agree with me on. The authors pretty convincingly show that transformers don't generalize well. I agree, but
5.
by smodad 3y ago
I bet you're right. Even if you take into account that a data center is a monster consumer of energy, in the grand scheme of things it's not that big. Some back of the envelope math: Global electrical production in 2022 was ~30,00
6.
by smodad 3y ago
What's funny is that even though the DGX GH200 is some of the most powerful hardware available, there's such a voracious demand that it's not gonna be enough to quench it. In fact, this is one of those cases where I think the
7.
by smodad 3y ago
> You found one of your ideas appears in a recently published paper. You can no longer work on it. This is one of the things I thought of right away when ChatGPT got released last year. "God, there's probably so many PhD cand
8.
by smodad 3y ago
I just realized that Justine was the person responsible for the massive reduction in the memory footprint of the Llama models back in March.[1] Super impressive! These are my favorite kinds of blog posts. [1] https://github.com&#
9.
by smodad 3y ago
For sure. Just to be clear, I'm not saying the situation we're in where we have to release it to the general public is a great situation to be in. But I think we're at a point where there's not any optimal solutions, onl
10.
by smodad 3y ago
I have to disagree. Not releasing it to the public makes it more dangerous. One major downside of all development being done in private is that the AI can very easily be co-opted by our self-appointed betters and you end up with a different
11.
by smodad 3y ago
Now someone build a bot that goes around looking for rogue bots and warns sysadmins.
12.
by smodad 3y ago
I'm sure I'm not the only one, but I'd love to see a write-up on that. Prod your colleague to put it somewhere on YT, HN, Reddit or somewhere.
13.
by smodad 3y ago
I agree with you on that. If it's less than 1% then it does make more sense to think of it that way.
14.
by smodad 3y ago
Really? I've always been the opposite for some reason. I find it easier to visualize 20% of a pie chart, or of a crowd, or a length, etc. It's a lot harder for me to think about 1/5 of a crowd or 1/5 of a length. It&#x27
15.
by smodad 3y ago
We should all be so lucky. It's always sad when it happens though. RIP
16.
by smodad 4y ago
I'm curious as to what your take on all this recent progress is Gwern. I checked your site to see if you had written something, but didn't see anything recent other than your very good essay "It Looks Like You’re Trying To Ta
17.
by smodad 5y ago
Q: How many psychiatrists does it take to change a lightbulb? A: None, the lightbulb has to want to change.
18.
by smodad 5y ago
I feel like I always learn something new whenever I read Gwern's articles. I found this idea particularly hilarious for some reason: ceiling-mounted infrared beam emitters+guidance infrared camera, steered by segmenting CNN: it automa
19.
by smodad 5y ago
My first thought was that a built-in GPU could measure certain parameters about the display and adjust the color profile to be "pro" level accurate. Most default / "auto" profiles for monitors and TVs are accurate e
20.
by smodad 5y ago
Thanks for posting the link, because I was about to ask. It seems like printing out the reference on a home printer would make the accuracy dependent on whether or not your printer was well calibrated (and then you're back to the same