eugenhotaj

7 ms·

1.
Spelled out implementation of LLM parallelisms in pure C(github.com)
3 points by eugenhotaj 2y ago|0 comments
2.
by eugenhotaj 2y ago
This is because everyone is training with synchronous sgd. all gpus need to synchronize on each gradient step so tail latency will kill you.
3.
Zig GPT-2 inference engine(github.com)
4 points by eugenhotaj 3y ago|1 comments
4.
by eugenhotaj 3y ago
This is pretty cool. I had the same idea but in zig: https://github.com/EugenHotaj/zig_gpt2 Not fully finished yet, haven't gotten around to implementing bpe encoding/decoding and only some ops use BLAS.
5.
by eugenhotaj 4y ago
In my experience processes like these rarely work out as intended and usually add layers of bureaucracy for marginal benefit. It’s usually senior engineers or middle managers looking for “org wide impact” so they can get promoted.
6.
by eugenhotaj 4y ago
This post would sound so dumb if it didn’t come from the almighty pg.
7.
by eugenhotaj 4y ago
The issue is not that ChatGPT will kill things off, the issue is that ChatGPT 4.0 will kill things off. If you don’t think that’s a real possibility, you’re sleeping.
8.
by eugenhotaj 4y ago
Some of these are so odd. When Kylie Jenner launched her company she could have sold celery and still made $1B+. It wasn’t because of the effectiveness of her small team.
9.
by eugenhotaj 4y ago
My experience is exactly the opposite in almost all cases. Most software is much more complicated than it needs to be. Reads like the author is just butthurt at feedback they received about their work.
10.
by eugenhotaj 4y ago
I’ve likely had it all my life, but really started noticing about two years ago during the pandemic. Now I can’t unhear it. Went to a doctor a couple of times but nothing they tried really helped. Thankfully mine is not too bad and I’m most
11.
by eugenhotaj 4y ago
The network communication overhead would be way too high to make this useful. At least for current methods of training large models.
12.
by eugenhotaj 4y ago
Source: trust me bro
13.
by eugenhotaj 4y ago
Now someone just needs to pipe the output into stable diffusion.
14.
by eugenhotaj 4y ago
Just take a look at big tech salaries on levels.fyi. Very senior engineers (L8/L9) easily make 10x the salary of entry to mid-level engineers (L3/L4/L5). For a lot of tech companies, L5 is a terminal level meaning most engine
15.
by eugenhotaj 5y ago
Let me guess, someone pushed a bad config.
16.
by eugenhotaj 5y ago
This is neat for toy problems but I don't see it working well for "real" pipelines. The magical DAG creation is going to be super hard to wrap your head around and even worse to debug. This reminds me of an internal Google to
17.
by eugenhotaj 5y ago
> Since the bigger network contains the smaller network, it is perfectly capable of achieving the same performance, so the only reason why this does not happen is that SGD cannot find it. This is maybe true in the limit of infinite data,
18.
by eugenhotaj 6y ago
If your pipeline uses only “classic” ml models, then this won’t make too much sense. It’s mostly applicable to NNs.
19.
by eugenhotaj 6y ago
I also don’t have any data, but I feel like these types of tactics don’t account for higher order effects of pushing people to more extreme platforms, and in general seem extremely short sighted.
20.
by eugenhotaj 6y ago
For kernel SVMs, one needs to keep around part of the training data (the support vectors) right? With DNNs, after training, all you need are the model parameters. For very large datasets, keeping around even a small part of your training da
21.
by eugenhotaj 6y ago
True, but it sounds like you’re just shifting computation from training to inference. And I’m not sure that’s a very good trade off to make, you’re likely to predict on much more data than you trained on (e.g. ranking models at google, fb,
22.
by eugenhotaj 6y ago
Fair enough, but the number of support vectors for non trivial problems is still pretty large (as I understand but could be wrong), e.g. 20-30% of the dataset. Having to iterate over 30% of say imagenet on each batch of predictions seems un
23.
by eugenhotaj 6y ago
Don’t kernel SVMs need a full pass through the data they were trained on to make predictions? How is that faster?
24.
by eugenhotaj 6y ago
It's also strange because what they've trained is essentially a recommender system. However, at inference time you're not trying to rank 2 different articles, but the same article with 2 different titles, which I guess you co
25.
Understanding RL Vision(distill.pub)
1 points by eugenhotaj 6y ago|0 comments
26.
NYU DS-GA 1008 – Deep Learning(atcold.github.io)
121 points by eugenhotaj 6y ago|25 comments
27.
by eugenhotaj 6y ago
Hi HN! I’ve been working on the pytorch-generative library in my spare time for a couple of months now as a way to dive deeper into recent work on neural autoregressive generative modeling. The goal of the library is to provide clean, high
28.
Show HN: pytorch-generative: A nascent library for autoregressive modeling(github.com)
3 points by eugenhotaj 6y ago|1 comments
29.
Talko: An End-to-End Chat Application(medium.com)
3 points by eugenhotaj 6y ago|0 comments
30.
Writing Web Servers from Scratch in Python Using Only TCP Sockets(medium.com)
1 points by eugenhotaj 6y ago|0 comments

More ›