5 ms·
This is excellent. Thanks for sharing. It's always good to go back to the fundamentals. There's another resource that is also quite good: https://jaykmody.com/b
by eclectic29 2y ago
This is excellent. Thanks for sharing. It's always good to go back to the fundamentals. There's another resource that is also quite good: https://jaykmody.com/blog/gpt-from-scratch/ https://jaykmody.com/blog/gpt-from-scratch/
- _giorgio_ 2y agoNot true. Your resource is really bad. "We'll then load the trained GPT-2 model weights released by OpenAI into our implementation and generate some text."
- skinner_ 2y ago> Your resource is really bad. What a bad take. That resource is awesome. Sure, it is about inference, not training, but why is that a bad thing?
- szundi 2y agoThis is not “building from the ground up”
- abustamam 2y agoWhy is that bad?
- deleted 2y ago[deleted]
- skinner_ 2y agoNeither the author of the GPT from scratch post, nor eclectic29 who recommended it above did ever promise that the post is about building LLMs from the ground up. That was the original post. The GPT from scratch post explains, from the ground up, ground being numpy, what calculations take place inside a GPT model.
- _giorgio_ 2y agoInference is nothing without training.
- deleted 2y ago[deleted]