8 ms·
The author of the article should had provided an implementation of the transformer using only numpy or pure C++.
by metalloid 3y ago
The author of the article should had provided an implementation of the transformer using only numpy or pure C++.
- jaymody 3y agoI wrote a minimal implementation in NumPy here (the forward pass code is only 40 lines): https://github.com/jaymody/picoGPT https://github.com/jaymody/picoGPT And also a related blog post: https://news.ycombinator.com/item?id=34726115 https://news.ycombinator.com/item?id=34726115 Although this is for a decoder-only transformer (aka GPT) and doesnt include the encoder part.
- newhouseb 3y agoAnd I adapted Jay's work to Typescript (without the numpy obviously, just raw typescript/javascript): https://github.com/newhouseb/potatogpt https://github.com/newhouseb/potatogpt