joennlae

7 ms·

1.
by joennlae 2mo ago
what a find
2.
by joennlae 1y ago
How can I make sure that each github runner uses exactly one cpu core?
3.
by joennlae 2y ago
Trainable Llama-like transformer (with backpropagation) in numpy only (~600 lines) https://github.com/joennlae/tensorli
4.
Different Sampling Strategies for LLMs(huggingface.co)
1 points by joennlae 3y ago|0 comments
5.
by joennlae 3y ago
+1. Does someone know how to do that?
6.
by joennlae 3y ago
Author here: Let me try to give an overview as I saw some questions repeating itself. * This accelerator is for an Edge/Inference case, so there is no training on this chip. * We introduce a differentiable form of Maddness, allowing Ma
7.
by joennlae 3y ago
Thank you for the feedback :-) We have to be careful with the comparisons we make. The TPUv3 is a training and datacenter chip and not an Edge/Inference chip. They optimise for a different tradeoff, so while the comparison looks good,
8.
by joennlae 3y ago
Author here: Thank you for the feedback :-) A lot of the work regarding the comparison with „simple“ approximate matrix multiplication has been done in the preceding paper: https://arxiv.org/abs/2106.10860 While I shar
9.
Show HN: Stella Nera – Maddness Hardware Accelerator(github.com)
154 points by joennlae 3y ago|26 comments
10.
Show HN: Maddness – Energy-efficient DNNs without multiplying(arxiv.org)
2 points by joennlae 3y ago|0 comments
11.
by joennlae 3y ago
That is true. I went for a simple implementation of the layer norm and included it in the tensorli definition. But it would have been better to define it as a moduli for clarity.
12.
by joennlae 3y ago
This would be interesting to consider. But at the moment, nothing is optimized, so many things must be tackled first (especially in the backwards path, for example, buffering) to justify moving to cupy. The goal was to use it as an educatio
13.
by joennlae 3y ago
They are still applying: https://tmsearch.uspto.gov/bin/showfield?f=doc&state=4805:wl...
14.
by joennlae 3y ago
The author here: I absolutely agree with you. I went for a bit more catchy title.
15.
Show HN: less than 650 LOC trainable GPT only using NumPy(github.com)
90 points by joennlae 3y ago|18 comments
16.
Show HN: Tensorli – NumPy Only Transformer Training (<650 lines)(github.com)
2 points by joennlae 3y ago|0 comments
17.
GithubNext: AI-Powered Auto-Fix Pull Requests for Issues(githubnext.com)
3 points by joennlae 3y ago|0 comments
18.
Show HN: Tensorli - GPT-Like Transformers in Pure NumPy(github.com)
3 points by joennlae 3y ago|0 comments
19.
Nvidia uses internal LLM to support chip design efforts(blogs.nvidia.com)
2 points by joennlae 3y ago|0 comments
20.
by joennlae 3y ago
Llama1 --> 1.0T Llama2 --> 2.0T Mistral --> ?? They do not publish how many tokens it is pre-trained on, additionally to sharing no info on datasets used (except for fine-tuning). To my knowledge, no one has trained a larger LLM (&
21.
by joennlae 3y ago
You will probably notice it also when trying to push a new branch: `error: failed to push some refs to ` when using --set-upstream
22.
“THIS-IS-NOT-A-BACKUP” forking popular FOSS repos with a cron job(github.com)
1 points by joennlae 4y ago|0 comments