sasjaws

5 ms·

1.
by sasjaws 7mo ago
The idea i tried to express was purely the loss function thing you mentioned, and how both tasks (1 vs 2 vs n) lead to identical training runs. At least with nanogpt. I dont know if that extrapolates well to current llm internals and curren
2.
by sasjaws 7mo ago
No blog post, my llm expert friend told me this was kinda obvious when i shared it with him so i didnt think it was worth it. I can tell you how i got there, i did nanogpt, then tried to be smart and train a model with a loss function that
3.
by sasjaws 7mo ago
This is about reasoning tokens right? I didnt mean that, nanogpt doesnt do that. Nanogpt inference just outputs letters directly, no intermediate tokens.
4.
by sasjaws 7mo ago
Thats actually an interesting way to look at it. However i just posted that because i often see articles expressing amazement at how training an llm at next token prediction can take it so far. Seemingly ontrasting the simplicity of the tra
5.
by sasjaws 7mo ago
Thats the one, lots of fun and a great entrypoint for experimentation.
6.
by sasjaws 7mo ago
A while ago i did the nanogpt tutorial, i went through some math with pen and paper and noticed the loss function for 'predict the next token' and 'predict the next 2 tokens' (or n tokens) is identical. That was a bit of
7.
by sasjaws 10mo ago
Personalized audio streams for language learners. Ideal for during driving or while doing chores. https://listen.longyan.io/ At the intermediate level lots of learners struggle to find suitable content that matches their le
8.
by sasjaws 1y ago
I've added support for Lithuanian and created a stream about version control for you to try it out. Just 'select language' -> Lithuanian -> Play If you find it useful, you can register for free and create new streams on
9.
by sasjaws 1y ago
I'm building a service that generates audio streams about subjects and vocab of your choosing, currently notebookLM based. If you have intermediate listening skills its pretty useful for deepening regular vocab and acquiring specialize
10.
by sasjaws 1y ago
Thanks for having a look, I actually started out from traditional characters, but once I realized >90% of the students only do simplified I switched. I also tend to believe to just convert between them is not the best approach. Better to
11.
by sasjaws 1y ago
I'm building a reader app that tries to solve this exact problem by providing a range of gradually simplified versions of each article to match your proficiency. So you can stay in the sweet spot, or work your way up version by version