ianand

6 ms·

1.
by ianand 4mo ago
In that case I humbly suggest my talk from AI Engineer World's Fair https://www.youtube.com/watch?v=ZuiJjkbX0Og Gives you the basics on LLM internals in about 90 minutes and includes an already built model in JavaScrip
2.
by ianand 5mo ago
It's like "GPT is all you need for the backend" [1] on steroids [1] https://news.ycombinator.com/item?id=34503418
3.
by ianand 8mo ago
Since you mention serverless it might be worth mentioning firecracker and v8 isolates.
4.
by ianand 8mo ago
I'm not a fan of the database lookup analogy either. The analogy I prefer when teaching attention is celestial mechanics. Tokens are like planets in (latent) space. The attention mechanism is like a kind of "gravity" where ea
5.
by ianand 1y ago
The site’s domain name is the best use of a .fail tld ever.
6.
by ianand 1y ago
Lol. You joke but... https://www.reddit.com/r/GraphicsProgramming/comments/pg9dw8...
7.
by ianand 1y ago
Dude, been forever. Thanks. Will DM you.
8.
by ianand 1y ago
As the guy who did GPT2 in Excel, very cool and kudos!! Curious why you chose WebGL over WebGPU? Just to show it can be done? (Also see my other comment about fetching weights from huggingface)
9.
by ianand 1y ago
Checkout https://github.com/jseeio/gpt2-tfjs fetches the weights for GPT2 from huggingface on the fly.
10.
by ianand 1y ago
Thanks. Took a quick look and definitely needed to use Google Translate but seems to have worked to get the gist of it.
11.
by ianand 1y ago
Sounds like an interesting masters thesis. Is your masters thesis available online somewhere?
12.
by ianand 1y ago
Fun fact: A decade ago the designer of HAML and Sass created a modern alternative to XSLT. https://en.wikipedia.org/wiki/Tritium_(programming_language)
13.
by ianand 1y ago
The model architecture is the same during RL but the training algorithm is substantially different.
14.
by ianand 1y ago
> LLMs that haven't gone through RL are useless to users. They are very unreliable, and will frequently go off the rails spewing garbage, going into repetition loops, etc...RL learning involves training the models on entire response
15.
by ianand 2y ago
hey, creator of spreadsheets-are-all-you-need.ai here. Thanks for mentioning! I now have a web version of GPT2 implemented in pure JavaScript for web developers at https://spreadsheets-are-all-you-need.ai/gpt2/ . The be
16.
by ianand 2y ago
Reminder that Microsoft ships RWKV with Windows (~1.5 billion devices), making it probably the most widely deployed non transformer model out there. Amazing work! https://blog.rwkv.com/p/rwkvcpp-shipping-to-half-a-billi
17.
by ianand 2y ago
> Even things like how cells are frozen (if you're on B2, are you freezing the first row, or the first and second row?) just feels wrong. FWIW one my frustrations with Excel is how it does freezing of rows. I find Google's appr
18.
by ianand 2y ago
Google Sheets does struggle past a certain size. For Spreadsheets-are-all-you-need (GPT2 Small implemented in Spreadsheet, ~1.25GB Excel file, 124M parameters), I have only been able to get the complete model working in Excel. In Google She
19.
by ianand 2y ago
Don't entirely disagree but I think "mad" is an unfair characterization and simplifies how the fascinating competitive dynamics have played out that resulted in the Llama models. Meta as well as Amazon and MS have been strate
20.
by ianand 2y ago
This has been my mental model as well. I often use two analogies to describe this. The processed a food is the more likely you’re eating a “Product”, i.e. something that’s been designed to consumed and enjoyed. I don’t say that as a necessa
21.
by ianand 2y ago
They did invent their own functions to test if the results were due to these functions being on the training date. See the section on data contamination in the paper. Agree it both kind of makes sense (regression is the best way to predict
22.
by ianand 2y ago
Thanks for this. Listening to an author on a podcast is great hack for getting a good summary of a researcher’s or book author’s views. (Especially business books)
23.
by ianand 3y ago
It's actually available in beta. It was announced while I working on this project but I kept going with pure Excel functions because I wanted to illustrate the transformer without abstractions getting in the way. It would make many asp
24.
by ianand 3y ago
This is so awesome!! I'm going to have to show this in the embeddings video I'm working on when I discuss non-text embeddings and CLIP. While I created spreadsheet-are-all-you-need.ai as teaching tool, as I've been playing wi
25.
by ianand 3y ago
Hopefully this outlasts the harlem shake bookmarklet...
26.
by ianand 3y ago
Thanks for the kind words. DM or email to reconnect!
27.
by ianand 3y ago
Author here. Appreciate the workarounds suggested but as a religious user of RSS feeds myself I can empathize with the requestor. Have logged as something to look into https://github.com/ianand/spreadsheets-are-all-you-
28.
by ianand 3y ago
Sadly I do not believe it will work in office for web due to size limits. https://learn.microsoft.com/en-us/microsoft-365/cloud-storag... Interestingly it could fit within Powerpoint for the web's limits (my
29.
by ianand 3y ago
Love this convo and the resources. Thanks for sharing this @DuncanMak and @DonHopkins
30.
by ianand 3y ago
I try to look at it as motivation to stay committed. Like spreadsheets, domain names are a bit of an obsession for me. One of the other AI side projects I'm working on is a CustomGPT to help come up with domain names ( https:/&#x

More ›