jayalammar

7 ms·

1.
by jayalammar 2y ago
We actually just wrote a book with your profile in mind -- especially if by "AI" you're especially interested in LLMs and if you're a visual learner. It's called Hands-On Large Language Models and it contains 300 or
2.
A Visual Guide to Mixture of Experts (Moe) LLMs(newsletter.maartengrootendorst.com)
3 points by jayalammar 2y ago|0 comments
3.
Building a RAG chatbot with query rewriting and citations(txt.cohere.com)
4 points by jayalammar 3y ago|0 comments
4.
by jayalammar 3y ago
This is my sense as well. Text generation LLMs haven't been the best source of embeddings for other downstream use cases. If you're optimizing for token embeddings (e.g., for NER, span detection, or token classification tasks), th
5.
by jayalammar 3y ago
Contribution page: https://sites.google.com/cohere.com/aya-en/home
6.
by jayalammar 3y ago
That's beautiful! Hope you're getting to do more of these!
7.
by jayalammar 3y ago
Additional ones that come to mind now are: Transformer Feed-Forward Layers Are Key-Value Memories https://arxiv.org/abs/2012.14913 The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Trainin
8.
by jayalammar 3y ago
Thank you so much (and others for your kind messages). Glad you found them useful! Writing is the best way for me to learn, I find.
9.
by jayalammar 3y ago
This is a field I find fascinating. It's generally the research field of Machine Learning Interpretability. The BlackboxNLP workshop is one of the main places for investigating this and is a very popular academic workshop https:/
10.
by jayalammar 3y ago
The goal is to make the materials as accessible as possible. So we're definitely not limited to the structure of a typical university course and are happy to iterate on it. I appreciate you elaborating on your feedback. Thank you.
11.
by jayalammar 3y ago
The landing page is technically the course overview. I'd love to hear what you think would've made it more engaging for you. We can probably pull up some of the visuals to it as a preview. Let me see what we can do on that front.
12.
Show HN: Visual intuitive explanations of LLM concepts (LLM University)
303 points by jayalammar 3y ago|36 comments
13.
by jayalammar 3y ago
I'm the author of https://jalammar.github.io/illustrated-transformer/ and have spent years since introducing people to Transformers and thinking of how best to communicate those concepts. I've found that diff
14.
by jayalammar 3y ago
How would you add that data? As new columns you mean? Or add the paragraph headings to the text of the paragraphs before embedding them?
15.
by jayalammar 3y ago
There's a lot you can do with the vectors themselves without needing to embed any more text (e.g., clustering, exploration, visualization after dimensionality reduction...etc). Here's a previous embeddings exploration of top HN po
16.
by jayalammar 3y ago
Cohere actually trains its own models and they are not based on models from other providers [I work at Cohere]. Your prompt suggestion is a good one for LLMs as a whole. Any information added to the context informs the model and nudges it t
17.
by jayalammar 3y ago
For Cohere, make sure you're using Command-Xlarge-Nightly. Otherwise, you may be prompting a Base LLM expecting the behavior of a different kind of LLM (an instruction-tuned chat model). Cohere's Command model builds on top of the
18.
Will AI take away the coding jobs?(noahpinion.substack.com)
1 points by jayalammar 4y ago|2 comments
19.
by jayalammar 4y ago
Hi. Author here. This is the first in a series I've been writing for a while to help orient people about useful perspectives to have in catching up to all that's happening in AI/ML. It's based on what I've seen in t
20.
Is Generative AI the future or the present?(txt.cohere.ai)
1 points by jayalammar 4y ago|1 comments
21.
by jayalammar 4y ago
They're trained and focused on language data, actually, not code specifically. There are both generation models and multilingual text embedding models (100+ languages, single model).
22.
by jayalammar 4y ago
We train and serve large models at cohere.ai. We've shared some optimization techniques here: https://txt.cohere.ai/running-large-language-models-in-produ...
23.
by jayalammar 4y ago
Agreed. "Stable Diffusion with Diffusers" and "The Annotated Diffusion Model" were excellent and are linked in the article. The code in Diffusers was also a good reference.
24.
by jayalammar 4y ago
I updated the post to say "multi-dimensional array". In a context like this, we use tensor because it allows for any number of dimensions (while vector/ array is only one, matrix is two). When you get into ML libraries, both
25.
by jayalammar 4y ago
Two diffusion processes are involved: 1- Forward Diffusion (adding noise, and training the Unet to predict how much noise is added in each step) 2- Generating the image by denoising. This doesn't predict the final image, each step only
26.
A neural network layer is a key-value/attention memory(arxiv.org)
2 points by jayalammar 4y ago|0 comments
27.
by jayalammar 4y ago
Thanks. I just added a disclosure to the comment (can't edit the parent anymore). The full embeddings are freely provided here without the need to use the service.
28.
by jayalammar 4y ago
Disclosure: These were made by Cohere's embeddings, a company where I work. The process should work on text embeddings from other sources.
29.
by jayalammar 4y ago
I think that's part of the expectations out of "ask HN". I don't know that the same effect happens outside of Ask HN.
30.
by jayalammar 4y ago
TIL louvain clustering! I see it used for graphs. Can also be used for vectors/points? Thank you!

More ›