araghuvanshi

6 ms·

1.
by araghuvanshi 1y ago
Wait this is actually pretty good! What interesting use cases have you seen so far?
2.
by araghuvanshi 2y ago
Look man, Claude 3, GPT4 etc didn't work for my startup out of the box. I thought it would be helpful to tell others what I went through. Why hate on the truth?
3.
by araghuvanshi 2y ago
Fair, valid point. I do admit that this is far from a perfect analysis. I do hope, though, that it helps people at least classify their problems into categories where they need to design around the flaw rather than just assuming that the th
4.
by araghuvanshi 2y ago
Why should I? If a person told you that they can multiply, divide, add and subtract, would you not also assume that they can at least count? The point here is: the justifications from AI engineers for why counting vs math aren't the sa
5.
by araghuvanshi 2y ago
Well LLMs are claimed to be good at math too, and yet they can't count. Same point with the long contexts. And our actual use case (insurance) does need it to do both. My hope from this article is to help non-AI experts figure out when
6.
by araghuvanshi 2y ago
Well the same principle of false advertising re: context window sizes also applies to its inability to count, no? AI companies claim that their models can do math, so wouldn't a regular developer assume that they can also count? And if
7.
by araghuvanshi 2y ago
Direct quote from Anthropic's website: "Opus -Our most intelligent model, which can handle complex analysis, longer tasks with multiple steps, and higher-order math and coding tasks." So you tell me: if a regular developer re
8.
by araghuvanshi 2y ago
I don't think that this is obvious at all. Yes, AI people who read papers on arxiv and know what "SOTA" stands for know it, but that is no longer the main user base of LLMs. This is meant to be for the developer who doesn&#x2
9.
by araghuvanshi 2y ago
How much context? One sentence? Two? One paragraph? One page? It's very similar to the insurance policy problem - the text surrounding the information you're looking for, which could be surrounding it by one sentence or 10 pages,
10.
by araghuvanshi 2y ago
Totally agree there. And that's one of my points: you have to design around this flaw by doing things like what you proposed (or build an ontology like we did, which is also helpful). And the first step in this process is figuring out
11.
by araghuvanshi 2y ago
I'm talking about the fact that they boast about their models having large context windows. And Anthropic says: "Opus - Our most intelligent model, which can handle complex analysis, longer tasks with multiple steps, and higher-or
12.
by araghuvanshi 2y ago
That's true, but the problem of long context understanding (say, "summarize each of the situations where the word 'wizard' is mentioned") remains. And that gets much closer to the insurance policy thing.
13.
by araghuvanshi 2y ago
Then why do the creators of this vacuum advertise the fact that it's really good at raking? And unlike your analogy, to actually figure out that it's bad at raking you have to read a bunch of academic papers?
14.
by araghuvanshi 2y ago
Do most readers know that if you give a so-called million token context model that many tokens, it'll actually stop paying attention after the first ~30k tokens? And that if they were to try to use this product for anything serious, th
15.
by araghuvanshi 2y ago
Please see my comment below, and the "Why should I care" section of the post. Yes you can count the number of times the word "wizard" is mentioned, but for tasks that aren't quite as cut-and-dry (say, listing out al
16.
by araghuvanshi 2y ago
Some counterarguments: 1. If an AI company promises that their LLM has a million token context window, but in practice it only pays attention to the first and last 30k tokens, and then hallucinates, that is a bad practice. And prompt constr
17.
by araghuvanshi 2y ago
Good share, thank you! Yeah I think Contextual AI has also been doing some interesting work in this area. Glossary is definitely interesting and an area we're looking into. Curious to see what work is being done with building knowledge
18.
LLMs and the Harry Potter problem(pyqai.com)
65 points by araghuvanshi 2y ago|61 comments
19.
by araghuvanshi 3y ago
The cost comparisons for the same model are interesting. I'm curious about why certain providers are a lot cheaper than others - for example, mistral-8-7b on OctoAI costs $0.2/1m tokens whereas it's $0.66 using mistral's
20.
by araghuvanshi 3y ago
We use this product at pyq and I have to say, the speedup in our build times is amazing. Especially when HuggingFace is slow/backed up
21.
by araghuvanshi 4y ago
That’s exactly right! We’re definitely more targeted towards non AI experts
22.
by araghuvanshi 4y ago
Thank you! We've wondered the same. There are a few useful open-source models out there (doctr, TrOCR to name a couple) but our best guess is that it comes down to the relatively lower availability of good, public OCR datasets, especia
23.
by araghuvanshi 4y ago
Of course, we're happy to help :) The pace of change, especially in AI right now, is pretty dizzying so I can certainly relate. That's a super interesting use case. I'm curious to see if the model can achieve that out of the
24.
by araghuvanshi 4y ago
So I will admit that I don't know a lot about how elastic search works under the hood, but I can describe how ours does. Under the hood we use a large language model called BART which has 2 key benefits: 1. It has more "general kn
25.
by araghuvanshi 4y ago
Thank you! Totally hear you on the part about convincing people who know how to do it to use a tool. We've found that those folks respond well to the counterargument that they shouldn't have to do it, even if they know how to, unl
26.
Launch HN: Pyq (YC W23) – Simple APIs to Popular AI Models
123 points by araghuvanshi 4y ago|24 comments