15 ms·
Anthropic's 100k context is now available in the web UI
- tikkun 3y agoI requested access when it was released. Other HN readers, how many days did it take you from requesting access to Claude to having API access? I didn't use it prior to 100K so I don't have an existing API account.
- famouswaffles 3y agoRequested access way before 100k and still haven't gotten in.
- malux85 3y agoYeah me too, waiting patiently as context windows are our biggest blocker on more complex chemistry simulations
- peytoncasper 3y agoInteresting use case, would you be open to sharing more information on how you're using LLMs for chemistry simulations?
- famouswaffles 3y agoNot the person you responded to but these two interesting papers kind of tackle that. https://arxiv.org/abs/2304.05376 https://arxiv.org/abs/2304.05376 https://arxiv.org/abs/2304.05332 https://arxiv.org/abs/2304.05332
- npsomaratna 3y agoSame here. Been waiting for a couple of months now.
- tikkun 3y agoCould you send me an email? I've liked a few of your comments, want to say hi over email. Email in profile.
- weird-eye-issue 3y agoCreepy
- tikkun 3y agoCan someone else chime in and let me know whether they agree? Seems like the equivalent of a twitter DM to me, but maybe I'm out of touch.
- qumpis 3y agoI've tried to google the person you replied to, and it they seem to have many social/online media profiles that allow direct contacting. In that case I think publicly reaching out isn't the best way to go and seems out of place, imo.
- tikkun 3y agoGood call, I didn't think to do that - thanks
- famouswaffles 3y agoDon't think it's particularly creepy and I did send one like you asked, but my email is in my GitHub anyway and not particularly hard to find. Generally, some might not feel comfortable letting strangers know their email, especially considering this is a site that encourages anonymity. Some might not appreciate doing so publicly either.
- ryanklee 3y agoI think it's pretty inappropriate. If you have a legit reason to reach out, then you can find a way to do it privately. Letting your private intentions leak into public forums is a bad look and a red flag. If I were the person you are replying to, I'd do my best to not interact with you on the basis of your comment.
- Mockapapella 3y agobeen a couple months for me as well. Actually forgot about `claude` and have just been using OpenAI's API instead.
- ntonozzi 3y agoI requested access on March 14th or 15th and got it on March 20th.
- absentmoon 3y agoDid you fill in the form with super compelling use case or something?
- lachlan_gray 3y agoRandomly gained access long after I had forgotten I signed up, maybe 3 or 4 months
- anotheryou 3y agodid any of you get a confirmation mail or something?
- rmckayfleming 3y agoNope. Nothing. I’ve been waiting since they released it. Part of me thinks it might be because I responded yes to “Outside of the US”.
- anotheryou 3y agothanks. I'm non-US, too, but really can't remember when I requested initially. Hope I didn't bump myself down by doing it twice.
- bulbosaur123 3y agoWhere can I actually physically use it? Or is it again only limited to chosen ones?
- emptysongglass 3y agoAny magic tricks to gaining access apart from waiting for months? I've been using GPT-4 and love it but would really love to test that 100k context window with long running chatbots.
- famouswaffles 3y agoClaude-Instant-100k is available on Poe.com (but only usable as a paying subscriber). Claude-plus-100k isn't up yet but I'm guessing that's a matter of time.
- dmix 3y agoNice to see Poe is an actual iOS app for AI chat. Using ChatGPT via the Home Screen “app” is extremely frustrating because it logs you out constantly (maybe due to using Google to auth).
- costco 3y agoI don't have any evidence but I think it's probably done on purpose to make amateur automated free ChatGPT use more annoying.
- dmix 3y agoBut I have plus :(
- visarga 3y agoEvery other time I switch back to chatGPT tab it requires re-login. That’s a bad UX. Also, there is no way to search the history. The sidebar only shows titles, not contents. I have to click each one to see what’s inside. I can’t scroll much because it loads more only when I click. I ended up exporting the conversations and converting JSON to txt. Another issue: editing a long past message makes it scroll up and hide the cursor if the message is longer than one screen. I have to type in another editor and then copy&paste the whole text. The typing experience is poor.
- hacky11 3y ago[dead]
- jlowin 3y agoThe 100k context was originally released only via API, but I just noticed that it's now available in the Claude web UI.
- nico 3y agoIt’s also available here on google collab: https://twitter.com/gpt_index/status/1657757847965380610?s=46 https://twitter.com/gpt_index/status/1657757847965380610?s=4...
- anotheryou 3y agono. you still need to bring your own api key for that.
- okdood64 3y agoNew to ML here, what’s the difference between parameters and context?
- Closi 3y agoParamters - number of internal variables/weights in the model Context - Length of input/output buffer (number of input/output tokens possible).
- sghiassy 3y agoParameters is like the number of neurons in your brain Context is how much short term memory you can retain at any one time (think how many cards you can remember the order of in a deck of cards)
- capableweb 3y agoOther answers are already good, just offering yet another difference. Parameters is something that gets set indirectly via training, it's kept within the weights of the model itself. Context is what you as a user passes to the model when you're using it, it decides how much text you can actually pass it. Being able to pass more context means you can (hopefully) make it understand more things that wasn't part of the initial training.
- marcopicentini 3y agoAny timeframe when it will be released to the public? We are in the middle of developing and app and we are not able to do it with the limited context window of Open Ai. We already submitted the request of access.
- pmarreck 3y agoThere are tricks you can do to better utilize the smaller context window, such as sub-summaries and attention tricks. That's how there are already products on the market that consume entire big PDF's and let you query them. Granted, a larger context window would still work better, but it's possible to do.
- yawnxyz 3y agoit's using "overlapping chunking" methods and it usually works for generic PDFs. It really falls apart on technical documents, SOPs and research articles where you need to get context from chunks way above. Using vector DBs also doesn't work well bc you have to twiddle around with window size / overlappy-ness, which changes depending on what kind of paper you're uploading. It's a mess and takes too long
- marcopicentini 3y agoThe problem is that making a summary of a text of 100k token costs 2$ using Davinci.
- modernpink 3y agoWhat are the commercial applications of mega context window LLMs at current prices? I would guess mainly legal. And what strategies would you rely on to reduce the accumulating costs over the course of a session?
- deleted 3y ago[deleted]
- pr337h4m 3y agoAlso available on poe.com
- arpowers 3y agoIs it useful?
- arpowers 3y agoThe vast majority of AI tools are vaporware mock-ups … Adobe Firefly is best example of “just ship a mock-up of the feature” Ai marketing
- viggity 3y agoFirefly has some genuinely cool shit in it (their text treatments are pretty neat), but overall quality is dramatically lacking because they only train on images they have explicit rights to.
- adamsmith143 3y agoOf course Adobe put out crap but Claude is a real product, not vaporware...
- s3p 3y agoNeither of them put out "crap"
- adamsmith143 3y agoAdobe isn't an AI company so it stands to reason that the AI product they put out is crap. Photoshop and their other products while not "Crap" are certainly overpriced relative to opensource competitors.
- anaganisk 3y agoI'm with you if you're talking about krita, GIMP doesn't even register as a blip, if you are comparing it with Photoshop. Not opened but photopea.com and Affinity are the only best ones that come very close.
- 3y ago
- nightski 3y agoThe discourse has made it seem that with context length larger is always better. I'm wondering if there is any degradation in quality of results when the context is scaled this large. Does it scale without loss of performance? Or is there a point where even though you can fit in a lot more information it causes the performance to degrade?
- phillipcarter 3y agoIn a brief test, I found that the bigger context window only meant that I could stuff a whole schema into the input. It still hallucinated a value. When I plugged in a call to a vector embedding to only use the top k most "relevant" fields it did exactly what I wanted: https://twitter.com/_cartermp/status/1657037648400117760 https://twitter.com/_cartermp/status/1657037648400117760 YMMV.
- koboll 3y agoThe fundamental problem seems to be that it's still slightly sub-GPT-3.5-quality, and even a long context window can't fix that. It will remember things from many many tokens ago, but it still doesn't reliably produce passable work. The combination of a GPT-4-quality model and a long context window will unlock a lot of applications that now rely on somewhat lossy window-prying hacks (i.e. summarizing chunks). But any model quality below that won't move the needle much in terms of what useful work is possible, with the exception of fairly simple summarization and text analysis tasks.
- phillipcarter 3y agoMaybe! I certainly look forward to that. Although in my testing GPT-4 also hallucinates a bit (less than gpt-3.5), and the latency is so poor that it's unworkable for our product.
- koboll 3y agoAgreed. My heuristic is that GPT-4 is good for compile time tasks but bad for runtime tasks for both cost and speed reasons.
- thomasahle 3y agoThis is the world we are entering of "commercial AI" rather than public, peer reviewed AI. No benchmarks. No discussion of pros and cons. No careful comparison with state of the art. Just big numbers and big announcements.
- dmix 3y agoThey released the product to the public… we might not have formal academic studies but millions of people trying it and determining it’s utility vs the competition is as good of a test as any. If pushing the context window turns out to not be the right approach it’s not like there won’t be 10 other companies chomping at the bit to prove them wrong with their own hypothesis. And it’s entirely possible there are multiple correct answers for different usecases.
- dandellion 3y agoIt could also end up like with the transition to digital cameras and megapixels. With companies adding more and more context just because the consumers minds are already imprinted with the idea that more is better. So in a few years we might have models with a window of 30 megatokens and it'll mean absolutely nothing.
- idopmstuff 3y agoYeah, it's a weird comment to call it not "public, peer reviewed" when this article is about how it went public, giving people the opportunity to review it.
- deleted 3y ago[deleted]
- whimsicalism 3y agoIf I started selling a previously unknown cancer treatment over-the-counter in CVS, people would be justified in calling it not peer-reviewed, untested, etc. even if it is available to the public (giving people the opportunity to try it).
- atemerev 3y agoI don't understand this "slow rollout" thing about OpenAI competition. The chat / instruction models are continuously fine-tuned on real dialogues. To get these dialogues en masse, you need to deploy models to wide public. Otherwise, you will forever be on the losing side, if you can't quickly grab the streams of real time human-generated content. People at OpenAI are smart, they understood that quickly, GPT-4 is available nearly everywhere, and lesser models are even free for anyone to use. This required hiring huge teams of moderators, but we are at land grab stage, everyone in the business needs to move fast and break a lot of things. However, GPT-4 and open source models are the only thing I can use. Bard "is not available in my country" (Switzerland), and the first thing that Claude access form is asking is whether I am based in US. Well, their loss.
- s3p 3y agoI think it's cloud limitations. Anthropic probably doesn't have the ability to scale up extremely fast and accomodating hundreds of millions of users probably isn't as easy for them as it is for OpenAI.
- dataangel 3y agoIt's probably the GPUs, they don't have enough capacity to handle more users. My guess is that GPT4 set off a buying spree. Even for CPUs, I've recently heard lead times for Sapphire Rapids servers are 2-3 months, high end switches 6 months, and those probably have way less demand.
- williamcotton 3y agoIf they are resource constrained and then opened up the flood gates resulting in poor performance and timeouts for every user it seems like it would sour more milk than otherwise.
- nl 3y agoIs Bard still unavailable? It was unavailable to Australia until last week but was made more widely available at Google I/O. It's pretty good, too!
- 3y ago
- ryan93 3y ago[flagged]
- jkukul 3y ago> The gap is about 1 standard deviation Do you have any studies to link?
- ryan93 3y ago[flagged]
- famouswaffles 3y agoThis is not a study. It's a poorly backed/argued opinion piece.
- ryan93 3y ago[flagged]
- famouswaffles 3y agoBecause i've seen the post before lol. It's been on the internet for a couple years.
- ryan93 3y ago[flagged]
- famouswaffles 3y agoHow does that make sense ? You read something and then you see it's poorly argued. I'm not a magician. I don't care if people read that lol. I don't even really care if they believe the nonsense he's spouting. I reckon people like that will always exist. I'm just telling you that that's not a study. You say you had a study and then you link an opinion piece.
- ChikkaChiChi 3y agoIs there a place I can track all releases, announcements, and invite links?
- syntaxing 3y agoIs there a trick to getting access? I’ve been on the waitlist for GPT-4 and Claude for a while. Been building some proof of concepts with GPT-3.5 but having better models would be a huge help.
- gee_m_cee 3y agoIf you're referring to a paid account, I never received a notification about my GPT-4 waitlist spot. I waited awhile for one, and then, at the prompting of a colleague, I just found a spot in the web UI to sign up. After one false start, it just worked.
- pmoriarty 3y agoTry going through poe.com. I got access right away.
- wangg 3y agoSharing that this is available on Poe.com from Quora.
- celestialcheese 3y agoClaude 100k 1.3 blew me away. Giving it a task of extracting a specific column of information, using just the table header column text, from a table inside a PDF, with text extracted using tesseract, no extra layers on top. (for those that haven't tried extracting tables with OCR, it's a non-trivial problem, and the output is a mess) > 40k tokens in context, it performed at extracting the data, at 100% accuracy. Changing the prompt to target a different column from the same table, worked perfectly as well. Changing a character in the table in the OCR context to test if it was somehow hallucinating, also accurately extracted the new data. One of those "Jaw to the floor" moments for me. Did the same task in GPT-4 (just limiting the context window to just 8k tokens), and it worked, but at ~4x more expensive, and without being able to feed it the whole document.
- modernpink 3y agoWhat was the dollar cost to do this work? To iterate over a 40k context must be expensive.
- celestialcheese 3y ago~$0.45
- anonymouse008 3y ago> text extracted using tesseract You're saying 'the text' without normalizing the rows and columns (basically the tab, space or newline delimited text with sporadic lines per row) was all you needed to send? I still have to normalize my tables even for GPT-4, I guess because I have weird merged rows and columns that attempt to do grouping info on top of the table data itself.
- celestialcheese 3y agoexactly. Just sent raw tesseract output, no formatting or "fix the OCR text" step. So the data looked like: ``` col1col2col3\nrow label\tdatapoint1\tdatapoint2... ``` Very messy. I don't think this is generalizable with the same 100% accuracy across any OCR output (they can be _really_ bad). I'm still planning on doing a first pass with a better Table OCR system like Textract, DocumentAI, PaddPaddle Table, etc which should improve accuracy.
- flerovium 3y agoPOC or STFU We can't assess how good it is if it's in closed beta. It's all cherry-picked twitter.