8 ms·
Opus just crushed Gemini Pro and GPT4 on a pretty complex question I have asked all of them, including Claude 2. It involved taking a 43 page life insurance inv
by ActVen 3y ago
Opus just crushed Gemini Pro and GPT4 on a pretty complex question I have asked all of them, including Claude 2.
It involved taking a 43 page life insurance investment pdf and identifying various figures in it. No other model has gotten close. Except for Claude 3 sonnet, which just missed one question.
- technics256 3y agoI am curious on this. can you share more?
- ActVen 3y agoHere is the list of the questions. https://imgur.com/a/D4xwczU https://imgur.com/a/D4xwczU The PDF can't be shared. But, it looks something like the one here: https://content.naic.org/sites/default/files/call_materials/CEJ%20Lincoln%20Max%20Income%20illustration%20att1.pdf https://content.naic.org/sites/default/files/call_materials/...
- uptownfunk 3y agoReally? I tried the sonnet and it just was not very good.
- zooq_ai 3y agoDid you compare it with Gemini Pro 1.5 with 1 million context window? (Ideal for 43 pg pdfs) I have access to it and I can test it against Pro 1.5
- spaceman_2020 3y agoI tried Sonnet with a question about GANs and it seemed pretty good, better than GPT-3.5