8 ms·

Demo of =GPT3() as a spreadsheet feature

400 points by drewp 4y ago

est 4y ago
Kinda reminds me of Google Sets.
Quarrelsome 4y ago
"lack of dark mode" should be "features" not "usability"?
- deleted 4y ago
  [deleted]
- jedberg 4y ago
  If anything it's a lack of a usability feature. Sounds like both would be right.
- camtarn 4y ago
  Similarly, "Not sure what to do with the button" is clearly a usability issue, not features. And for the second Kindle review, it summarized one point from the actual review, then completely made up two additional points! Really impressive Sheets extension, but you'd have to be so careful what you applied this to.
  - adrianmonk 4y ago
    > completely made up two additional points I wonder if this means the AI is dumb or that the AI is smart enough to notice that humans just make shit up sometimes, like when they're not reading carefully or when they need filler.
scanr 4y ago
That’s amazing. It does rely on a level of comfort with a fuzzy error budget.
- vntok 4y ago
  About 20% of the generated postcodes are absurdly wrong.
layer8 4y ago
From one of the replies: “This is awesome. I also love how 20% of the zip codes are wrong. Messy AI future seems really fun and chaotic.”
- dylan604 4y ago
  A human developer once told me that bad data is better than no data. <facepalm> So of course a human developer made an AI that makes bad data.
  - a1369209993 4y ago
    FWIW, the actual saying is that for the purposes of collection by enemies (like Facebook and Google or KGB and NSA), the only thing better than no data is bad data.
    - dylan604 4y ago
      FWIW, I was told that well before Facebook was a thought in the Zuck's head. Pre-socials, nobody shared data like is done today, so the NSA had to actually work to get it. Kids these days... /s
- forgotusername6 4y ago
  You need an AI that can understand when not to answer as opposed to some best effort guessing. Some of that input didn't have numbers in the right format so no zip code. The hilarious one is changing the zip code to 90210. The AI basically accusing you of a typo because you obviously meant that more famous zip code. General purpose AIs in situations where more targeted, simpler solutions are needed are going to be incredibly dangerous. Sure this AI can fly a plane 99.999% of the time, but every once in a while it does a nose dive because of reasons we cannot possibly understand or debug.
- ren_engineer 4y ago
  yeah, most of these demos for GPT-3 are that go viral are cherry picked at best
- 4y ago
greenie_beans 4y ago
if you need that address parser, this is a bit more robust and easier to use: https://workspace.google.com/u/0/marketplace/app/parserator_parse_and_split_addresses/945974620840 https://workspace.google.com/u/0/marketplace/app/parserator_...
gpderetta 4y ago
I think that the function should be called DWIM instead. Amazing feature otherwise, we really live in interesting times!
- chime 4y ago
  DWIM: Do What I mean https://en.wikipedia.org/wiki/DWIM https://en.wikipedia.org/wiki/DWIM
ACV001 4y ago
This particular example is an inadeqate application of AI. This is static data which can be looked up in a table (at least zip code).
- krossitalk 4y ago
  What about subtle formatting differences (Country, Territory, Postal code is the norm. Doesn't have to be.). What if we applied this to hand written addresses? (Adding an OCR component). I'm sure the USPS is already doing this and more, and if not, there's probably some AI jobs lined up for it :)
  - lifthrasiir 4y ago
    Yes, USPS has Remote Encoding Centers (REC) to handle handwritten address that can't be recognized with OCR. So AI is already there, just that humans are for harder and tedious jobs ;-)
    - appletrotter 4y ago
      > So AI is already there, just that humans are for harder and tedious jobs ;-) Increasingly less so ;-)
- a1371 4y ago
  This is compared to inadequate application of humans. it is not competing with people who know how to do regex and string parsing. It is for the people who put an office assistant to the task. It is better to inadequately apply AI here as opposed to inadequately apply a human who probably has more fun things to do.
swyx 4y ago
also previously from 2020 https://twitter.com/pavtalk/status/1285410751092416513?s=20&t=ppZhNO_OuQmXkjHQ7dl4wg https://twitter.com/pavtalk/status/1285410751092416513?s=20&...
Imnimo 4y ago
The ability of language models to do zero-shot tasks like this is cool and all, but there is no way you should actually be doing something like this on data you care about. Like think about how much compute is going into trying to autofill a handful of zip codes, and you're still getting a bunch of them wrong.
- mritchie712 4y ago
  yeah, determining the zip for an address is a really bad example. Better one would be "based on these three columns, generate a cold outbound email for the person..." it would suck to be on the receiving end of those, but the use case makes much more sense.
  - cycomachead 4y ago
    I've been wondering about this... It definitely doesn't feel great to be on the receiving end of something auto-generated. But a "unique" message is at least more interesting to read, and doesn't feel quite as frustrating. A yet if P(someone unknown is a robot) gets too large, it's going to be a weird adjustment period.
- layer8 4y ago
  Yeah, I think energy efficiency considerations will become important at some point. Or at least they should.
  - ajmurmann 4y ago
    They only will if we price in the true cost of energy including negative externalities
- dylan604 4y ago
  I've used/added the USPS API into a system and it took practically no time at all to do it. I'm guessing that is significantly less time than building an AI tool. What's worse is that the thing that takes the least time to implement actually provides good data.
skrebbel 4y ago
The amount of 90% sensible, 10% ridiculously wrong computer generated crap we’re about to send into real humans’ brains makes my head spin. There’s truly an awful AI Winter ahead and it consists of spending a substantial amount of your best brain cycles on figuring out whether a real person wrote that thing to you (and it’s worth figuring out what they meant in case of some weird wording) or it was a computer generated fucking thank you note.
- roody15 4y ago
  Think the winter is here
- Domenic_S 4y ago
  Mail merge has existed since the 1980s. https://en.wikipedia.org/wiki/Mail_merge https://en.wikipedia.org/wiki/Mail_merge
- jedberg 4y ago
  It depends on how people use the tools. For example the thank you note one -- if someone just prints off the output of this and sends it, yeah, that's bad. But if someone uses this to do 90% of the work and then just edits it to make it personal and sound like themselves, then it's just a great time saving tool. I mean, in this exact example, 70 years ago you'd have to hand address each thank you card by hand from scratch. 10 years ago you could use a spreadsheet just like this to automatically print off mailing labels from your address list. It didn't make things worse, just different. This is just the next step in automation.
  - agf 4y ago
    > But if someone uses this to do 90% of the work and then just edits it to make it personal and sound like themselves, then it's just a great time saving tool. This is still way too optimistic. Reading through something that's "almost right", seeing the errors when you already basically know what it says / what it's meant to say, and fixing them, is hard. People won't do it well, and so even in this scenario we often end up with something much worse than if it was just written directly. There is a lot of evidence for this, from the generally low quality of lightly-edited speech-to-text material, to how hard it is to look at a bunch of code and find all of the bugs without any extra computer-generated information, to how hard editing text for readability can be without serious restructuring.
Havoc 4y ago
Of all the places spreadsheet is probably the one place you don’t want AI generated content. Half the time it’s financial info so sorta correct simply isn’t good enough
- anon25783 4y ago
  I was going to say: This can only end well for the economy... /s
  - dylan604 4y ago
    Let's see if we can tell what data was used to train the model by watching where the money starts to be moved around into offshore accounts and what not. Was the model trained on the data dumps of from those "off shore" banks recently-ish leaked.
- unnah 4y ago
  Most real-world spreadsheets contained significant errors in this 2005 review: https://www.researchgate.net/publication/228662532_What_We_Know_About_Spreadsheet_Errors https://www.researchgate.net/publication/228662532_What_We_K... Is there any reason to think the situation has substantially improved since then?
  - mrguyorama 4y ago
    This is not a good excuse to actively and knowingly make it worse
- contravariant 4y ago
  When I see something is in a spreadsheet I immediately assume there are at least 3 things wrong with the data, 1 of which is obvious.
- cdrini 4y ago
  Spreadsheets are used for _waaaay_ more than just finances. I don't think it's anywhere near 50% finances. I can't recall where, but I saw a study from I think the 90s saying most of the spreadsheets they found were being used as Todo lists. Maybe like 1 in my past 2y of many, many spreadsheets has been financing related. I think you might be overgeneralizing to an ungeneralizeably large group -- the set of all human spreadsheets.
thatguymike 4y ago
51s -- "We are truly grateful", says the heartfelt thankyou card that was written by an algorithm in a spreadsheet.
- bee_rider 4y ago
  If people want to put this sort of language in a thank-you note, I guess... I dunno, it always comes off as inauthentic to me, so I don't really care if I got mass produced or artisanal hand-crafted lies.
- semi-extrinsic 4y ago
  Oh, this is nothing new. I remember in like 2007 or something, in the early days of Facebook, someone made a CLI interface to the FB API. And I wrote a random-timed daily cron job that ran a Bash script that checked "which of my FB friends have their birthday today", went through that list, selected a random greeting from like 15 different ones I'd put into an array, and posted this to the wall of person $i. Complete with a "blacklist" with names of close friends and family, where the script instead sent me an email reminder to write a manual, genuine post. I used to have a golfed version of that script as my Slashdot signature.
- deleted 4y ago
  [deleted]
- CobrastanJorji 4y ago
  The problem's worse than you know. I've heard that sometimes real humans who tell a lot of people "thank you" aren't even that thankful.
breck 4y ago
If you do a startup for this please email me wire instructions.
chmod775 4y ago
This seems to be doing much worse than existing solutions: Google Maps probably wouldn't have gotten quite as many wrong if you just pasted those addresses into the search bar. However it could be interesting as a last shot if parsing the input failed using any other way. "I tried parsing your messy input. Here's what I came up with. Please make sure it's correct then proceed with the checkout."
- sacrosancty 4y ago
renewiltord 4y ago
This is terrific stuff, honestly. I could see an Airtable integration being really quite useful. There were lots of times when I will run some quick scraping, some cleaning up via an Upworker, and then join against something else. Here volume matters, and all misses are just lost data which I'm fine with. The general purpose nature of the tool makes it tremendous. There was a time when I would have easily paid $0.05 / query for this. The only problem with the spreadsheet setting is that I don't want it to repeatedly execute and charge me so I'll be forced to use `=GPT3()` and then copy-paste "as values" back into the same place which is annoying.
- A4ET8a8uTh0 4y ago
  Was anyone able to test if the Airtable implementation works as well as the twitter 'ad'?
cstross 4y ago
Now wait for =deep_dream() or maybe =stable_diffusion() as a graph-generating function! (Graphs plotted with this function will of course zoom in infinitely but the further you go the more eyes and shiba dogs you'll notice in the corners ...)
- dj_mc_merlin 4y ago
  Sadly current models are bad at plotting graphs with any kind of accuracy. We're still quite far off from getting them to do a properly labeled, colored pie chart with a legend.
  - dplavery92 4y ago
    For what it's worth, I'm also very bad at plotting graphs with any kind of accuracy, which is why I use plotting software instead of doing it by hand. I get the feeling that my visual system and the language I use are respectively pretty bad at processing and conveying precise information from a plot, (beyond simple descriptors like "A is larger than B" or "f(x) has a maximum"). I guess I would find it mildly surprising if any Vision-Language model were able to perform those tasks very well, because the representations in question seem pretty poorly suited. I get that popular diffusion models for image generation are doing a bad job composing concepts in a scene and keeping relationships constant over the image--even if Stable Diffusion could write in human script, it's a bad bet that the contents of a legend would match a pie chart that it drew. But other Vision-Language models, designed for image captioning or visual question answering, rather than generating diverse, stylistic images, are pretty good at that compositional information (up to, again, the "simple descriptions" level of granularity I mentioned before.)
- bee_rider 4y ago
  Plots that devolve into madness as if you zoom in too close? Finally the programmers get to experience electronics class.
mike256 4y ago
Do I understand that correctly? When I have to create a spreadsheet like this, there are 2 options. Option 1 I write a table zipcode to state and use this table to generate my column. If I carefully check my table my spreadsheet would be okay. Option 2 I ask GPT3 to do my work. But I have to check the whole spreadsheet for errors.
- deleted 4y ago
  [deleted]
- giarc 4y ago
  I dealt with something similar. I was creating a large directory of childcare centres in Canada. I had thousands of listings with a url but no email address. I created a Mechanical Turk job to ask turkers to go to website and find an email address. Many came back with email addresses like admin@<<actualURL>>.com. After checking a few, I realized that the turkers were just guessing that admin@ would work and I'd approve their work. I ended up having to double check all the work.
  - xvector 4y ago
    I wonder if those workers can be reported and fired.
    - csunbird 4y ago
      I mean, depending on how the OP phrased the work to be done, they probably did valid work.
    - giarc 4y ago
      It was quite a while ago so I don't remember what my options were. If I recall correctly I was able to see individual workers responses, so once I found one that was obviously faking results, I was able to reject all their submissions.
  - visarga 4y ago
    > I ended up having to double check all the work. Me too, different project and different labelling company but the conclusion - it's better to do it in house. Labelling is hard. You need to see, talk with and train your labelling team.
armchairhacker 4y ago
I said it before: we need Copilot flash fill. Infer what the user wants the output to be from patterns and labels, so they can enter a few examples and then “extend” and automatically do the equivalent of a complex formula. e.g. Formal | Informal Lane, Thomas | Tommy Lane Brooks, Sarah | Sarah Brooks Yun, Christopher | Doe, Kaitlyn | Styles, Chris | … Automating something like this is extremely hard with an algorithm and extremely easy with ML. Even better, many people who use spreadsheets aren’t very familiar with coding and software, so they do things manually even in cases where the formula is simple.
- dwringer 4y ago
  I posed this exact question to character.ai's "Ask Me Anything" bot. It decided to redo the examples, too. The results: > Lane, Thomas => Thomas Layne > Brooks, Sarah => Sarah Brooksy > Yun, Christopher => Chris Yun > Doe, Kaitlyn => KD > Styles, Chris => Chris Spice, Chris Chasm I'm sure the bot overcomplicated an otherwise simple task, but I think there's always gonna be some creative error if we rely on things like that. It's funny though because these results are plausible for what a real person might come up with as informal nicknames for their friends.
  - swyx 4y ago
    try to tweak the prompts so you get to the level of Katie Dee, KDizzle, Kdawg
    - dwringer 4y ago
      I ... sort of got there. But then it rapidly escalated to extremely long and incoherent chains of hyphenated names with "money" and "$" and things all throughout.
      - swyx 4y ago
        i mean thats pretty realistic
pbmango 4y ago
There is a huge potential for language models to get close to messy text problems (many of which are in Excel and Sheet). I am the founder of promptloop.com - the author of this tweet has been an early user. The challenge to making something like this, or Co-pilot / Ghostwrite, work well is about meeting users where they are. Spreadsheet users dont want to deal with API keys or know what temperature is - but anyone (like this tweet) can set up direct API use with generic models in 10 minutes. This document has all the code to do so ;). [1] For non-engineers - or folks who need a reliable and familiar syntax to use at scale and across their org - promptloop [2] is the best way to do that. All comments in here are great though. We have been live with users since the summer - no waitlist. And as a note - despite the name "prompt engineering" has almost nothing to do with making this work at scale. [1] https://docs.google.com/spreadsheets/d/1lSpiz2dIswCXGIQfE69dtH_VgdOFxjEBOK1WpT5uh8k/template/preview https://docs.google.com/spreadsheets/d/1lSpiz2dIswCXGIQfE69d... [2] https://www.promptloop.com/ https://www.promptloop.com/
- geoduck14 4y ago
  How much does it cost to run?
- tomcam 4y ago
  > The author of this tweet has been an early user Maybe not good to reveal customer names this way, unless they already disclosed it publicly
- elil17 4y ago
  Any plans to bring this to Excel? I would love to recommend this to folks at my company but we aren't allowed to use G-Suite.
  - pbmango 4y ago
    Yes - Not open access yet but drop me an email!
tonmoy 4y ago
The tasks on the first sheet is easily accomplished by flash fill in MS Excel and I suspect less prone to error. Not sure why flash fill is not more popular
- baxtr 4y ago
  What is flash fill? I have worked a lot on Excel sheets and still haven’t heard anything about it.
  - brassattax 4y ago
    ctrl+e
  - deleted 4y ago
    [deleted]
  - localhost 4y ago
    Flash fill is an implementation of program synthesis, a technique invented by Sumit Gulwani formerly of Microsoft Research. Here's a paper that explains more about how it works [1]. It's not a very discoverable feature of Excel though [2] [1] https://arxiv.org/pdf/1703.03539.pdf https://arxiv.org/pdf/1703.03539.pdf [2] https://support.microsoft.com/en-us/office/using-flash-fill-in-excel-3f9bcf1e-db93-4890-94a0-1578341f73f7 https://support.microsoft.com/en-us/office/using-flash-fill-...
gbro3n 4y ago
The most sensible use for AI that I can see at this time is for supporting humans in their work, but only where the system is set up so that the human has to do the work first, with the AI system looking for possible errors. For example the human drives the car, and the AI brakes when it senses dangerous conditions ahead, or the human screens test results for evidence of cancer and the AI flags where it disagrees so that the human might take another look. The opposite scenario with AI doing the work and humans checking for errors as is the case here will lead to humans being over reliant on less than perfect systems and producing outcomes with high rates of error. As AI improves and gains trust in a field, it can then replace the human. But this trust has to come from evidence of AI superiority over the long term, not from companies over-selling the reliability of their AI.
- leereeves 4y ago
  I mostly agree. Happily, that's also what people will want as long as human participation is necessary. We'd generally prefer to write rather than correct an AI's writing, and prefer to drive rather than carefully watch an AI drive. But when the AI is capable of something the person can't do (like Stable Diffusion creating images compared to me) the AI should take first chair.
  - gbro3n 4y ago
    This is a good point. Where less than perfect AI is better than the alternative, it is useful.
- uh_uh 4y ago
  Humans are also less than perfect systems. Especially if they have to deal with monotonous tasks. A human might perform better on a 100 entries than an AI, but on 10 thousand? Of course you can distribute the workload, but you will balloon the costs (I'm talking about a future where GPT3 costs come down). There must be a set of projects which are cost prohibited now due to having to pay humans but will become feasible exactly because of this tech. For a good portion of these, higher-than-human error rate will also be tolerable or at least correctable via a small degree of human intervention.
wesleyyue 4y ago
Also check out https://usedouble.com https://usedouble.com (YC W23) if you're interested in using something like this today. Note: I'm the founder :) Happy to answer any questions. Reply below with some sample data/problem and I'll reply with a demo to see if we can solve it out of the box!
- trialskid86 4y ago
  Just signed up. How long is the wait list?
  - conductr 4y ago
    > Get this AI tool for FREE. Next waitlist closes in: > 0 day 7 hour 31 min 42 sec I've never seen rolling waitlists, it's kind of strange tbh
    - irthomasthomas 4y ago
      It's the modern equivalent of creating a long queue outside your Apple store to generate buzz.
      - conductr 4y ago
        It sounds more like a series of short queues. So short, that Apple employees are sitting around twiddling their thumbs until the next queue gets long enough that they decide to start working. If the app isn’t growing faster than infrastructure can be provisioned, these mini queues are just making people wait for no reason.
  - wesleyyue 4y ago
    It's about 3 weeks right now. Mostly limited by me trying to make sure each person onboarded gets a high touch experience from me since the product is still rough around the edges. If you need access to solve an urgent problem, email me at founder@usedouble.com and I can get you in right away if you don't mind the early unpolished nature of the product.
orblivion 4y ago
Could we hook GPT3 up to our dating apps? On both sides. That way we can just let the computers do the small talk and if they hit it off we can meet.
- CPLX 4y ago
  Google: Black Mirror Hang The DJ
  - hairofadog 4y ago
    Such an excellent episode! A master class in what it means for actors to have chemistry.
planetsprite 4y ago
GPT3 charges for every token read/written. What may be more useful is using GPT-3 not to manually run itself on every row, but to take the task and generate a sufficient function that fulfills the task.
- miohtama 4y ago
  Like with Stable Diffusion, maybe there will be an open model for the language prompts which less or no restrictions in the near future.
  - CrypticShift 4y ago
    Amen.
  - persedes 4y ago
    Gpt-j?
- visarga 4y ago
  Code as Policies: - Language Model Programs for Embodied Control generates python, then executes https://code-as-policies.github.io/ https://code-as-policies.github.io/
ninefathom 4y ago
Fascinating and terrifying all at once. Queue Fry "I'm scare-roused" meme...
mensetmanusman 4y ago
Once it becomes an excel function, then it will be spun off into a start up.
29athrowaway 4y ago
GPT3 is not deterministic though.
appleflaxen 4y ago
This is so freaking cool. Brilliant idea.
wstuartcl 4y ago
I do not know what's worse. Missing data you need to populate, or having an AI populate data that LOOKS right. ugh I do know what is worse.
magic_hamster 4y ago
As a casual Google user, how do you start developing something like this? Does Google Sheets allow loading custom plugins?
DeathArrow 4y ago
I would love to see a tool which uses GPT-3 to generate SQL from English. Like: give me a list of all customers from London who purchased in January a laptop with more than 16GB of RAM and used a coupon between 10% and 25%. Sort it by price payd.
- sandkoan 4y ago
  Everything required to make the tool already exists. Just ran your exact query through OpenAI's Codex (model: code-davinci-002), and this was the result: SELECT * FROM customers WHERE city = 'London' AND purchase_date BETWEEN '2019-01-01' AND '2019-01-31' AND product_name = 'laptop' AND product_ram > 16 AND coupon_percentage BETWEEN 10 AND 25 ORDER BY price_paid DESC; I'd say it's pretty damn accurate.

400 points by drewp 4y ago

est 4y ago

Kinda reminds me of Google Sets.

Quarrelsome 4y ago

"lack of dark mode" should be "features" not "usability"?

deleted 4y ago
[deleted]
jedberg 4y ago
If anything it's a lack of a usability feature. Sounds like both would be right.
camtarn 4y ago
Similarly, "Not sure what to do with the button" is clearly a usability issue, not features. And for the second Kindle review, it summarized one point from the actual review, then completely made up two additional points! Really impressive Sheets extension, but you'd have to be so careful what you applied this to.
- adrianmonk 4y ago
  > completely made up two additional points I wonder if this means the AI is dumb or that the AI is smart enough to notice that humans just make shit up sometimes, like when they're not reading carefully or when they need filler.

scanr 4y ago

That’s amazing. It does rely on a level of comfort with a fuzzy error budget.

vntok 4y ago
About 20% of the generated postcodes are absurdly wrong.

layer8 4y ago

From one of the replies: “This is awesome. I also love how 20% of the zip codes are wrong. Messy AI future seems really fun and chaotic.”

dylan604 4y ago
A human developer once told me that bad data is better than no data. <facepalm> So of course a human developer made an AI that makes bad data.
- a1369209993 4y ago
  FWIW, the actual saying is that for the purposes of collection by enemies (like Facebook and Google or KGB and NSA), the only thing better than no data is bad data.
  - dylan604 4y ago
    FWIW, I was told that well before Facebook was a thought in the Zuck's head. Pre-socials, nobody shared data like is done today, so the NSA had to actually work to get it. Kids these days... /s
forgotusername6 4y ago
You need an AI that can understand when not to answer as opposed to some best effort guessing. Some of that input didn't have numbers in the right format so no zip code. The hilarious one is changing the zip code to 90210. The AI basically accusing you of a typo because you obviously meant that more famous zip code. General purpose AIs in situations where more targeted, simpler solutions are needed are going to be incredibly dangerous. Sure this AI can fly a plane 99.999% of the time, but every once in a while it does a nose dive because of reasons we cannot possibly understand or debug.
ren_engineer 4y ago
yeah, most of these demos for GPT-3 are that go viral are cherry picked at best
4y ago

greenie_beans 4y ago

if you need that address parser, this is a bit more robust and easier to use: https://workspace.google.com/u/0/marketplace/app/parserator_parse_and_split_addresses/945974620840 https://workspace.google.com/u/0/marketplace/app/parserator_...

gpderetta 4y ago

I think that the function should be called DWIM instead. Amazing feature otherwise, we really live in interesting times!

chime 4y ago
DWIM: Do What I mean https://en.wikipedia.org/wiki/DWIM https://en.wikipedia.org/wiki/DWIM

ACV001 4y ago

This particular example is an inadeqate application of AI. This is static data which can be looked up in a table (at least zip code).

krossitalk 4y ago
What about subtle formatting differences (Country, Territory, Postal code is the norm. Doesn't have to be.). What if we applied this to hand written addresses? (Adding an OCR component). I'm sure the USPS is already doing this and more, and if not, there's probably some AI jobs lined up for it :)
- lifthrasiir 4y ago
  Yes, USPS has Remote Encoding Centers (REC) to handle handwritten address that can't be recognized with OCR. So AI is already there, just that humans are for harder and tedious jobs ;-)
  - appletrotter 4y ago
    > So AI is already there, just that humans are for harder and tedious jobs ;-) Increasingly less so ;-)
a1371 4y ago
This is compared to inadequate application of humans. it is not competing with people who know how to do regex and string parsing. It is for the people who put an office assistant to the task. It is better to inadequately apply AI here as opposed to inadequately apply a human who probably has more fun things to do.

swyx 4y ago

also previously from 2020 https://twitter.com/pavtalk/status/1285410751092416513?s=20&t=ppZhNO_OuQmXkjHQ7dl4wg https://twitter.com/pavtalk/status/1285410751092416513?s=20&...

Imnimo 4y ago

The ability of language models to do zero-shot tasks like this is cool and all, but there is no way you should actually be doing something like this on data you care about. Like think about how much compute is going into trying to autofill a handful of zip codes, and you're still getting a bunch of them wrong.

mritchie712 4y ago
yeah, determining the zip for an address is a really bad example. Better one would be "based on these three columns, generate a cold outbound email for the person..." it would suck to be on the receiving end of those, but the use case makes much more sense.
- cycomachead 4y ago
  I've been wondering about this... It definitely doesn't feel great to be on the receiving end of something auto-generated. But a "unique" message is at least more interesting to read, and doesn't feel quite as frustrating. A yet if P(someone unknown is a robot) gets too large, it's going to be a weird adjustment period.
layer8 4y ago
Yeah, I think energy efficiency considerations will become important at some point. Or at least they should.
- ajmurmann 4y ago
  They only will if we price in the true cost of energy including negative externalities
dylan604 4y ago
I've used/added the USPS API into a system and it took practically no time at all to do it. I'm guessing that is significantly less time than building an AI tool. What's worse is that the thing that takes the least time to implement actually provides good data.

skrebbel 4y ago

The amount of 90% sensible, 10% ridiculously wrong computer generated crap we’re about to send into real humans’ brains makes my head spin. There’s truly an awful AI Winter ahead and it consists of spending a substantial amount of your best brain cycles on figuring out whether a real person wrote that thing to you (and it’s worth figuring out what they meant in case of some weird wording) or it was a computer generated fucking thank you note.

roody15 4y ago
Think the winter is here
Domenic_S 4y ago
Mail merge has existed since the 1980s. https://en.wikipedia.org/wiki/Mail_merge https://en.wikipedia.org/wiki/Mail_merge
jedberg 4y ago
It depends on how people use the tools. For example the thank you note one -- if someone just prints off the output of this and sends it, yeah, that's bad. But if someone uses this to do 90% of the work and then just edits it to make it personal and sound like themselves, then it's just a great time saving tool. I mean, in this exact example, 70 years ago you'd have to hand address each thank you card by hand from scratch. 10 years ago you could use a spreadsheet just like this to automatically print off mailing labels from your address list. It didn't make things worse, just different. This is just the next step in automation.
- agf 4y ago
  > But if someone uses this to do 90% of the work and then just edits it to make it personal and sound like themselves, then it's just a great time saving tool. This is still way too optimistic. Reading through something that's "almost right", seeing the errors when you already basically know what it says / what it's meant to say, and fixing them, is hard. People won't do it well, and so even in this scenario we often end up with something much worse than if it was just written directly. There is a lot of evidence for this, from the generally low quality of lightly-edited speech-to-text material, to how hard it is to look at a bunch of code and find all of the bugs without any extra computer-generated information, to how hard editing text for readability can be without serious restructuring.

Havoc 4y ago

Of all the places spreadsheet is probably the one place you don’t want AI generated content. Half the time it’s financial info so sorta correct simply isn’t good enough

anon25783 4y ago
I was going to say: This can only end well for the economy... /s
- dylan604 4y ago
  Let's see if we can tell what data was used to train the model by watching where the money starts to be moved around into offshore accounts and what not. Was the model trained on the data dumps of from those "off shore" banks recently-ish leaked.
unnah 4y ago
Most real-world spreadsheets contained significant errors in this 2005 review: https://www.researchgate.net/publication/228662532_What_We_Know_About_Spreadsheet_Errors https://www.researchgate.net/publication/228662532_What_We_K... Is there any reason to think the situation has substantially improved since then?
- mrguyorama 4y ago
  This is not a good excuse to actively and knowingly make it worse
contravariant 4y ago
When I see something is in a spreadsheet I immediately assume there are at least 3 things wrong with the data, 1 of which is obvious.
cdrini 4y ago
Spreadsheets are used for _waaaay_ more than just finances. I don't think it's anywhere near 50% finances. I can't recall where, but I saw a study from I think the 90s saying most of the spreadsheets they found were being used as Todo lists. Maybe like 1 in my past 2y of many, many spreadsheets has been financing related. I think you might be overgeneralizing to an ungeneralizeably large group -- the set of all human spreadsheets.

thatguymike 4y ago

51s -- "We are truly grateful", says the heartfelt thankyou card that was written by an algorithm in a spreadsheet.

bee_rider 4y ago
If people want to put this sort of language in a thank-you note, I guess... I dunno, it always comes off as inauthentic to me, so I don't really care if I got mass produced or artisanal hand-crafted lies.
semi-extrinsic 4y ago
Oh, this is nothing new. I remember in like 2007 or something, in the early days of Facebook, someone made a CLI interface to the FB API. And I wrote a random-timed daily cron job that ran a Bash script that checked "which of my FB friends have their birthday today", went through that list, selected a random greeting from like 15 different ones I'd put into an array, and posted this to the wall of person $i. Complete with a "blacklist" with names of close friends and family, where the script instead sent me an email reminder to write a manual, genuine post. I used to have a golfed version of that script as my Slashdot signature.
deleted 4y ago
[deleted]
CobrastanJorji 4y ago
The problem's worse than you know. I've heard that sometimes real humans who tell a lot of people "thank you" aren't even that thankful.

breck 4y ago

If you do a startup for this please email me wire instructions.

chmod775 4y ago

This seems to be doing much worse than existing solutions: Google Maps probably wouldn't have gotten quite as many wrong if you just pasted those addresses into the search bar. However it could be interesting as a last shot if parsing the input failed using any other way. "I tried parsing your messy input. Here's what I came up with. Please make sure it's correct then proceed with the checkout."

sacrosancty 4y ago

renewiltord 4y ago

This is terrific stuff, honestly. I could see an Airtable integration being really quite useful. There were lots of times when I will run some quick scraping, some cleaning up via an Upworker, and then join against something else. Here volume matters, and all misses are just lost data which I'm fine with. The general purpose nature of the tool makes it tremendous. There was a time when I would have easily paid $0.05 / query for this. The only problem with the spreadsheet setting is that I don't want it to repeatedly execute and charge me so I'll be forced to use `=GPT3()` and then copy-paste "as values" back into the same place which is annoying.

A4ET8a8uTh0 4y ago
Was anyone able to test if the Airtable implementation works as well as the twitter 'ad'?

cstross 4y ago

Now wait for =deep_dream() or maybe =stable_diffusion() as a graph-generating function! (Graphs plotted with this function will of course zoom in infinitely but the further you go the more eyes and shiba dogs you'll notice in the corners ...)

dj_mc_merlin 4y ago
Sadly current models are bad at plotting graphs with any kind of accuracy. We're still quite far off from getting them to do a properly labeled, colored pie chart with a legend.
- dplavery92 4y ago
  For what it's worth, I'm also very bad at plotting graphs with any kind of accuracy, which is why I use plotting software instead of doing it by hand. I get the feeling that my visual system and the language I use are respectively pretty bad at processing and conveying precise information from a plot, (beyond simple descriptors like "A is larger than B" or "f(x) has a maximum"). I guess I would find it mildly surprising if any Vision-Language model were able to perform those tasks very well, because the representations in question seem pretty poorly suited. I get that popular diffusion models for image generation are doing a bad job composing concepts in a scene and keeping relationships constant over the image--even if Stable Diffusion could write in human script, it's a bad bet that the contents of a legend would match a pie chart that it drew. But other Vision-Language models, designed for image captioning or visual question answering, rather than generating diverse, stylistic images, are pretty good at that compositional information (up to, again, the "simple descriptions" level of granularity I mentioned before.)
bee_rider 4y ago
Plots that devolve into madness as if you zoom in too close? Finally the programmers get to experience electronics class.

mike256 4y ago

Do I understand that correctly? When I have to create a spreadsheet like this, there are 2 options. Option 1 I write a table zipcode to state and use this table to generate my column. If I carefully check my table my spreadsheet would be okay. Option 2 I ask GPT3 to do my work. But I have to check the whole spreadsheet for errors.

deleted 4y ago
[deleted]
giarc 4y ago
I dealt with something similar. I was creating a large directory of childcare centres in Canada. I had thousands of listings with a url but no email address. I created a Mechanical Turk job to ask turkers to go to website and find an email address. Many came back with email addresses like admin@<<actualURL>>.com. After checking a few, I realized that the turkers were just guessing that admin@ would work and I'd approve their work. I ended up having to double check all the work.
- xvector 4y ago
  I wonder if those workers can be reported and fired.
  - csunbird 4y ago
    I mean, depending on how the OP phrased the work to be done, they probably did valid work.
  - giarc 4y ago
    It was quite a while ago so I don't remember what my options were. If I recall correctly I was able to see individual workers responses, so once I found one that was obviously faking results, I was able to reject all their submissions.
- visarga 4y ago
  > I ended up having to double check all the work. Me too, different project and different labelling company but the conclusion - it's better to do it in house. Labelling is hard. You need to see, talk with and train your labelling team.

armchairhacker 4y ago

I said it before: we need Copilot flash fill. Infer what the user wants the output to be from patterns and labels, so they can enter a few examples and then “extend” and automatically do the equivalent of a complex formula. e.g. Formal | Informal Lane, Thomas | Tommy Lane Brooks, Sarah | Sarah Brooks Yun, Christopher | Doe, Kaitlyn | Styles, Chris | … Automating something like this is extremely hard with an algorithm and extremely easy with ML. Even better, many people who use spreadsheets aren’t very familiar with coding and software, so they do things manually even in cases where the formula is simple.

dwringer 4y ago
I posed this exact question to character.ai's "Ask Me Anything" bot. It decided to redo the examples, too. The results: > Lane, Thomas => Thomas Layne > Brooks, Sarah => Sarah Brooksy > Yun, Christopher => Chris Yun > Doe, Kaitlyn => KD > Styles, Chris => Chris Spice, Chris Chasm I'm sure the bot overcomplicated an otherwise simple task, but I think there's always gonna be some creative error if we rely on things like that. It's funny though because these results are plausible for what a real person might come up with as informal nicknames for their friends.
- swyx 4y ago
  try to tweak the prompts so you get to the level of Katie Dee, KDizzle, Kdawg
  - dwringer 4y ago
    I ... sort of got there. But then it rapidly escalated to extremely long and incoherent chains of hyphenated names with "money" and "$" and things all throughout.
    - swyx 4y ago
      i mean thats pretty realistic

pbmango 4y ago

There is a huge potential for language models to get close to messy text problems (many of which are in Excel and Sheet). I am the founder of promptloop.com - the author of this tweet has been an early user. The challenge to making something like this, or Co-pilot / Ghostwrite, work well is about meeting users where they are. Spreadsheet users dont want to deal with API keys or know what temperature is - but anyone (like this tweet) can set up direct API use with generic models in 10 minutes. This document has all the code to do so ;). [1] For non-engineers - or folks who need a reliable and familiar syntax to use at scale and across their org - promptloop [2] is the best way to do that. All comments in here are great though. We have been live with users since the summer - no waitlist. And as a note - despite the name "prompt engineering" has almost nothing to do with making this work at scale. [1] https://docs.google.com/spreadsheets/d/1lSpiz2dIswCXGIQfE69dtH_VgdOFxjEBOK1WpT5uh8k/template/preview https://docs.google.com/spreadsheets/d/1lSpiz2dIswCXGIQfE69d... [2] https://www.promptloop.com/ https://www.promptloop.com/

geoduck14 4y ago
How much does it cost to run?
tomcam 4y ago
> The author of this tweet has been an early user Maybe not good to reveal customer names this way, unless they already disclosed it publicly
elil17 4y ago
Any plans to bring this to Excel? I would love to recommend this to folks at my company but we aren't allowed to use G-Suite.
- pbmango 4y ago
  Yes - Not open access yet but drop me an email!

tonmoy 4y ago

The tasks on the first sheet is easily accomplished by flash fill in MS Excel and I suspect less prone to error. Not sure why flash fill is not more popular

baxtr 4y ago
What is flash fill? I have worked a lot on Excel sheets and still haven’t heard anything about it.
- brassattax 4y ago
  ctrl+e
- deleted 4y ago
  [deleted]
- localhost 4y ago
  Flash fill is an implementation of program synthesis, a technique invented by Sumit Gulwani formerly of Microsoft Research. Here's a paper that explains more about how it works [1]. It's not a very discoverable feature of Excel though [2] [1] https://arxiv.org/pdf/1703.03539.pdf https://arxiv.org/pdf/1703.03539.pdf [2] https://support.microsoft.com/en-us/office/using-flash-fill-in-excel-3f9bcf1e-db93-4890-94a0-1578341f73f7 https://support.microsoft.com/en-us/office/using-flash-fill-...

gbro3n 4y ago

The most sensible use for AI that I can see at this time is for supporting humans in their work, but only where the system is set up so that the human has to do the work first, with the AI system looking for possible errors. For example the human drives the car, and the AI brakes when it senses dangerous conditions ahead, or the human screens test results for evidence of cancer and the AI flags where it disagrees so that the human might take another look. The opposite scenario with AI doing the work and humans checking for errors as is the case here will lead to humans being over reliant on less than perfect systems and producing outcomes with high rates of error. As AI improves and gains trust in a field, it can then replace the human. But this trust has to come from evidence of AI superiority over the long term, not from companies over-selling the reliability of their AI.

leereeves 4y ago
I mostly agree. Happily, that's also what people will want as long as human participation is necessary. We'd generally prefer to write rather than correct an AI's writing, and prefer to drive rather than carefully watch an AI drive. But when the AI is capable of something the person can't do (like Stable Diffusion creating images compared to me) the AI should take first chair.
- gbro3n 4y ago
  This is a good point. Where less than perfect AI is better than the alternative, it is useful.
uh_uh 4y ago
Humans are also less than perfect systems. Especially if they have to deal with monotonous tasks. A human might perform better on a 100 entries than an AI, but on 10 thousand? Of course you can distribute the workload, but you will balloon the costs (I'm talking about a future where GPT3 costs come down). There must be a set of projects which are cost prohibited now due to having to pay humans but will become feasible exactly because of this tech. For a good portion of these, higher-than-human error rate will also be tolerable or at least correctable via a small degree of human intervention.

wesleyyue 4y ago

Also check out https://usedouble.com https://usedouble.com (YC W23) if you're interested in using something like this today. Note: I'm the founder :) Happy to answer any questions. Reply below with some sample data/problem and I'll reply with a demo to see if we can solve it out of the box!

trialskid86 4y ago
Just signed up. How long is the wait list?
- conductr 4y ago
  > Get this AI tool for FREE. Next waitlist closes in: > 0 day 7 hour 31 min 42 sec I've never seen rolling waitlists, it's kind of strange tbh
  - irthomasthomas 4y ago
    It's the modern equivalent of creating a long queue outside your Apple store to generate buzz.
    - conductr 4y ago
      It sounds more like a series of short queues. So short, that Apple employees are sitting around twiddling their thumbs until the next queue gets long enough that they decide to start working. If the app isn’t growing faster than infrastructure can be provisioned, these mini queues are just making people wait for no reason.
- wesleyyue 4y ago
  It's about 3 weeks right now. Mostly limited by me trying to make sure each person onboarded gets a high touch experience from me since the product is still rough around the edges. If you need access to solve an urgent problem, email me at founder@usedouble.com and I can get you in right away if you don't mind the early unpolished nature of the product.

orblivion 4y ago

Could we hook GPT3 up to our dating apps? On both sides. That way we can just let the computers do the small talk and if they hit it off we can meet.

CPLX 4y ago
Google: Black Mirror Hang The DJ
- hairofadog 4y ago
  Such an excellent episode! A master class in what it means for actors to have chemistry.

planetsprite 4y ago

GPT3 charges for every token read/written. What may be more useful is using GPT-3 not to manually run itself on every row, but to take the task and generate a sufficient function that fulfills the task.

miohtama 4y ago
Like with Stable Diffusion, maybe there will be an open model for the language prompts which less or no restrictions in the near future.
- CrypticShift 4y ago
  Amen.
- persedes 4y ago
  Gpt-j?
visarga 4y ago
Code as Policies: - Language Model Programs for Embodied Control generates python, then executes https://code-as-policies.github.io/ https://code-as-policies.github.io/

ninefathom 4y ago

Fascinating and terrifying all at once. Queue Fry "I'm scare-roused" meme...

mensetmanusman 4y ago

Once it becomes an excel function, then it will be spun off into a start up.

29athrowaway 4y ago

GPT3 is not deterministic though.

appleflaxen 4y ago

This is so freaking cool. Brilliant idea.

wstuartcl 4y ago

I do not know what's worse. Missing data you need to populate, or having an AI populate data that LOOKS right. ugh I do know what is worse.

magic_hamster 4y ago

As a casual Google user, how do you start developing something like this? Does Google Sheets allow loading custom plugins?

DeathArrow 4y ago

I would love to see a tool which uses GPT-3 to generate SQL from English. Like: give me a list of all customers from London who purchased in January a laptop with more than 16GB of RAM and used a coupon between 10% and 25%. Sort it by price payd.

sandkoan 4y ago
Everything required to make the tool already exists. Just ran your exact query through OpenAI's Codex (model: code-davinci-002), and this was the result: SELECT * FROM customers WHERE city = 'London' AND purchase_date BETWEEN '2019-01-01' AND '2019-01-31' AND product_name = 'laptop' AND product_ram > 16 AND coupon_percentage BETWEEN 10 AND 25 ORDER BY price_paid DESC; I'd say it's pretty damn accurate.