5 ms·

I haven't really seen evidence that any ai can reliably draw a pelican riding a bicycle. Not if you look at the image long enough to take it in. Even the best o

I haven't really seen evidence that any ai can reliably draw a pelican riding a bicycle. Not if you look at the image long enough to take it in. Even the best ones have something wrong with them. Not a matter of taste but a matter of having both legs peddling on the viewer's side of the bicycle or having two beaks. I'm actually beginning to wonder if some people who ignore these things have a different, somewhat lesser ability to percieve image details than I do. I mean I guess its fine to go on to another test despite never actually passing the pelican bike test, but there's a sense that we have to use another test because AI is now good at pelicans on bikes, which is just not true.

enos_feedler 2mo ago
AI has deeply changed the way I think, feel and act around a computer. In the same way that dialing into the internet changed things for me. Since using ChatGPT the first time until now I have never cared once to look at these pelicans on bikes people seem to get hung up about. It could never have been a thing and nothing would change. See the forest through the trees.
- Demiurge 2mo ago
  What you’re saying is that you’re not interested in benchmarks. But then you go a step further and state that this particular benchmark is entirely inconsequential. That’s like telling you that if you didn’t exist, nothing would change. Even if that were true, it would still be an insensitive and rude thing to say, wouldn’t it?
  - Buttons840 2mo ago
    It's okay to be rude to benchmarks though, they don't have feelings.
    - Demiurge 2mo ago
      Yeah, I agree, I didn’t mean to impose on this conversation between a man and a benchmark, my bad :p
  - enos_feedler 1mo ago
    Usually when I say insensitive things there are more downvotes then upvotes. That isn't the case here. I might be rubbing against a truth somewhere here.
w4yai 2mo ago
> I haven't really seen evidence that any ai can reliably draw a pelican riding a bicycle Please remember, we've started from there : https://simonwillison.net/2024/Oct/25/pelicans-on-a-bicycle/ https://simonwillison.net/2024/Oct/25/pelicans-on-a-bicycle/ When it started, it was clear what LLM would stand out, its style, etc. Nowadays, the pelicans look similar, the difference is in details and sometimes hard to catch. Sure, the task is not completed perfectly, but that's not the point. It was supposed to be a benchmark to quickly benchmark a LLM against others.
- reaperducer 2mo ago
  Sure, the task is not completed perfectly, but that's not the point. Isn't it? If the computer can't do it better than a human being, then what's the point? Being wrong at scale is not better than being right.
  - matthewfcarlson 2mo ago
    Many humans would struggle with this even with very good tooling (ie not writing raw svg and using illustrator). I struggle to draw a bicycle accurately. But yes, I suspect it will be diminishing returns and I doubt it will ever be perfect due to the average nature of AI but I’d like to be wrong.
    - skydhash 2mo ago
      > Many humans would struggle with this even with very good tooling But no ones hire random humans for things like this. You go and hire a vector artist and they will get your a very good pelican on a a bike. That's how you get things done when you can't do it.
      - w4yai 1mo ago
        > You go and hire a vector artist Yeah, but then, you recruit the artist for $XXX - whereas you "recruit" your LLM for $0.XXX for the same task. Of course the quality difference is huge. But sometimes you don't need that level of quality. Also, finding a vector artist takes days of communication, payment settlement, revisions, etc. Not always the most practical solution.
paulddraper 2mo ago
Pelicans don't ride bicycles. It's physically impossible. The problem is to draw it in the least disturbing way possible.
- Morromist 2mo ago
  True! But somehow Disney has been drawing ducks riding bikes in a way that seems to satisfy everyone since before my grandfather was born. https://ridesabike.com/donald-duck-daisy-duck-huey-dewey-and-louie-ride-a-bike/ https://ridesabike.com/donald-duck-daisy-duck-huey-dewey-and...
  - monk_grilla 2mo ago
    Wow, that was a highly relevant and specific website to source here!
    - Morromist 2mo ago
      I love these little websites with amazingly focused content. <3
  - jodrellblank 1mo ago
    It’s better than many of the AI offerings and the bike could steer and the ducks are sitting on saddles, but the three nephews can’t reach the bottom of the pedal stroke and by the looks of their feet on the far side of the bike they aren’t trying. That shouldn’t satisfy Donald and Daisy, leaving them with all the work.

enos_feedler 2mo ago

AI has deeply changed the way I think, feel and act around a computer. In the same way that dialing into the internet changed things for me. Since using ChatGPT the first time until now I have never cared once to look at these pelicans on bikes people seem to get hung up about. It could never have been a thing and nothing would change. See the forest through the trees.

Demiurge 2mo ago
What you’re saying is that you’re not interested in benchmarks. But then you go a step further and state that this particular benchmark is entirely inconsequential. That’s like telling you that if you didn’t exist, nothing would change. Even if that were true, it would still be an insensitive and rude thing to say, wouldn’t it?
- Buttons840 2mo ago
  It's okay to be rude to benchmarks though, they don't have feelings.
  - Demiurge 2mo ago
    Yeah, I agree, I didn’t mean to impose on this conversation between a man and a benchmark, my bad :p
- enos_feedler 1mo ago
  Usually when I say insensitive things there are more downvotes then upvotes. That isn't the case here. I might be rubbing against a truth somewhere here.

w4yai 2mo ago

> I haven't really seen evidence that any ai can reliably draw a pelican riding a bicycle Please remember, we've started from there : https://simonwillison.net/2024/Oct/25/pelicans-on-a-bicycle/ https://simonwillison.net/2024/Oct/25/pelicans-on-a-bicycle/ When it started, it was clear what LLM would stand out, its style, etc. Nowadays, the pelicans look similar, the difference is in details and sometimes hard to catch. Sure, the task is not completed perfectly, but that's not the point. It was supposed to be a benchmark to quickly benchmark a LLM against others.

reaperducer 2mo ago
Sure, the task is not completed perfectly, but that's not the point. Isn't it? If the computer can't do it better than a human being, then what's the point? Being wrong at scale is not better than being right.
- matthewfcarlson 2mo ago
  Many humans would struggle with this even with very good tooling (ie not writing raw svg and using illustrator). I struggle to draw a bicycle accurately. But yes, I suspect it will be diminishing returns and I doubt it will ever be perfect due to the average nature of AI but I’d like to be wrong.
  - skydhash 2mo ago
    > Many humans would struggle with this even with very good tooling But no ones hire random humans for things like this. You go and hire a vector artist and they will get your a very good pelican on a a bike. That's how you get things done when you can't do it.
    - w4yai 1mo ago
      > You go and hire a vector artist Yeah, but then, you recruit the artist for $XXX - whereas you "recruit" your LLM for $0.XXX for the same task. Of course the quality difference is huge. But sometimes you don't need that level of quality. Also, finding a vector artist takes days of communication, payment settlement, revisions, etc. Not always the most practical solution.

paulddraper 2mo ago

Pelicans don't ride bicycles. It's physically impossible. The problem is to draw it in the least disturbing way possible.

Morromist 2mo ago
True! But somehow Disney has been drawing ducks riding bikes in a way that seems to satisfy everyone since before my grandfather was born. https://ridesabike.com/donald-duck-daisy-duck-huey-dewey-and-louie-ride-a-bike/ https://ridesabike.com/donald-duck-daisy-duck-huey-dewey-and...
- monk_grilla 2mo ago
  Wow, that was a highly relevant and specific website to source here!
  - Morromist 2mo ago
    I love these little websites with amazingly focused content. <3
- jodrellblank 1mo ago
  It’s better than many of the AI offerings and the bike could steer and the ducks are sitting on saddles, but the three nephews can’t reach the bottom of the pedal stroke and by the looks of their feet on the far side of the bike they aren’t trying. That shouldn’t satisfy Donald and Daisy, leaving them with all the work.