6 ms·

Now that it's a solved benchmark, can we get the animated version?

by swalsh 15d ago

leumon 15d ago
how about trying to draw an airbus a320 in 3d space using only one brush tool that can be moved to specific x,y,z coordinates (and its color, size & hardness can be changed). i think fable 5.1 did quite a good job (reasoning high, cost $0,261): https://files.catbox.moe/umx102.png https://files.catbox.moe/umx102.png for comparision, this is fable 5: https://files.catbox.moe/ihl4m1.png https://files.catbox.moe/ihl4m1.png
pampas 15d ago
I made a 3D browser game a few months ago where you play as a pelican riding a bike via a bluetooth trainer.
alexandra_au 15d ago
Not to be "that" person but it's not solved. The feet are reversed and isn't accurate bird anatomy. In real life, what people think of as bird's feet is actually their toes, and their "knee" is actually their tarsal (ankle bone), and their actual knee is almost hidden in their feathers.
- frumiousirc 15d ago
  > not solved I agree. Other reasoning traces simonw quoted in his blog post showed that the model made changes to consider realistic fork rake. I think this may also be the first case where the chain went inside the seat stays. The overall bike geometry is still comical but these bits show improvement in this model over prior ones. On the other hand, given the absurdity of the original prompt, I should not necessarily expect realistic bike geometry.
- stevage 15d ago
  The problem is it's kind of a question without a well defined answer. If it was just drawing a picture of a pelican, we can compare against real pelicans for accuracy. But what exactly is a pelican riding a bike supposed to look like? It's impossible, so liberties have to be taken somewhere.
simonw 15d ago
I didn't want to shell out for Max again, so I piped the SVG created by Max back into Fable 5.1 at its default thinking level (of high): llm logs -cx | llm -m claude-fable-5.1 -s 'animate this' Here's the result, which cost $1.37: https://tools.simonwillison.net/markdown-svg-renderer?url=https%3A%2F%2Fgist.github.com%2Fsimonw%2F87282467acb3652e0f99c85155554a32#response https://tools.simonwillison.net/markdown-svg-renderer?url=ht... It's excellent!
- rafaelmn 15d ago
  First time this benchmark impressed me - 10/10 I would believe a human spent hours making this. There are details a human would have done better (beak attaching to the head) but still impressive.
  - psygn89 15d ago
    I like how precise it got about the math behind the chain/gear/wheel/leg movement... these are the details that ain't no one really got time for.
- throw03172019 15d ago
  He looks very happy riding along the beach. I love the basket fish animation as well. Impressive!
- saretup 15d ago
  Now that it's a solved benchmark, can we get the 3d animated version?
  - consumer451 15d ago
    I can't waste Fable for play time, but I was curius, so I used plain claude.ai with Opus 5 High. It's pretty cool. I asked "create a 3d animation", and pasted the Simon’s animation code. I must admit I used a second prompt to adjust the opening camera angle, but that is all. Originally the camera was above, about 30 degrees. https://claude.ai/public/artifacts/b37a9ee2-f5bc-4ff9-ae90-a1fb8917c737 https://claude.ai/public/artifacts/b37a9ee2-f5bc-4ff9-ae90-a...

Now that it's a solved benchmark, can we get the animated version?

by swalsh 15d ago

Now that it's a solved benchmark, can we get the animated version?

how about trying to draw an airbus a320 in 3d space using only one brush tool that can be moved to specific x,y,z coordinates (and its color, size & hardness can be changed). i think fable 5.1 did quite a good job (reasoning high, cost $0,261): https://files.catbox.moe/umx102.png https://files.catbox.moe/umx102.png for comparision, this is fable 5: https://files.catbox.moe/ihl4m1.png https://files.catbox.moe/ihl4m1.png

pampas 15d ago

I made a 3D browser game a few months ago where you play as a pelican riding a bike via a bluetooth trainer.

alexandra_au 15d ago

Not to be "that" person but it's not solved. The feet are reversed and isn't accurate bird anatomy. In real life, what people think of as bird's feet is actually their toes, and their "knee" is actually their tarsal (ankle bone), and their actual knee is almost hidden in their feathers.

frumiousirc 15d ago
> not solved I agree. Other reasoning traces simonw quoted in his blog post showed that the model made changes to consider realistic fork rake. I think this may also be the first case where the chain went inside the seat stays. The overall bike geometry is still comical but these bits show improvement in this model over prior ones. On the other hand, given the absurdity of the original prompt, I should not necessarily expect realistic bike geometry.
stevage 15d ago
The problem is it's kind of a question without a well defined answer. If it was just drawing a picture of a pelican, we can compare against real pelicans for accuracy. But what exactly is a pelican riding a bike supposed to look like? It's impossible, so liberties have to be taken somewhere.

simonw 15d ago

I didn't want to shell out for Max again, so I piped the SVG created by Max back into Fable 5.1 at its default thinking level (of high): llm logs -cx | llm -m claude-fable-5.1 -s 'animate this' Here's the result, which cost $1.37: https://tools.simonwillison.net/markdown-svg-renderer?url=https%3A%2F%2Fgist.github.com%2Fsimonw%2F87282467acb3652e0f99c85155554a32#response https://tools.simonwillison.net/markdown-svg-renderer?url=ht... It's excellent!

rafaelmn 15d ago
First time this benchmark impressed me - 10/10 I would believe a human spent hours making this. There are details a human would have done better (beak attaching to the head) but still impressive.
- psygn89 15d ago
  I like how precise it got about the math behind the chain/gear/wheel/leg movement... these are the details that ain't no one really got time for.
throw03172019 15d ago
He looks very happy riding along the beach. I love the basket fish animation as well. Impressive!
saretup 15d ago
Now that it's a solved benchmark, can we get the 3d animated version?
- consumer451 15d ago
  I can't waste Fable for play time, but I was curius, so I used plain claude.ai with Opus 5 High. It's pretty cool. I asked "create a 3d animation", and pasted the Simon’s animation code. I must admit I used a second prompt to adjust the opening camera angle, but that is all. Originally the camera was above, about 30 degrees. https://claude.ai/public/artifacts/b37a9ee2-f5bc-4ff9-ae90-a1fb8917c737 https://claude.ai/public/artifacts/b37a9ee2-f5bc-4ff9-ae90-a...