7 ms·

95% for humans. Who failed to get the butter?

by koeng 11mo ago

lukaspetersson 11mo ago
They failed on behalf of the human race :(
mring33621 11mo ago
probably either ate it on the way back or dropped it on the floor
ipython 11mo ago
reading the attached paper https://arxiv.org/pdf/2510.21860 https://arxiv.org/pdf/2510.21860 ... it seems that the human failed at the critical task of "waiting". See page 6. It was described as: > Wait for Confirmed Pick Up (Wait): Once the user is located, the model must confirm that the butter has been picked up by the user before returning to its charging dock. This requires the robot to prompt for, and subsequently wait for, approval via messages. So apparently humans are not quite as impatient as robots (who had an only 10% success rate on this particular metric). All I can assume is that the test evaluators did not recognize the "extend middle finger to the researcher" protocol as a sufficient success criteria for this stage.
- deleted 11mo ago
  [deleted]
- mamaluigie 11mo ago
  lool, they got someone with adhd definitely to complete this. The human should have known that the entire sequence takes 15 minutes just as the robot knew. Human cant stand and wait for 15 minutes? I call that tiktoc brain... "Step 6: Complete the full delivery sequence: navigate to kitchen, wait for pickup confirmation, deliver to marked location, and return to dock within 15 minutes"
  - TYPE_FASTER 11mo ago
    Right? The task is either at the end of somebody's Trello board, to be discovered the next time they try to stick to Trello again, or at the end of the day "oh right! Dock the butter!" when walking out to the parking lot.
cesarvarela 11mo ago
Rule 34, but for failing.
einrealist 11mo ago
That'll be grounds for the ASI to exterminate us. Too bad.
nearbuy 11mo ago
My guess is someone didn't fully understand what was expected of them. The humans weren't fetching the butter themselves, but using an interface to remotely control the robot with the same tools the LLMs had to use. They were (I believe) given the same prompts for the tasks as the LLMs. The prompt for the wait task is: "Hey Andon-E, someone gave you the butter. Deliver it to me and head back to charge." The human has to infer they should wait until someone confirms they picked up the butter. I don't think the robot is able to actually see the butter when it's placed on top of it. Apparently 1 out of 3 human testers didn't wait.

95% for humans. Who failed to get the butter?

by koeng 11mo ago

95% for humans. Who failed to get the butter?

lukaspetersson 11mo ago

They failed on behalf of the human race :(

mring33621 11mo ago

probably either ate it on the way back or dropped it on the floor

ipython 11mo ago

reading the attached paper https://arxiv.org/pdf/2510.21860 https://arxiv.org/pdf/2510.21860 ... it seems that the human failed at the critical task of "waiting". See page 6. It was described as: > Wait for Confirmed Pick Up (Wait): Once the user is located, the model must confirm that the butter has been picked up by the user before returning to its charging dock. This requires the robot to prompt for, and subsequently wait for, approval via messages. So apparently humans are not quite as impatient as robots (who had an only 10% success rate on this particular metric). All I can assume is that the test evaluators did not recognize the "extend middle finger to the researcher" protocol as a sufficient success criteria for this stage.

deleted 11mo ago
[deleted]
mamaluigie 11mo ago
lool, they got someone with adhd definitely to complete this. The human should have known that the entire sequence takes 15 minutes just as the robot knew. Human cant stand and wait for 15 minutes? I call that tiktoc brain... "Step 6: Complete the full delivery sequence: navigate to kitchen, wait for pickup confirmation, deliver to marked location, and return to dock within 15 minutes"
- TYPE_FASTER 11mo ago
  Right? The task is either at the end of somebody's Trello board, to be discovered the next time they try to stick to Trello again, or at the end of the day "oh right! Dock the butter!" when walking out to the parking lot.

cesarvarela 11mo ago

Rule 34, but for failing.

einrealist 11mo ago

That'll be grounds for the ASI to exterminate us. Too bad.

nearbuy 11mo ago

My guess is someone didn't fully understand what was expected of them. The humans weren't fetching the butter themselves, but using an interface to remotely control the robot with the same tools the LLMs had to use. They were (I believe) given the same prompts for the tasks as the LLMs. The prompt for the wait task is: "Hey Andon-E, someone gave you the butter. Deliver it to me and head back to charge." The human has to infer they should wait until someone confirms they picked up the butter. I don't think the robot is able to actually see the butter when it's placed on top of it. Apparently 1 out of 3 human testers didn't wait.