MahiShafiullah

8 ms·

1.
Show HN: Robot Utility Models: open-source robots working in new envs zero shot(robotutilitymodels.com)
11 points by MahiShafiullah 2y ago|0 comments
2.
by MahiShafiullah 3y ago
Thank you!
3.
by MahiShafiullah 3y ago
User fishbotics already answers a lot of these questions downstream, but just confirming it here as an author of the project/paper: > - How does it know what objects are? Does it use some sort of realtime object classifier neural ne
4.
by MahiShafiullah 3y ago
A large motivation behind this line of home-robot work for me is thinking about the elderly, people with disabilities, or busy parents who simply don't have enough time to do it all. I am personally hopeful that we can teach AI to take
5.
by MahiShafiullah 3y ago
In fact, Hello Robot already shared a teleoperated demo of folding shirts! https://www.youtube.com/watch?v=QtG8nJ78x2M&t=180s But yes, a second arm is needed.
6.
by MahiShafiullah 3y ago
Thank you! A large motivation behind this line of home-robot work for me is thinking about the elderly, people with disabilities, or busy parents who simply don't have enough time to do it all. I am personally hopeful that we can teach
7.
by MahiShafiullah 3y ago
It's always a trade-off! You could have more accurate sensors and motors that are more expensive, or you can have cheaper motors with no sensors and higher accumulated errors. Since this is more of a research project than a product, we
8.
by MahiShafiullah 3y ago
The title has multiple meanings, some credits definitely should go to OK computer/radiohead, but also "OK Google" for controlling a home assistant, open-knowledge (OK) models, etc.
9.
by MahiShafiullah 3y ago
No, although it has some of the same people on the team (aka I'm the first author there, and my advisor is advising both projects :) ) The primary difference is that this is zero-shot (meaning the robot needs 0 (zero!) new data in a ne
10.
by MahiShafiullah 3y ago
The mapping process can be done with any RGB-D cameras, we use an iPhone pro but any apple devide with AR-Kit should work. Once we have a sequence of RGB-D images with associated camera poses, we can just backproject the pixels (and any ass
11.
by MahiShafiullah 3y ago
Here is the page for Hello Robots: https://hello-robot.com/
12.
by MahiShafiullah 3y ago
Thanks! We collect all the data and analyze it post-facto to see what may have caused the failure. For example, on the 5 trials you mentioned, the Realsense gave wrong depth on transparent or semi-transparent objects, and so the pointcloud
13.
Show HN: OK-Robot: open, modular home robot framework for pick-and-drop anywhere(ok-robot.github.io)
542 points by MahiShafiullah 3y ago|110 comments
14.
by MahiShafiullah 3y ago
Second that. Something I put out recently had an (admittedly video heavy) webpage that had 1TB of traffic over the past month. Cloudflare handled it for free for me, but at ArXiv’s scale it’s bound to be a problem.
15.
by MahiShafiullah 3y ago
Dobb·E is free (and open source) now :)
16.
by MahiShafiullah 3y ago
I am glad you mentioned automobiles! They are among the most heavily engineered everyday product that we use every day, and this engineering and cost optimization has been going on for close to a hundred years now. So, I would read this sam
17.
by MahiShafiullah 3y ago
I'm sorry, should have been clearer. The Spot robot with a manipulator arm (which has been used in some home projects before, see [0]) I believe totals up to about $200K. The base robot is $75K, which is what you linked to. I mentioned
18.
by MahiShafiullah 3y ago
Smaller robots are unfortunately limited in how much force/torque they can apply and how high they can reach, two factors that we found can severely limit the applicability of a home robot. As a result, unfortunately, we don't hav
19.
by MahiShafiullah 3y ago
> For the iPhone with the depth sensor cinematic format from the standard camera app seem to contain depth/RGB/traj. Maybe a candidate format. Also it’s much easier to operate standard camera app. I agree with you there. My imp
20.
by MahiShafiullah 3y ago
I am sorry, what problems are you facing? I tested it with Safari on both my phone and laptop just now and couldn't find an issue, so I would really appreciate it if you can reply here or (more appreciated) email me with images etc. on
21.
by MahiShafiullah 3y ago
Thank you Suren! It will get there when it gets there and so will I :) I do miss you folks, though.
22.
by MahiShafiullah 3y ago
I discuss this a little bit on another thread: ( https://news.ycombinator.com/item?id=38461559 ) But the tl;dr is that building prototypes is pricey, and the cost should definitely come down once the community moves from prot
23.
by MahiShafiullah 3y ago
Tool usage is one of the next ideas we would like to explore; because, if you think about it, the reacher-grabber that is currently used as an end-effector is just another tool. If we can switch between tools quickly we can possibly unlock
24.
by MahiShafiullah 3y ago
Great observation! Right now we rely on the out-of-the-box Hello Robots controller [0] implementing position control. Since our policy only predicts a single position into the future at a time, the motion ends up being either jerky and fast
25.
by MahiShafiullah 3y ago
OpenEXR looks very interesting! Especially if OpenEXR can compress depth data nearly as efficiently as we can compress image data by making everything a video, it would be a game changer. For example, our RGB dataset is 814 MB right now whi
26.
by MahiShafiullah 3y ago
The robot can carry small weights on its own, yes, although it's way more stable when the arm is retracted closer to the body. Pick-and-place style problems are one of the earliest tried on our robot platform, Hello Stretch, which is w
27.
by MahiShafiullah 3y ago
We don't want any private information, so currently we are manually going over every new demo to make sure private data (face, hands, any other identifying information) doesn't get included in the dataset. Pretty hard to "bli
28.
by MahiShafiullah 3y ago
Thank you! Yes, we have considered launching an open source effort to scale up the data, perhaps to be comparable to what Google has collected through paid tele-operators [0]. What do you think would be the right incentive structure for eve
29.
by MahiShafiullah 3y ago
Thanks for the words of encouragement! If I may ask, what is the level of "maturity" you expect before adopting a home robot? Clearly, a robot that can wash and fold laundry is already there, but what are the other necessary/
30.
by MahiShafiullah 3y ago
Not in our plans anytime soon, sorry :) We're focusing on home robots right now instead of rather unsettling space mutants.

More ›