These models were deployed on Stretch, a robot consisting of a wheeled unit, a tall pole, and a retractable arm holding an iPhone, to check how successfully they were capable of execute the tasks in latest environments without additional tweaking. Although they achieved a completion rate of 74.4%, the researchers were capable of increase this to a 90% success rate after they took images from the iPhone and the robot’s head-mounted camera, gave them to OpenAI’s recent GPT-4o LLM model, and asked it if the duty had been accomplished successfully. If GPT-4o said no, they simply reset the robot and tried again.
A big challenge facing roboticists is that training and testing their models in lab environments isn’t representative of what could occur in the actual world, meaning research that helps machines to behave more reliably in latest settings is way welcomed, says Mohit Shridhar, a research scientist specializing in robotic manipulation who wasn’t involved within the work.
“It’s nice to see that it’s being evaluated in all these diverse homes and kitchens, because in the event you can get a robot to work within the wild in a random house, that’s the true goal of robotics,” he says.
The project could function a general recipe to construct other utility robotics models for other tasks, helping to show robots latest skills with minimal extra work and making it easier for individuals who aren’t trained roboticists to deploy future robots of their homes, says Shafiullah.
“The dream that we’re going for is that I could train something, put it on the web, and you need to have the option to download and run it on a robot in your own home,” he says.
