Artificial intelligence also controls robots…MS, robot control research with ‘ChatGPT’

-

(Photo = MS)

The era of artificial intelligence (AI) controlling robots in keeping with user commands is fast approaching. The synthetic intelligence ‘Jarvis’ that appears within the movie ‘Iron Man’ shouldn’t be far off.

Research for that is already in full swing. Microsoft (MS) recently announced the outcomes of research on robot control using ‘ChatGPT’ on its blog. It’s a study that experimented with a way to regulate a robot in the center, reminiscent of when an individual gives a verbal command to a robot, ChatGPT codes it in robot language and delivers it.

Until now, robotics has been at the extent of repeating tasks during which engineers code within the robot language, input commands, and observe and proper robot feedback.

Then again, on this MS study, when an engineer gives a command in human language, ChatGPT converts it into robot code, transmits the command to the robot, and takes over all roles of observing and providing feedback.

Current robotics pipelines require specialized engineers in the loop to write code to improve processes.  The goal of ChatGPT is to allow non-technical users to stay in the loop, interact with language models through high-level language commands, and seamlessly deploy tasks with various platforms. (Picture = MS)
Current robotics pipelines require specialized engineers within the loop to jot down code to enhance processes. The goal of ChatGPT is to permit non-technical users to remain within the loop, interact with language models through high-level language commands, and seamlessly deploy tasks with various platforms. (Picture = MS)

For this purpose, MS taught ChatGPT robot-specific APIs. ChatGPT writes special code to regulate robot movements by utilizing natural language commands and robot APIs. For instance, when instructed to ‘pick up the ball’, ChatGPT generates robot control code, reminiscent of writing a poem or completing an essay, then inspects and corrects the code until the robot succeeds.

This study shows a robot controlled by code written by ChatGPT under human instructions, reminiscent of using a robot arm to rearrange blocks on the MS logo, flying a drone to examine the contents of a shelf, or using a robot to seek out an object with its vision function.

A robotics pipeline using ChatGPT with users in a loop to evaluate the quality and safety of the output (Picture=MS)
A robotics pipeline using ChatGPT with users in a loop to judge the standard and safety of the output (Picture=MS)

Through trial and error, Microsoft has established a technique and set of design principles for creating prompts for robot tasks.

First define a set of high-level robot APIs or function libraries. This library will be specific to a specific robot and should be mapped to an existing low-level implementation of the robot’s control stack or perception library. It is extremely vital to make use of descriptive names for high-level APIs in order that ChatGPT can infer about their behavior.

Next, write a text prompt for ChatGPT that explains the goal of the duty, while explicitly explaining which features can be found within the higher-level library. Prompts can even contain details about task limits or how ChatGPT constructs a response.

Users evaluate ChatGPT’s code output by direct inspection or using a simulator. When mandatory, users provide ChatGPT with feedback on the standard and safety of their answers using natural language.

Finally, when the user is satisfied with the answer, the ultimate code will be deployed to the robot.

ChatGPT demo for robots (video = MS)

Based on this research, Microsoft launched ‘PromptCraft’, a collaborative open-source platform that enables users to share prompting strategies for various robot categories. This included all prompts and dialogs that Microsoft utilized in its research.

Microsoft plans so as to add robot simulators and interfaces to check the algorithms created by ChatGPT in the longer term.

Chan Park, cpark@aitimes.com

ASK DUKE

What are your thoughts on this topic?
Let us know in the comments below.

4 COMMENTS

0 0 votes
Article Rating
guest
4 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments

Share this article

Recent posts

4
0
Would love your thoughts, please comment.x
()
x