The era of artificial intelligence (AI) controlling robots in keeping with user commands is fast approaching. The synthetic intelligence ‘Jarvis’ that appears within the movie ‘Iron Man’ shouldn’t be far off.

Research for that is already in full swing. Microsoft (MS) recently announced the outcomes of research on robot control using ‘ChatGPT’ on its blog. It’s a study that experimented with a way to regulate a robot in the center, reminiscent of when an individual gives a verbal command to a robot, ChatGPT codes it in robot language and delivers it.

Until now, robotics has been at the extent of repeating tasks during which engineers code within the robot language, input commands, and observe and proper robot feedback.

Then again, on this MS study, when an engineer gives a command in human language, ChatGPT converts it into robot code, transmits the command to the robot, and takes over all roles of observing and providing feedback.

For this purpose, MS taught ChatGPT robot-specific APIs. ChatGPT writes special code to regulate robot movements by utilizing natural language commands and robot APIs. For instance, when instructed to ‘pick up the ball’, ChatGPT generates robot control code, reminiscent of writing a poem or completing an essay, then inspects and corrects the code until the robot succeeds.

This study shows a robot controlled by code written by ChatGPT under human instructions, reminiscent of using a robot arm to rearrange blocks on the MS logo, flying a drone to examine the contents of a shelf, or using a robot to seek out an object with its vision function.

Through trial and error, Microsoft has established a technique and set of design principles for creating prompts for robot tasks.

First define a set of high-level robot APIs or function libraries. This library will be specific to a specific robot and should be mapped to an existing low-level implementation of the robot’s control stack or perception library. It is extremely vital to make use of descriptive names for high-level APIs in order that ChatGPT can infer about their behavior.

Next, write a text prompt for ChatGPT that explains the goal of the duty, while explicitly explaining which features can be found within the higher-level library. Prompts can even contain details about task limits or how ChatGPT constructs a response.

Users evaluate ChatGPT’s code output by direct inspection or using a simulator. When mandatory, users provide ChatGPT with feedback on the standard and safety of their answers using natural language.

Finally, when the user is satisfied with the answer, the ultimate code will be deployed to the robot.

Based on this research, Microsoft launched ‘PromptCraft’, a collaborative open-source platform that enables users to share prompting strategies for various robot categories. This included all prompts and dialogs that Microsoft utilized in its research.

Microsoft plans so as to add robot simulators and interfaces to check the algorithms created by ChatGPT in the longer term.

Chan Park, cpark@aitimes.com