Inside a large autonomous warehouse, lots of of robots dart down aisles as they collect and distribute items to satisfy a gentle stream of customer orders. On this busy environment, even small traffic jams or minor collisions can snowball into massive slowdowns.
To avoid such an avalanche of inefficiencies, researchers from MIT and the tech firm Symbotic developed a brand new method that routinely keeps a fleet of robots moving easily. Their method learns which robots should go first at each moment, based on how congestion is forming, and adapts to prioritize robots which might be about to get stuck. In this fashion, the system can reroute robots prematurely to avoid bottlenecks.
The hybrid system utilizes deep reinforcement learning, a strong artificial intelligence method for solving complex problems, to work out which robots needs to be prioritized. Then, a quick and reliable planning algorithm feeds instructions to the robots, enabling them to reply rapidly in always changing conditions.
In simulations inspired by actual e-commerce warehouse layouts, this latest approach achieved a couple of 25 percent gain in throughput over other methods. Importantly, the system can quickly adapt to latest environments with different quantities of robots or varied warehouse layouts.
“There are numerous decision-making problems in manufacturing and logistics where firms depend on algorithms designed by human experts. But we’ve got shown that, with the ability of deep reinforcement learning, we are able to achieve super-human performance. This can be a very promising approach, because in these giant warehouses even a 2 or 3 percent increase in throughput can have a big impact,” says Han Zheng, a graduate student within the Laboratory for Information and Decision Systems (LIDS) at MIT and lead writer of a paper on this latest approach.
Zheng is joined on the paper by Yining Ma, a LIDS postdoc; Brandon Araki and Jingkai Chen of Symbotic; and senior writer Cathy Wu, the Class of 1954 Profession Development Associate Professor in Civil and Environmental Engineering (CEE) and the Institute for Data, Systems, and Society (IDSS) at MIT, and a member of LIDS. The research appears today within the .
Rerouting robots
Coordinating lots of of robots in an e-commerce warehouse concurrently is not any easy task.
The issue is particularly complicated since the warehouse is a dynamic environment, and robots continually receive latest tasks after reaching their goals. They should be rapidly redirected as they leave and enter the warehouse floor.
Corporations often leverage algorithms written by human experts to find out where and when robots should move to maximise the variety of packages they will handle.
But when there may be congestion or a collision, a firm may don’t have any alternative but to shut down the complete warehouse for hours to manually sort the issue out.
“On this setting, we don’t have a precise prediction of the longer term. We only know what the longer term might hold, when it comes to the packages that are available in or the distribution of future orders. The planning system must be adaptive to those changes because the warehouse operations go on,” Zheng says.
The MIT researchers achieved this adaptability using machine learning. They began by designing a neural network model to take observations of the warehouse environment and judge easy methods to prioritize the robots. They train this model using deep reinforcement learning, a trial-and-error method through which the model learns to manage robots in simulations that mimic actual warehouses. The model is rewarded for making decisions that increase overall throughput while avoiding conflicts.
Over time, the neural network learns to coordinate many robots efficiently.
“By interacting with simulations inspired by real warehouse layouts, our system receives feedback that we use to make its decision-making more intelligent. The trained neural network can then adapt to warehouses with different layouts,” Zheng explains.
It’s designed to capture the long-term constraints and obstacles in each robot’s path, while also considering dynamic interactions between robots as they move through the warehouse.
By predicting current and future robot interactions, the model plans to avoid congestion before it happens.
After the neural network decides which robots should receive priority, the system employs a tried-and-true planning algorithm to inform each robot easy methods to move from one point to a different. This efficient algorithm helps the robots react quickly within the changing warehouse environment.
This mix of methods is essential.
“This hybrid approach builds on my group’s work on easy methods to achieve one of the best of each worlds between machine learning and classical optimization methods. Pure machine-learning methods still struggle to resolve complex optimization problems, and yet it is incredibly time- and labor-intensive for human experts to design effective methods. But together, using expert-designed methods the proper way can tremendously simplify the machine learning task,” says Wu.
Overcoming complexity
Once the researchers trained the neural network, they tested the system in simulated warehouses that were different than those it had seen during training. Since industrial simulations were too inefficient for this complex problem, the researchers designed their very own environments to mimic what happens in actual warehouses.
On average, their hybrid learning-based approach achieved 25 percent greater throughput than traditional algorithms in addition to a random search method, when it comes to variety of packages delivered per robot. Their approach could also generate feasible robot path plans that overcame congestion brought on by traditional methods.
“Especially when the density of robots within the warehouse goes up, the complexity scales exponentially, and these traditional methods quickly start to interrupt down. In these environments, our method is far more efficient,” Zheng says.
While their system remains to be far-off from real-world deployment, these demonstrations highlight the feasibility and advantages of using a machine learning-guided approach in warehouse automation.
In the longer term, the researchers want to incorporate task assignments in the issue formulation, since determining which robot will complete each task impacts congestion. Additionally they plan to scale up their system to larger warehouses with hundreds of robots.
This research was funded by Symbotic.
