Researchers at Aalto University have developed an progressive bio-inspired sensor that may detect moving objects in a single video frame and accurately predict their future movements. Described in a paper, this advanced sensor has quite a few potential applications in fields similar to dynamic vision sensing, automatic inspection, industrial process control, robotic guidance, and autonomous driving technology.
Traditional motion detection systems require quite a few components and complicated algorithms that perform frame-by-frame analyses, leading to inefficiency and high energy consumption. To handle these limitations, the Aalto University team looked to the human visual system for inspiration and created a neuromorphic vision technology that unifies sensing, memory, and processing right into a single device able to detecting motion and predicting trajectories.
Photomemristors: The Core of the Latest Technology
The researchers’ technology is built on an array of photomemristors, electrical devices that generate electric current in response to light. Photomemristors possess a singular characteristic: the present doesn’t stop immediately when the sunshine is turned off, but decays steadily. This feature allows photomemristors to effectively “remember” their recent exposure to light, enabling a sensor composed of an array of those devices to capture not only instantaneous details about a scene but additionally a dynamic memory of preceding moments.
“The unique property of our technology is its ability to integrate a series of optical images in a single frame,” explains Hongwei Tan, the research fellow who led the study. “The data of every image is embedded in the next images as hidden information. In other words, the ultimate frame in a video also has details about all of the previous frames. That lets us detect motion earlier within the video by analyzing only the ultimate frame with an easy artificial neural network. The result’s a compact and efficient sensing unit.”
Demonstrating the Technology’s Capabilities
To showcase their technology, the researchers used videos that displayed the letters of a word one by one. Although all of the words ended with the letter “E,” conventional vision sensors couldn’t discern whether the “E” on the screen had followed the opposite letters in “APPLE” or “GRAPE.” Nonetheless, the photomemristor array could utilize hidden information in the ultimate frame to deduce which letters had preceded it and predict the word with nearly 100% accuracy.
In one other experiment, the team showed the sensor videos of a simulated person moving at three different speeds. The system couldn’t only recognize motion by analyzing a single frame, but it surely also accurately predicted subsequent frames.
Implications for Autonomous Vehicles and Intelligent Transport
Accurate motion detection and trajectory prediction are crucial for self-driving technology and intelligent transport systems. Autonomous vehicles depend on precise predictions of how cars, bikes, pedestrians, and other objects will move to be able to make informed decisions. By incorporating a machine learning system into the photomemristor array, the researchers demonstrated that their integrated system could predict future motion based on in-sensor processing of an all-informative frame.
“Motion recognition and prediction by our compact in-sensor memory and computing solution provides recent opportunities in autonomous robotics and human-machine interactions,” says Professor Sebastiaan van Dijken. “The in-frame information that we attain in our system using photomemristors avoids redundant data flows, enabling energy-efficient decision-making in real time.”