Why did humans evolve the eyes we now have today?
While scientists can’t return in time to check the environmental pressures that shaped the evolution of the varied vision systems that exist in nature, a brand new computational framework developed by MIT researchers allows them to explore this evolution in artificial intelligence agents.
The framework they developed, during which embodied AI agents evolve eyes and learn to see over many generations, is sort of a “scientific sandbox” that enables researchers to recreate different evolutionary trees. The user does this by changing the structure of the world and the tasks AI agents complete, akin to finding food or telling objects apart.
This enables them to check why one animal can have evolved easy, light-sensitive patches as eyes, while one other has complex, camera-type eyes.
The researchers’ experiments with this framework showcase how tasks drove eye evolution within the agents. As an illustration, they found that navigation tasks often led to the evolution of compound eyes with many individual units, just like the eyes of insects and crustaceans.
However, if agents focused on object discrimination, they were more more likely to evolve camera-type eyes with irises and retinas.
This framework could enable scientists to probe “what-if” questions on vision systems which might be difficult to check experimentally. It could also guide the design of novel sensors and cameras for robots, drones, and wearable devices that balance performance with real-world constraints like energy efficiency and manufacturability.
“While we will never return and determine every detail of how evolution took place, on this work we’ve created an environment where we will, in a way, recreate evolution and probe the environment in all these other ways. This approach to doing science opens to the door to numerous possibilities,” says Kushagra Tiwary, a graduate student on the MIT Media Lab and co-lead writer of a paper on this research.
He’s joined on the paper by co-lead writer and fellow graduate student Aaron Young; graduate student Tzofi Klinghoffer; former postdoc Akshat Dave, who’s now an assistant professor at Stony Brook University; Tomaso Poggio, the Eugene McDermott Professor within the Department of Brain and Cognitive Sciences, an investigator within the McGovern Institute, and co-director of the Center for Brains, Minds, and Machines; co-senior authors Brian Cheung, a postdoc within the Center for Brains, Minds, and Machines and an incoming assistant professor on the University of California San Francisco; and Ramesh Raskar, associate professor of media arts and sciences and leader of the Camera Culture Group at MIT; in addition to others at Rice University and Lund University. The research appears today in .
Constructing a scientific sandbox
The paper began as a conversation among the many researchers about discovering latest vision systems that may very well be useful in numerous fields, like robotics. To check their “what-if” questions, the researchers decided to use AI to explore the various evolutionary possibilities.
“What-if questions inspired me after I was growing up to check science. With AI, we now have a singular opportunity to create these embodied agents that allow us to ask the sorts of questions that may normally be not possible to reply,” Tiwary says.
To construct this evolutionary sandbox, the researchers took all the weather of a camera, just like the sensors, lenses, apertures, and processors, and converted them into parameters that an embodied AI agent could learn.
They used those constructing blocks as the start line for an algorithmic learning mechanism an agent would use because it evolved eyes over time.
“We couldn’t simulate the whole universe atom-by-atom. It was difficult to find out which ingredients we wanted, which ingredients we didn’t need, and easy methods to allocate resources over those different elements,” Cheung says.
Of their framework, this evolutionary algorithm can select which elements to evolve based on the constraints of the environment and the duty of the agent.
Each environment has a single task, akin to navigation, food identification, or prey tracking, designed to mimic real visual tasks animals must overcome to survive. The agents start with a single photoreceptor that appears out on the world and an associated neural network model that processes visual information.
Then, over each agent’s lifetime, it’s trained using reinforcement learning, a trial-and-error technique where the agent is rewarded for accomplishing the goal of its task. The environment also incorporates constraints, like a certain variety of pixels for an agent’s visual sensors.
“These constraints drive the design process, the identical way we now have physical constraints in our world, just like the physics of sunshine, which have driven the design of our own eyes,” Tiwary says.
Over many generations, agents evolve different elements of vision systems that maximize rewards.
Their framework uses a genetic encoding mechanism to computationally mimic evolution, where individual genes mutate to manage an agent’s development.
As an illustration, morphological genes capture how the agent views the environment and control eye placement; optical genes determine how the attention interacts with light and dictate the variety of photoreceptors; and neural genes control the educational capability of the agents.
Testing hypotheses
When the researchers arrange experiments on this framework, they found that tasks had a significant influence on the vision systems the agents evolved.
As an illustration, agents that were focused on navigation tasks developed eyes designed to maximise spatial awareness through low-resolution sensing, while agents tasked with detecting objects developed eyes focused more on frontal acuity, quite than peripheral vision.
One other experiment indicated that a much bigger brain isn’t at all times higher relating to processing visual information. Only a lot visual information can go into the system at a time, based on physical constraints just like the variety of photoreceptors within the eyes.
“In some unspecified time in the future a much bigger brain doesn’t help the agents in any respect, and in nature that may be a waste of resources,” Cheung says.
In the long run, the researchers wish to use this simulator to explore the most effective vision systems for specific applications, which could help scientists develop task-specific sensors and cameras. In addition they wish to integrate LLMs into their framework to make it easier for users to ask “what-if” questions and study additional possibilities.
“There’s an actual profit that comes from asking questions in a more imaginative way. I hope this inspires others to create larger frameworks, where as a substitute of specializing in narrow questions that cover a particular area, they wish to answer questions with a much wider scope,” Cheung says.
This work was supported, partly, by the Center for Brains, Minds, and Machines and the Defense Advanced Research Projects Agency (DARPA) Mathematics for the Discovery of Algorithms and Architectures (DIAL) program.
