“In a method it operates very like our brain does, where not the entire brain prompts on a regular basis,” says Oriol Vinyals, a deep learning team lead at DeepMind. This compartmentalizing saves the AI computing power and might generate responses faster.

“That form of fluidity going backwards and forwards across different modalities, and using that to look and understand, could be very impressive,” says Oren Etzioni, former technical director of the Allen Institute for Artificial Intelligence, who was not involved within the work. “That is stuff I even have not seen before.”

An AI that may operate across modalities would more closely resemble the way in which that human beings behave. “Persons are naturally multimodal,” Etzioni says; we will effortlessly switch between speaking, writing, and drawing images or charts to convey ideas. 

Etzioni cautioned against taking an excessive amount of meaning from the developments, nonetheless. “There’s a famous line,” he says. “Never trust an AI demo.” 

For one thing, it’s not clear how much the demonstration videos overlooked or cherry-picked from various tasks (Google indeed received criticism for its early Gemini launch for not disclosing that the video was sped up). It’s also possible the model wouldn’t have the opportunity to duplicate a number of the demonstrations if the input wording were barely tweaked. AI models generally, says Etzioni, are brittle. 

Today’s release of Gemini 1.5 Pro is restricted to developers and enterprise customers. Google didn’t specify when it should be available for wider release.