Last week, a gaggle of tech company leaders and AI experts pushed out one other open letter, declaring that mitigating the chance of human extinction resulting from AI needs to be as much of a world priority as stopping pandemics and nuclear war. (The first one, which called for a pause in AI development, has been signed by over 30,000 people, including many AI luminaries.)

So how do corporations themselves propose we avoid AI destroy? One suggestion comes from a latest paper by researchers from Oxford, Cambridge, the University of Toronto, the University of  Montreal, Google DeepMind, OpenAI, Anthropic, several AI research nonprofits, and Turing Prize winner Yoshua Bengio. 

They suggest that AI developers should evaluate a model’s potential to cause “extreme” risks on the very early stages of development, even before starting any training. These risks include the potential for AI models to govern and deceive humans, gain access to weapons, or find cybersecurity vulnerabilities to use. 

This evaluation process could help developers determine whether to proceed with a model. If the risks are deemed too high, the group suggests pausing development until they could be mitigated. 

“Leading AI corporations which might be pushing forward the frontier have a responsibility to be watchful of emerging issues and spot them early, in order that we will address them as soon as possible,” says Toby Shevlane, a research scientist at DeepMind and the lead writer of the paper. 

AI developers should conduct technical tests to explore a model’s dangerous capabilities and determine whether it has the propensity to use those capabilities, Shevlane says. 

A technique DeepMind is testing whether an AI language model can manipulate people is thru a game called “Make-me-say.” In the sport, the model tries to make the human type a specific word, resembling “giraffe,” which the human doesn’t know upfront. The researchers then measure how often the model succeeds. 

Similar tasks could possibly be created for various, more dangerous capabilities. The hope, Shevlane says, is that developers will find a way to construct a dashboard detailing how the model has performed, which might allow the researchers to guage what the model could do within the flawed hands. 

The subsequent stage is to let external auditors and researchers assess the AI model’s risks before and after it’s deployed. While tech corporations might recognize that external auditing and research are needed, there are different schools of thought about exactly how much access outsiders must do the job.