Our mission is to be sure that artificial general intelligence—AI systems which are generally smarter than humans—advantages all of humanity.

If AGI is successfully created, this technology could help us elevate humanity by increasing abundance, turbocharging the worldwide economy, and aiding in the invention of latest scientific knowledge that changes the boundaries of possibility.

AGI has the potential to offer everyone incredible latest capabilities; we will imagine a world where all of us have access to assist with almost any cognitive task, providing a fantastic force multiplier for human ingenuity and creativity.

Alternatively, AGI would also include serious risk of misuse, drastic accidents, and societal disruption. Since the upside of AGI is so great, we don’t imagine it is feasible or desirable for society to stop its development perpetually; as a substitute, society and the developers of AGI should determine learn how to get it right.

Although we cannot predict exactly what is going to occur, and in fact our current progress could hit a wall, we will articulate the principles we care about most:

1. We would like AGI to empower humanity to maximally flourish within the universe. We don’t expect the longer term to be an unqualified utopia, but we would like to maximise the nice and minimize the bad, and for AGI to be an amplifier of humanity.
2. We would like the advantages of, access to, and governance of AGI to be widely and fairly shared.
3. We would like to successfully navigate massive risks. In confronting these risks, we acknowledge that what seems right in theory often plays out more strangely than expected in practice. We imagine now we have to constantly learn and adapt by deploying less powerful versions of the technology so as to minimize “one shot to get it right” scenarios.

The short term

There are several things we predict are essential to do now to organize for AGI.

First, as we create successively more powerful systems, we would like to deploy them and gain experience with operating them in the true world. We imagine that is the perfect strategy to fastidiously steward AGI into existence—a gradual transition to a world with AGI is best than a sudden one. We expect powerful AI to make the speed of progress on the earth much faster, and we predict it’s higher to regulate to this incrementally.

A gradual transition gives people, policymakers, and institutions time to grasp what’s happening, personally experience the advantages and drawbacks of those systems, adapt our economy, and to place regulation in place. It also allows for society and AI to co-evolve, and for people collectively to determine what they need while the stakes are relatively low.

We currently imagine the perfect strategy to successfully navigate AI deployment challenges is with a good feedback loop of rapid learning and careful iteration. Society will face major questions on what AI systems are allowed to do, learn how to combat bias, learn how to cope with job displacement, and more. The optimal decisions will rely upon the trail the technology takes, and like all latest field, most expert predictions have been flawed to date. This makes planning in a vacuum very difficult.

Generally speaking, we predict more usage of AI on the earth will result in good, and wish to put it on the market (by putting models in our API, open-sourcing them, etc.). We imagine that democratized access may also result in more and higher research, decentralized power, more advantages, and a broader set of individuals contributing latest ideas.

As our systems catch up with to AGI, we have gotten increasingly cautious with the creation and deployment of our models. Our decisions would require far more caution than society normally applies to latest technologies, and more caution than many users would really like. Some people within the AI field think the risks of AGI (and successor systems) are fictitious; we can be delighted in the event that they become right, but we’re going to operate as if these risks are existential.

As our systems catch up with to AGI, we have gotten increasingly cautious with the creation and deployment of our models.

In some unspecified time in the future, the balance between the upsides and drawbacks of deployments (resembling empowering malicious actors, creating social and economic disruptions, and accelerating an unsafe race) could shift, during which case we might significantly change our plans around continuous deployment.

Second, we’re working towards creating increasingly aligned and steerable models. Our shift from models just like the first version of GPT-3 to InstructGPT and ChatGPT is an early example of this.

Particularly, we predict it’s essential that society agree on extremely wide bounds of how AI could be used, but that inside those bounds, individual users have plenty of discretion. Our eventual hope is that the institutions of the world agree on what these wide bounds ought to be; within the shorter term we plan to run experiments for external input. The institutions of the world will have to be strengthened with additional capabilities and experience to be prepared for complex decisions about AGI.

The “default setting” of our products will likely be quite constrained, but we plan to make it easy for users to alter the behavior of the AI they’re using. We imagine in empowering individuals to make their very own decisions and the inherent power of diversity of ideas.

We’ll have to develop latest alignment techniques as our models turn out to be more powerful (and tests to grasp when our current techniques are failing). Our plan within the shorter term is to make use of AI to assist humans evaluate the outputs of more complex models and monitor complex systems, and in the long run to make use of AI to assist us provide you with latest ideas for higher alignment techniques.

Importantly, we predict we regularly should make progress on AI safety and capabilities together. It’s a false dichotomy to speak about them individually; they’re correlated in some ways. Our greatest safety work has come from working with our most capable models. That said, it’s essential that the ratio of safety progress to capability progress increases.

Third, we hope for a world conversation about three key questions: learn how to govern these systems, learn how to fairly distribute the advantages they generate, and learn how to fairly share access.

Along with these three areas, now we have attempted to establish our structure in a way that aligns our incentives with a great final result. We now have a clause in our Charter about assisting other organizations to advance safety as a substitute of racing with them in late-stage AGI development. We now have a cap on the returns our shareholders can earn in order that we aren’t incentivized to try and capture value without certain and risk deploying something potentially catastrophically dangerous (and in fact as a strategy to share the advantages with society). We now have a nonprofit that governs us and lets us operate for the nice of humanity (and may override any for-profit interests), including letting us do things like cancel our equity obligations to shareholders if needed for safety and sponsor the world’s most comprehensive UBI experiment.

We now have attempted to establish our structure in a way that aligns our incentives with a great final result.

We expect it’s essential that efforts like ours undergo independent audits before releasing latest systems; we are going to speak about this in additional detail later this 12 months. In some unspecified time in the future, it could be essential to get independent review before beginning to train future systems, and for probably the most advanced efforts to comply with limit the speed of growth of compute used for creating latest models. We expect public standards about when an AGI effort should stop a training run, determine a model is secure to release, or pull a model from production use are essential. Finally, we predict it’s essential that major world governments have insight about training runs above a certain scale.

The long run

We imagine that way forward for humanity ought to be determined by humanity, and that it’s essential to share details about progress with the general public. There ought to be great scrutiny of all efforts attempting to construct AGI and public consultation for major decisions.

The primary AGI shall be just some extent along the continuum of intelligence. We expect it’s likely that progress will proceed from there, possibly sustaining the speed of progress we’ve seen over the past decade for an extended time period. If that is true, the world could turn out to be extremely different from the way it is today, and the risks could possibly be extraordinary. A misaligned superintelligent AGI could cause grievous harm to the world; an autocratic regime with a decisive superintelligence lead could do this too.

AI that may speed up science is a special case value fascinated about, and maybe more impactful than the whole lot else. It’s possible that AGI capable enough to speed up its own progress could cause major changes to occur surprisingly quickly (and even when the transition starts slowly, we expect it to occur pretty quickly in the ultimate stages). We expect a slower takeoff is less complicated to make secure, and coordination amongst AGI efforts to decelerate at critical junctures will likely be essential (even in a world where we don’t need to do that to unravel technical alignment problems, slowing down could also be essential to offer society enough time to adapt).

Successfully transitioning to a world with superintelligence is maybe crucial—and hopeful, and scary—project in human history. Success is removed from guaranteed, and the stakes (boundless downside and boundless upside) will hopefully unite all of us.

We will imagine a world during which humanity flourishes to a level that might be inconceivable for any of us to totally visualize yet. We hope to contribute to the world an AGI aligned with such flourishing.