Midjourney’s founder, David Holz, says it’s banning these words as a stopgap measure to stop people from generating shocking or gory  content while the corporate “improves things on the AI side.” Holz says moderators watch how words are getting used and what sorts of images are being generated, and adjust the bans periodically. The firm has a community guidelines page that lists the form of content it blocks in this manner, including sexual imagery, gore, and even the 🍑 emoji, which is commonly used as a logo for the buttocks.

AI models comparable to Midjourney, DALL-E 2, and Stable Diffusion are trained on billions of images which were scraped from the web. Research by a team on the University of Washington has found that such models learn biases that sexually objectify women, that are then reflected in the photographs they produce. The huge size of the information set makes it almost inconceivable to remove unwanted images, comparable to those of a sexual or violent nature, or those that might produce biased outcomes. The more often something appears in the information set, the stronger the connection the AI model makes, which implies it’s more prone to appear in images the model generates.  

Midjourney’s word bans are a piecemeal attempt to handle this problem. Some terms referring to the male reproductive system, comparable to “sperm” and “testicles,” are blocked too, however the list of banned words seems to skew predominantly female. 

The prompt ban was first spotted by Julia Rockwell, a clinical data analyst at Datafy Clinical, and her friend Madeline Keenen, a cell biologist on the University of North Carolina at Chapel Hill. Rockwell used Midjourney to attempt to generate a fun image of the placenta for Keenen, who studies them. To her surprise, Rockwell found that using “placenta” as a prompt was banned. She then began experimenting with other words related to the human reproductive system, and located the identical.

Nevertheless, the pair also showed the way it’s possible to work around these bans to create sexualized images by utilizing different spellings of words, or other euphemisms for sexual or gory content. 

In findings they shared with MIT Technology Review, they found that the prompt “gynaecological exam”—using the British spelling—generated some deeply creepy images: one in every of two naked women in a physician’s office, and one other of a bald three-limbed person cutting up their very own stomach. 

Midjourney’s crude banning of prompts referring to reproductive biology highlights how tricky it’s to moderate content around generative AI systems. It also demonstrates how the tendency for AI systems to sexualize women extends all of the approach to their internal organs, says Rockwell.