CivitAI, possibly the most well-liked AI model repository on the web, has finally conceded to pressure from payment facilitators MasterCard and Visa to radically revise their policies on NSFW content – and particularly their TOS regarding celebrity LoRAs, certainly one of the positioning’s hottest user-submitted content streams, which enable people to make use of freely downloadable adjunct models corresponding to LoRAs to create AI depictions (including video depictions) of well-known people.
Source: civitai.com
Clearly operating under pressure in a Twitch live-stream on behalf of the corporate, the corporate’s Community Engagement Manager Alasdair Nicoll, himself a creator of (SFW) models at Civit, admitted that the changes have been forced upon the positioning by their payment processors’ concerns about adult content, and depiction of real people. He also admits the likelihood that the first forces behind these processors, Visa and MasterCard, are prone to demand even greater changes later:
The Civit domain has been down periodically for revisions over the previous couple of days, apparently to effect the changes. Though the positioning had already banned using NSFW themes in celebrity LoRA/model depictions, it’s now inconceivable to browse the model section of Civit and see celebrity LoRA previews side-by-side with the very large variety of generic NSFW models designed to provide mature content.
The official announcement states:
Within the Twitch session, Nicoll revealed further details of measures designed to guard famous figures and real people. Civit has all the time allowed real people to request that a Civit-hosted AI model depicting them be taken down, but now Nicoll alludes to a system that may prevent such images being re-uploaded after initial rejection, with the ability to discover a ‘protected’ personage even in images that the system has never seen before.
To this end, the positioning is now partnering with the Clavata AI moderation system – though the extent to which Clavata shall be powering these recent facilities shouldn’t be yet clear.
Nicoll said:
Protected by Default?
Over the past couple of years, the AI VFX company Metaphysic (full disclosure: I worked for Metaphysic.ai from early 2002 until late 2024) attempted to create a proprietary system that might allow anyone to register their very own likeness, though primarily aimed toward Hollywood names concerned about AI-based hijacking of their identities, with support from actors corresponding to Anne Hathaway, Octavia Spencer and Tom Hanks (with whom the corporate worked on the Robert Zemeckis outing [2024]).
Logically, the utility of the system would all the time depend upon eventual case law; based on the measures Civit is now being forced to take, the subscription-based service†proposed by Metaphysic could possibly be redundant within the face of the rapid growth of deepfake laws, and potential (free) coverage under common law. It isn’t currently known whether the Metaphysic Pro offering will transfer to the Double Negative VFX company, which acquired Metaphysic’s assets last 12 months.
In any case, it increasingly seems that global law and general market pressures usually tend to provide protection and remedies, versus industrial solutions of this sort.
Boiling the Frog
A 2023 report by 404 Media brought attention to the proclivity of celeb and porn AI models at Civit, though the positioning’s founder Justin Maier downplayed the connection between user-contributed celebrity likenesses and their use in generating pornographic material.
Though Civit makes money by facilitating the on-site use of LoRAs and other user-supplied models, Nicoll is evident that this shouldn’t be the first concern motivating Visa and MasterCard to stipulate changes to the positioning, so that it could possibly proceed to be monetized:
Community comment threads have marveled in recent times that Civit has been allowed to host celebrity likenesses. Aware of the chance, perhaps inevitability of a clampdown, a lot of initiatives to preserve LoRAs either removed by Civit or by their uploaders, have been proposed or implemented, including the (until now) somewhat neglected subreddit r/CivitaiArchives.
Though many have suggested that a torrent-based initiative is the natural solution, no well-followed domain seems yet to have emerged – and in any case, this might seem certain to maneuver activity banned at Civit and elsewhere to the outermost margins of the web; to walled gardens; and, most certainly, to the dark net, since a lot of the frameworks that would accommodate banned likeness LoRAs (corresponding to Reddit and Discord) either already ban such content or seem certain to ban it imminently.
In the mean time, celebrity LoRAs can still be seen with some restrictions at Civit, though most  of the generated content has been de-listed and shall be excluded from casual discovery. What seems likely, one commenter suggested to Nicoll within the Twitch session, is that the crackdown will deepen (presumably to the extent of banning all likenesses of real people in uploaded models or depictions).
Nicoll responded:
Despairing of the alternatives offered to Civit, Nicoll added:
Nicoll said that Civit had reached out to ‘every payment processor conceivable:
Where Next?
Prior to this announcement, Civit had been observed to be removing uploads covered by a few of the categories and sorts of content which are now banned. On the time of writing, an ’emergency repository’ for Wan 2.1 LoRAs has been established on the Hugging Face website. Though a few of the LoRAs archived there are designed to facilitate general sexual activities which are scantly-trained or else absent in recent video models corresponding to Wan 2.1, several of them fall under the now strictly-banned ‘undress’ category (i.e., ‘nudifying’), including some models that could possibly be argued to be ‘extreme’ or manifestly potentially offensive.
The subreddit r/datahoarders, which has been on the forefront of preserving online literature of the US government under Donald Trump’s mass-deletion campaign, has to this point shown contempt for the concept of saving lost CivitAI content.
Within the literature, CivitAI’s easy facilitation of NSFW AI generation has not gone unnoticed. Nonetheless, certainly one of the most-cited studies, the 2024 paper , is hamstrung by the indisputable fact that Civit has not allowed celebrity or illegal AI generations to this point, and by the researchers’ determination to search out their evidence at Civit itself.
Clearly, nonetheless, what concerns payment processors shouldn’t be what’s being produced with LoRAs at Civit itself, or what’s being published there, but what’s being done with these models in communities which are either closed or generally less-regulated.
The Mr. Deepfakes website, which was synonymous with the prevalent autoencoder-based method of NSFW deepfaking, until the appearance of Stable Diffusion and diffusion-based models in 2022, has recently begun to post examples of celeb-based pornographic videos using the most recent wave of text-to-video and image-to-video generators, including Hunyuan Video and Wan 2.1 – each very recent releases whose influence is nascent, but which seem set to garner incendiary headlines as their respective communities develop over the course of this 12 months.
Mandatory Metadata
One interesting change apparently being demanded by the payment processors, in line with Nicoll, is that each one images on the positioning must now contain metadata. When a picture or video is produced by a generative model in a typical workflow on a platform corresponding to ComfyUI, the output generally accommodates metadata that lists the model used (its hash in addition to its name, in order that in case the model is renamed by a user, its provenance stays clear) and multiple other settings.
Due to these hidden data points about how the image was made, users are capable of drag a video or image made by another person into their very own ComfyUI workflow and recreate the whole flow, and address any missing dependencies (corresponding to models or components that the unique creator had, which the user will then must locate and download).
Any image or video generations lacking this data will, Civit has announced, be deleted inside thirty days. Users may add such data manually, by typing it in on the Civit website itself.
Because the value of metadata is (presumably) evidentiary, this stipulation seems somewhat pointless; it’s trivial to repeat and paste metadata from one file to a different, and invention of metadata via a web-form makes this recent rule a bit baffling.
Nonetheless, several users (including one commenting within the Twitch session) have many hundreds of images uploaded at Civit. Their only recourse now’s to manually annotate each of them, or else delete and re-upload versions of the pictures with added metadata – which is able to erase any ‘likes’ or ‘buzz’ or conversations that the unique images generated.
The Latest Rules
Listed here are the summarized changes applicable at Civit from today:
- Content tagged with real individuals’ names or identified as real-person resources will now not appear in public feeds.
- Content with child/minor themes shall be filtered out of feeds.
- X and XXX rated content that lacks generation metadata shall be hidden from public view and flagged with a warning, allowing the uploader so as to add the missing details. Such content is not going to be deleted but will remain visible only to its creator until updated.
- Images made using the Bring Your Own Image (BYOI) feature must now apply at the very least 50% noise alteration during generation. This implies the AI must significantly modify the uploaded image, reducing the prospect of generating near-exact replicas. Nonetheless, images created entirely on CivitAI or remixed from other CivitAI content usually are not subject to this rule and may still use any denoise level, from no change in any respect (0.0) to full transformation (1.0). This variation is meant to cut back abuse of the BYOI tool, which could otherwise be used to provide subtle or undetectable deepfakes by barely altering real images. Forcing a minimum 50% change ensures the AI is not just calmly editing an existing photo of an actual person.
- When browsing with X or XXX content enabled, searches for celebrity names will return no results. Combining celebrity names with mature content stays prohibited.
- Advertisements is not going to appear on images or resources designed to duplicate the looks of real individuals.
- Tipping (Buzz) shall be disabled for images or resources that depict real individuals.
- Models designed to duplicate real people is not going to be eligible for Early Access, a Civitai feature that lets creators release content first to paying supporters. This limits monetization of celebrity or real-person likenesses.
- A 2257 Compliance Statement has been added to make clear that the platform doesn’t allow any non-AI-generated content. This helps ensure legal protection by affirming that each one explicit material is synthetic and never based on real photography or video.
- A brand new Content Removal Request page allows anyone to report abusive or illegal material without having to log in. Registered users should proceed using the built-in reporting tools on each post. That is separate from the present form for requesting the removal of 1’s likeness from the platform.
- CivitAI has introduced a brand new moderation system through a partnership with Clavata, whose image evaluation tools outperformed previous solutions corresponding to Amazon Rekognition and Hive.
Â
* I Dream of Jeannie
â€
