Daphne Ippolito, a senior research scientist at Google specializing in natural-language generation, who also didn’t work on the project, raises one other concern.

“If automatic detection systems are to be employed in education settings, it’s crucial to know their rates of false positives, as incorrectly accusing a student of cheating can have dire consequences for his or her academic profession,” she says. “The false-negative rate can be vital, because if too many AI-generated texts pass as human written, the detection system just isn’t useful.” 

Compilatio, which makes considered one of the tools tested by the researchers, says it is crucial to keep in mind that its system just indicates suspect passages, which it classifies as potential plagiarism or content potentially generated by AI.

“It’s as much as the faculties and teachers who mark the documents analyzed to validate or impute the knowledge actually acquired by the creator of the document, for instance by setting up additional technique of investigation—oral questioning, additional questions in a controlled classroom environment, etc.,” a Compilatio spokesperson said.

“In this fashion, Compilatio tools are a part of a real teaching approach that encourages learning about good research, writing, and citation practices. Compilatio software is a correction aid, not a corrector,” the spokesperson added. Turnitin and GPT Zero didn’t immediately reply to a request for comment.

We’ve known for a while that tools meant to detect AI-written text don’t all the time work the best way they’re purported to. Earlier this 12 months, OpenAI unveiled a tool designed to detect text produced by ChatGPT, admitting that it flagged only 26% of AI-written text as “likely AI-written.” OpenAI pointed MIT Technology Review towards a piece on its website for educator considerations, which warns that tools designed to detect AI-generated content are “removed from foolproof.”

Nonetheless, such failures haven’t stopped corporations from rushing out products that promise to do the job, says Tom Goldstein, an assistant professor on the University of Maryland, who was not involved within the research. 

“A lot of them will not be highly accurate, but they will not be all an entire disaster either,” he adds, declaring that Turnitin managed to realize some detection accuracy with a reasonably low false-positive rate. And while studies that shine a lightweight on the shortcomings of so-called AI-text detection systems are very vital, it could have been helpful to expand the study’s remit to AI tools beyond ChatGPT, says Sasha Luccioni, a researcher at AI startup Hugging Face.

For Kovanović, the entire idea of attempting to spot AI-written text is flawed.

“Don’t attempt to detect AI—make it in order that the usage of AI just isn’t the issue,” he says.