OpenAI confirms if AI writing detectors actually work or not

Since the emergence of ChatGPT, many students have taken advantage of the new software to turn in papers and reports.

2

Educators quickly caught wind of the new internet phenomena and adopted many tools that all fell under the umbrella of "AI writing detectors". Some educators copied and pasted work back into ChatGPT and asked the AI if an AI generated the work. Answers varied, with ChatGPT sometimes would reply with certainty and other times it would give an approximation. However, both answers were wrong, as the underpinning language model wasn't designed to detect AI-created content.

Educators turned to other services that were offering AI detection tools, which resulted in some students even being "caught" turning in AI-generated content. While there are certainly some students taking advantage of the new technology to make writing easier, none of these AI detection tools or services are legitimate, at least according to OpenAI, the creators of ChatGPT. In a recently updated FAQ found on the company's website, OpenAI answers the question "Do AI detectors work?", with "In short, no."

More answers to the question "Do I detectors work?"

• While some (including OpenAI) have released tools that purport to detect AI-generated content, none of these have proven to reliably distinguish between AI-generated and human-generated content.
• Additionally, ChatGPT has no "knowledge" of what content could be AI-generated. It will sometimes make up responses to questions like "did you write this [essay]?" or "could this have been written by AI?" These responses are random and have no basis in fact.
• To elaborate on our research into the shortcomings of detectors, one of our key findings was that these tools sometimes suggest that human-written content was generated by AI.
• When we at OpenAI tried to train an AI-generated content detector, we found that it labeled human-written text like Shakespeare and the Declaration of Independence as AI-generated.
• There were also indications that it could disproportionately impact students who had learned or were learning English as a second language and students whose writing was particularly formulaic or concise.
• Even if these tools could accurately identify AI-generated content (which they cannot yet), students can make small edits to evade detection.

To read more about OpenAI's FAQ on AI detection, check out this link here.