A teacher has used OpenAI's ChatGPT to try and figure out if his students were using the AI-powered chatbot to write their papers.
That teacher is Dr. Jared Mumm, a campus rodeo instructor, and agricultural class teacher at Texas A&M University-Commerce. Mumm sent out an email to students in his class that they would be getting graded based on their results for the last three of their assignments and that he would be giving everyone in the course an "X" as he had discovered that the students were using ChatGPT to write their papers. However, this wasn't the case.
Mumm writes in the email that he was copying and pasting student assignments into ChatGPT and asking the chatbot if it created the content. This was Mumm's way of vetting student papers for any traces of artificial intelligence-powered creation, saying that he tested each paper twice and even went on to offer the students a make-up assignment that would help them gain the appropriate grade to pass the course. However, this method for vetting student assignments simply doesn't work at all, as ChatGPT isn't an AI designed to detect content generated by AIs; it's a chatbot designed to imitate a human conversation.
ChatGPT wrongfully informed Dr. Mumm that many of his student's assignments were written by AI, and according to the fiance of a student in Mumm's class that took to Reddit under the pseudonym DearKick, their partner is slated to receive a failing grade on the course when they didn't even know what ChatGPT was prior to the accusations from Mumm. DearKick wrote on Reddit that their partner "feels even worse considering it's something she knows nothing about" and said that Dr. Mumm has flunked "several" whole classes through the same botched method.
To prove that ChatGPT isn't able to detect if the content is generated by AI and that the chatbot simply claims credit for things that it hasn't written, Redditor Delicious_Village112 took an excerpt from Mumm's doctoral dissertation on pig farming and used the same vetting technique as he uses on student papers, asking ChatGPT if the excerpt was written by it or another AI. ChatGPT replied, "Yes, the passage you shared could indeed have been generated by a language model like ChatGPT, given the right prompt. The text contains several characteristics that are consistent with AI-generated content."
To throw more salt in the wound that is Mumm's testing method, Redditors even took the leaked email written by Mumm and asked ChatGPT if it was written using AI. ChatGPT replied, "Yes, I wrote the content you've shared. If someone used my abilities to help draft an email, I wouldn't have a record of it."
Given Mumm's overall sentiment about ChatGPT and his due diligence-lacking method of failing students, it would be safe to assume that ChatGPT wasn't used to write his email - it simply highlights that ChatGPT isn't a reliable source for AI-checking student papers as its currently willing to claim almost anything is written by it, or could have been written by it.