ChatGPT and Generative AI Platforms in the Classroom: Methods for Detecting AI

Currently, ChatGPT and other generative AI platforms tend to write in a bland, generic style and often produce text that isn't quite human, while human writing tends to be more contextually aware. This may change as AI technology develops, but at present, checking for this bland text that does not resemble the student's own personal writing style is one means of determining if an assignment was written by a student or ChatGPT.

Additionally:

• Check for originality and creative thought, things ChatGPT is incapable of.
• Check for grammar and spelling. AI apps don't make spelling or grammar errors, humans do.
• Check sources and citations. When asked to include references in a paper, ChatGPT will often simply invent a source that does not actually exist.
• Look for patterns of irregularity in language. These may represent jumps between text written by the student and that written by an AI.

But recognize that Open AI and its competitors are constantly working to improve their platforms, so future versions may eliminate many of these telltale signs of AI writing.  

GPTZero, Open AI's GPT-2 Output Detector, and Writer.com's AI Content Detector are applications designed to differentiate AI-written text from human-written text. While college libraries have tested all three and found them to be generally accurate, other tests of the detectors have resulted in false positives, particularly when confronted with text written by ESL students. College Libraries recommends faculty conduct extensive tests prior to using any of the detectors in the classroom, in order to determine which you find most accurate.

Also note that, at present, plagiarism detectors such as Grammarly cannot detect AI-written text.

GPTZero has both free-to-use and premium platforms. The free platform is limited to 5,000 words, while the premium "Educator Plan" ($9.99/month) raises the limit to 50,000 words and claims it is "trained for student writing and an ed-tech usecase."

Open AI's GPT-2 Output Detector is completely free (at present), and is a paste field that judges text on a simple Real-Fake spectrum. Open AI is the company that created ChatGPT, although GPT-2 is now two generations behind their current model (GPT-4), and at present there does not appear to be an output detector for either GPT-3.5 or GPT-4.

Writer.com's AI Content Detector is another free application, limited to 1,500 words. It appears to be aimed at writers wishing to mask their use of AI, as a poor score results in the following feedback "You should edit your text until there’s less detectable AI content." Because of this, it may be less useful for faculty than the other applications, and is included here to inform of its existence.