ChatGPT and Generative AI Platforms in the Classroom: What is ChatGPT?

The past year has seen a wave of artificial intelligence writing applications surface, with ChatGPT being by far the best known. These "large language model" programs can quickly generate essays, papers, and other written material based upon prompts given by their users. Like it or not, these AI writing apps are here to stay and have the potential to seismically alter higher education, so we need to be prepared to deal with them and, where appropriate, incorporate them into the classroom.

Do AI generative platforms signal the end of writing as a means of assessment in education? Some observers have suggested so, while others have compared it to the introduction of the calculator or spell check; after all, the calculator did not make math obsolete. Yet, ChatGPT certainly presents educators with more challenges than did calculators and spell check software, and this guide is intended to provide ideas and resources to help deal with, and make use of, ChatGPT and its fellow AI generative programs. 

College Libraries has a student-centric AI guide, specifically focused on how to cite ChatGPT and other generative platforms, which can be found at: AI: Citing ChatGPT and Other Generative AI Platforms. 

ChatGPT and other AI writing applications are essentially overpowered predictive text programs, operating similarly to the predictive text feature on smartphones. But while a smartphone only looks back at the last few words to suggest what should come next, the current generation of ChatGPT is capable of looking back at the last 750 words, and does so extraordinarily quickly. 

Furthermore, these generative AI apps have been loaded with enormous amounts of information--literally hundreds of billions of words in the case of ChatGPT--through massive webcrawls (hence the "large language model" designation). When it launched late in 2023, ChatGPT held the equivalent of a person reading one hundred 90,000 page books a day for 80 years.

AI writing applications have been around for years, but ChatGPT's combination of this immense well of information and its powerful predictive abilities represented a quantum leap forward over earlier apps. And given the $10 billion investment Microsoft recently made in Open AI, not to mention the proliferation of other large language models (Google recently launched its own version, called Bard), such generative AI platforms will likely only become more common and more capable.

However, we should be very clear: "Artificial Intelligence" is just a marketing ploy. ChatGPT, and other applications like it, are not actually intelligent. They have no real knowledge and are incapable of original thought or creativity. ChatGPT is just really, really good at pattern recognition, enabling it to drawi from its well of information and pick the next word in a sentence. Here is what Open AI's CEO has to say about his company's killer app:

ChatGPT is incredibly limited but good enough at some things to create a misleading impression of greatness. It's a mistake to be relying on it for anything important but a preview of progress. We have lots of work to do on robustness and truthfulness.

--Sam Altman, Open AI's CEO

Beyond its innate lack of creativity, ChatGPT has other inherent flaws of which we should be aware.

• It is essentially unthinking and amoral.
• It can be racist and sexist, since it is simply drawing upon the information loaded into it, much of which came from internet crawls.
• It can include misinformation or disinformation into its text that, once again, it simply drew from the internet.
• It can give wrong answers to obvious questions, but will do so with utmost false confidence.
• It can engage in "popularity contests," in which the more often something is entered into the AI, the more it will think that thing is true, regardless of accuracy.
• It can ignore negative prompts, including information in its text that the user specifically asked to be excluded.
• It may make up information it does not know, referred to as hallucinations.
  • This is of particular importance with citations. If asked to include references in an essay, ChatGPT may simply invent nonexistent sources, going so far as to use a real scholar's name and a real journal's title but creating an article that scholar never wrote or journal ever published.
  • At present, checking references is one of the most effective means of determining if a paper was written by a student or an AI. But newer models may enable AI applications to quickly search for, and incorporate, actual sources into its text.
• OpenAI and other companies did not get creator permission before conducting the internet scrapes that loaded information into their large language models, resulting in some authors and artists undertaking legal action for their copyrighted material essentially being stolen & resold through AI apps.
• OpenAI is a for-profit company, does not make public the code behind ChatGPT, and could put the entire system behind a paywall at any time.
  • Currently, the free version of ChatGPT operates off the GPT-3.5 engine, while users can pay for a premium version that uses the newer GPT-4 engine.

A recent study by researchers at Stanford University found that between March and June, 2023, GPT-4 became progressively much worse at answering the question of whether 17077 is a prime number. In March, it was correct 97.6% of the time, but by June, that number had plummeted to 2.4%, apparently due to changes Open AI had made in the way GPT-4 processed questions.

And while ChatGPT is capable of writing college-level essays, it currently doesn't appear capable of writing particularly outstanding ones. When professors in Durham University's Department of Physics asked ChatGPT to answer standard short-form (300 word) essays, the AI-written essays received passing grades but none were higher than a 76/100, with a mean score of virtually the same as the overall student body. In other words, at present, ChatGPT is essentially a C student.