The recent hubbub surrounding ChatGPT-3 and other AI's like Elicit and SciSpace have caught my attention as it has many other librarians around the world.
Questions swirl around social media asking, "will this replace the need for teachers?" or "Is the college essay dead?" Twitter is full both of critiques of the AI systems out there, but also is increasingly showing hundreds of Tweets where educators share how they have been using ChatGPT and other AI in their classrooms. From using ChatGPT to help students write their resumes, to synthesizing the effects of hurricanes (for a 5th grade classroom), teachers are sharing some astonishing ways that the AI is aiding them in their teaching.
But before we get to that, I want to make sure to highlight some of the limitations.
The Limitations of ChatGPT and AI
Karen L. McKee posted a YouTube video about ChatGPT writing scientific research papers. Early on in her video, she stumbled upon one of the first issues ChatGPT has with doing this: the lack of citations in an informational block of text. When she asked if ChatGPT could provide the citations for the writing, it said, "I apologize, but as an AI language model, I do not have access to external sources or citations. The information I provided is based on my understanding of the topic and is not derived from specific sources. However, you may be able to find more information on this topic in scientific literature or online resources." The fact that it tells us that it cannot cite because it doesn't have access to external sources is a HUGE red flag. How then, can we believe any of the information it is sharing with us? And indeed, how then can students use that information in their own work?
John Royce, in his recent blog article, "Here we are again!" goes a step further and demonstrates that even when ChatGPT does provide citations for its output, in many cases, these citations are completely inaccurate and lead to articles that don't exist:
Interestingly, as I was researching and putting together a presentation for my Middle School faculty, I noticed that (at the time of writing) ChatGPT was no longer providing "citations" as the other blogs and articles I was reading had mentioned. Instead I got the following:
As you can see above, no citations were included in this (meaning that if a student were to use this, it would be a clear flag as I always require my students to cite their sources). So I then prompted ChatGPT to add citations to the paragraph:
Another teacher I was sitting in the room said, "ask it for references instead". I thought this wouldn't provide much back as ChatGPT specifically said, "I am not able to provide citations or reference to sources" but I thought I'd try:
So it can reference texts, but not online sources (which, let's be honest, 99.99% of our students [should] use for their research/writing). One of the pieces of advice I gave to our faculty when they asked how we can "catch" AI-generated work, was to check the citations (as they always should do) and make sure they are clicking on the URLs, but also to see what kinds of books were being references. If all the sources listed were books, then chances are it wasn't their work (unless that was part of the assignment), and then also to check the library to see if the school even had those books that the students might have been able to check out.
If ChatGPT can't search the internet,
how can it helps students with their research?
Again, I find error in this as it clearly tells me that it can find articles and websites but it has also told me it can't. So I ask again and am amazed by the result!
How great that it is telling us to use academic search engines, library databases, and journals?! And hey, maybe the kids will actually listen if an AI tells them to do it? Hah!
I think one of the most useful ways students can use AI is as a starting point: writing an essay outline that they can then expound upon; writing an introduction paragraph to an essay that they can then tweak and add to; provide test/quiz practice questions; give feedback on an essay; check a piece of writing for grammar/punctuation; explain topics that they don't understand:
"Catching" AI-Generated Work
As I mentioned earlier in this post, I have had a lot of questions from the faculty about how they can identify or "catch" AI-generated work that their students hand in.
So I went straight to the source and asked ChatGPT:
Many of these methods won't work for your average teacher. They need something quick and easy, something tangible that they can use/do in the midst of their busy schedules. So here is the list that I put together:
Interview: Students can be interviewed about their writing process, the sources they used, etc. this can give you an idea of the student's understanding of the subject and the effort they put in the essay. (Taken from ChatGPT)
Check the sources/links included in the work. Are they live/legitimate links? Are the citations formatted correctly? Along the same lines, check to see if the whole document is formatted properly (MLA/APA whatever your school may use).
Are the dates/facts/figures/names correct? I've read a few articles about how these have been found incorrect in some of what ChatGPT produces. Karen L McKee, in her YouTube video I mentioned earlier, demonstrates in her video how ChatGPT can also include inaccurate information, and when told that it has done so, will accept the new information provided without asking her for proof or citations. This reminds me of the dawn of Wikipedia, when anyone could go in and change articles, and is something that we must caution our students about when sharing this resource with them.
Similar to point #1, is it written in the same voice/style as the student's other work? As teachers we know what kind of work our students are capable of producing based on what they have handed in before. We know, if we suddenly receive something with a significant increase in academic language, better organization, etc., then something is probably going on.
There are a few AI detectors out there that you can also use, although I didn't initially have the best initial luck with them. My husband had ChatGPT write a children's story about our son and GPTZero found that there was only a 1% chance it was written by AI. I then put in a paragraph about the lasting effects of Agent Orange and all three of the following detectors caught it as being AI generated.
How can we use AI in the classroom?
Not all AI is bad AI! Despite the limitations and potential for cheating, I believe the benefits of ChatGPT and other AI far outweigh the negatives. The main positive is that it can make our lives as teachers much easier! So how can we use AI in the classroom? Eric Prochaska recently published an article that seeks to answer this exact question. He lists rhetorical analysis, peer review, revision, class presentations, and writing refinement as exercises that teachers can use with their students. In addition, AI like ChatGPT can help you:
write discussion prompts
produce quiz and test questions/answers
compose exemplars for critique
generate lesson plans (they're not half bad!)
create study guides
practice information literacy (have students evaluate a ChatGPT answer using the CRAAP test or other similar methodology)
help students generate essay outlines
and more
In fact, Kevin Roose for the the New York Times states,
"Creating outlines is just one of the many ways that ChatGPT could be used in class. It could write personalized lesson plans for each student (“explain Newton’s laws of motion to a visual-spatial learner”) and generate ideas for classroom activities (“write a script for a ‘Friends’ episode that takes place at the Constitutional Convention”). It could serve as an after-hours tutor (“explain the Doppler effect, using language an eighth grader could understand”) or a debate sparring partner (“convince me that animal testing should be banned”). It could be used as a starting point for in-class exercises, or a tool for English language learners to improve their basic writing skills. (The teaching blog Ditch That Textbook has a long list of possible classroom uses for ChatGPT.)"
So what next?
How are you using ChatGPT and other AI systems in your teaching? How are your students using it? I would love to connect with you on Twitter or email about anything related to these questions. Please feel free to reach out!
Email: haleykemper219@gmail.com
Twitter: @haley_kemper
Interested in viewing the presentation I shared with my Middle School teacher team? Find it here.