In 2019, my friend and colleague Grant Otsuki started experimenting with Generative AI (GenAI) – GPT-2 by OpenAI – to see how well it did at writing an essay for a first year Cultural Anthropology class he taught. As he wrote in a 2020 article for The Conversation, he trained GPT-2 on actual essays his students had written in response to the essay prompt, then worked with it to generate its own essay. I remember reading the work generated by GPT-2 and thinking that although I wouldn’t have awarded it a passing grade (for reasons Grant writes about in his 2020 article), it had potential and that a student could have turned it into a C-grade essay in less than 10 minutes. I also remember thinking that if Grant hadn’t told me it was generated by GPT-2, I probably wouldn’t have been able to tell that it hadn’t been written by a student.
It is no exaggeration to say that GenAI technology and use has exploded since Grant wrote his article, where he argued that we should be teaching students how to work with GenAI tools rather than ignore or try to ban them. Te Herenga Waka-Victoria University of Wellington, where I work, recently developed a policy on Student use of artificial intelligence tools to help write assignments that directs students to check in with their instructors about whether they can use GenAI:
Take time to understand expectations around the use of AI
Expectations may vary from course to course or even from assignment to assignment. This may include rules around how you can or can’t use AI, and what kinds of AI it is acceptable to use. For example, it might be ok to use translation software, but not generative AI like Chat GPT. Your course coordinators should make it clear when you can and cannot use AI and if there are any limitations on how you use it. If you’re not sure, just ask.
(https://www.wgtn.ac.nz/students/study/exams/academic-integrity/student-use-of-artificial-intelligence, accessed 24 November 2024)
This policy offers teaching staff a lot of flexibility and accommodates people like me, who want to bring GenAI into our classrooms so students can learn how to use it while discussing some of the concerns around its use (including its environmental impact and the looming issue of reliable information only being available to those who can afford it).
Class activity: Grade a mini-essay generated by ChatGPT
Earlier this year, I decided to adapt Grant’s activity for my undergraduate course Anthropology, Education, and Social Change. The first assignment for that course asked students to write two mini-essays of 500-600 words about different concepts and/or theories that were discussed in lectures and assigned reading material. I asked ChatGPT (version 2, by OpenAI) to generate a 500-word mini-essay on the topic of “grades” for a course on anthropology of education, that drew on and cited Susan Blum’s book “I Love Learning; I hate School” An Anthropology of College, which we read and discussed in class. I also asked ChatGPT to adhere to the assignment instructions (below) and spent a bit of time editing the mini-essay it generated.
In Week 4 of our course, I brought printed copies of the mini-essay that ChatGPT generated to class and asked students to read it and assign it a grade using the marking guide for this assignment. I had two goals in mind with this activity: 1) to encourage students to think critically about GenAI use in the classroom; and 2) to encourage students to engage with the marking criteria that I was going to use to assess their work.
We began the activity by discussing the following:
- how they can use GenAI in my class (e.g. to refine their ideas or fix errors with spelling, grammar, and referencing)
- the University’s policy on using AI to help write assignments
- the limitations of using GenAI (e.g. hallucinations)
- some of the ethical, environmental, and privacy considerations involved in working with GenAI
- how to acknowledge their use of GenAI by including an Acknowledgement Statement at the start of their work (our Library provides information about how to cite AI use)
After that, I handed out Chat-GPT’s essay (below) and asked students to give it marks and written feedback. They had the option of doing this individually or in pairs/small groups, and I invited them to share their marks and feedback to our class Miro board.
After about half an hour to mark the mini-essay, we shared our marks and comments with one another. The students – who, for the most part, failed the mini-essay with very critical feedback – were surprised to learn that I had given it a C- grade, which prompted a constructive conversation about the kinds of grading-related issues that Blum writes about in her book and that we had discussed in earlier lectures. It also provided a good entry point for me to discuss why and how I had designed their second assignment using task-based grading (the term I now use to describe my version of labour-based grading). Finally, several students said that they appreciated the opportunity to grade an assignment like this because it helped them better understand the marking criteria.
This activity achieved my goals and I was pleased to see students acknowledging and citing their use of GenAI in later assignments. I also noticed a slightly higher grade distribution for Assignment 1 compared with the previous year. While this approached worked well in my classroom, I know there are many other innovative and meaningful ways to integrate GenAI tools into teaching and assessment practices. If you are working in this space, I am keen to hear about your experiences!
Published by
Published
