Case study: Integrating Generative AI within an essay-based assessment on the year 3 module ‘India and Its Neighbourhood’.
Department of Politics
Indrajit Roy
Indrajit Roy describes a pilot in which AI was integrated into the assessment for a third-year politics module 'India and its Neighbourhood'. Indrajit required students to write traditional essays, then to engage with Gemini on their chosen topic and incorporate a short critique of the output. An integrated introductory session was delivered by the Digital Skills Team to support students. The AI critique was not marked directly to ensure that it would not become overly daunting for the students. Indrajit points to benefits of the exercise in encouraging critical engagement with Generative AI.
Watch their presentation:
Integrating Generative AI within an essay-based assessment (Panopto viewer) (8 mins 31 secs, UoY log-in required)
Transcript
So, uh, this was, as Rob, you mentioned, the idea was to have, uh, AI integrated within an essay based assessment. Um, and this was for third year students. Um, the principle was really that several principles around this. Um, the starting point was curiosity. Um, I'm neither, uh, techno obsessed or, uh, techno pessimist. I'm just techno curious. So I was just, um, you know, curious about how, um, students might or might not deal with AI. Um, and, uh, especially amongst all the onslaught of artificial intelligence, um, how, you know, things are likely to find out. Um, I thought it's just very important to reclaim our humanity. Um, and, uh, that involves, uh, teaching students, uh, the way to sort of really strengthen their critical thinking as well as, um, this entire, um, uh, you know, endeavour was collaborative, so to work with other colleagues. Um, and finally, because I was teaching, I teach the politics of India and its neighbourhood. Um, uh, talking about artificial intelligence was quite central. Um, information technology is quite central to what's called the India story. Uh, one here is about digital workers in India is political economy, but also, of course, about misinformation and what it does to democracy more generally, but also to Indian politics in particular. Um, if you get a feedback from what I'm saying, do let me know. I might sort of, uh, have to tune things a bit, but is that all right? Can can people hear me? Okay. Great.
So now the specifics was really it was an essay based, um, uh, sort of, you know, incorporation of AI. Um, we were very clear that, uh, students would write an essay, but within the essay there was an element of critically analysing what, uh, Gemini would say about an essay topic. So topics were assigned as normal and students would have to, um, of course, answer the topic, addressed the topic as they would address, uh, you know, uh, any, uh, uh, and then they would, um, enter the, uh, essay topic into Gemini and then analyse the, the response that Gemini would, uh, produce. Um, and then you can see the, you know, steps were very categorically sort of laid out for them and we, uh, sort of, uh, taught them, told them how they might, uh, critically analyse their response.
Uh, and I think you should look at, uh, point four if you wanted to sort of get more, more information on exactly what it is that students did. So they would answer the question as they would answer any essay, but incorporate one paragraph just before the conclusion on how they analysed, how they thought about the response that Gemini gave to the question they were answering. The essay titles are on the bottom right, so you can see the essay titles draw entirely from the politics of the South Asia, India, the neighbourhood, um, and uh. The students would address these topics using their allocated readings, uh, using the argumentative sort of, uh, you know, approaches that, uh, politics students are expected to, um, uh, sort of have um, and uh, the additional, uh, element was that they would, uh, engage with, uh, Gemini and uh, have uh, paragraphs about 100, 125 words that would, um, uh, you know, that they would then uh, and uh, would sort of see what Gemini's response was. Um, and here I've just put a snapshot of, uh, what students did. So on, on my left, uh, you know, there's one student who was looking at, uh, Democratic politics and economic development and how it affects ethnic conflict. Uh, and so they wrote the essay as they would write it, to write an essay, and then they incorporated that, uh, they fed that prompt into Gemini. And then this is the one paragraph that they analysed Gemini's response to the question. And you can see how they are sort of critically considering what it is that Gemini has thrown up in response to the question. Um, and then they sort of look at how, you know, the Gemini response doesn't consider, uh, counterarguments of, uh, things like postcolonial legacies and overdeveloped states, concepts that were taught in the module. Um, likewise, you can see on the top right, uh, there was a the question was on electoral votes on Hindu nationalism. And, uh, the student, uh, said that into that question, into Gemini. And then this is their response that this is their sorry, this is their analysis of Gemini's response. And again, sort of talks about excessive nuance, uh, that Gemini produces. And finally, you can see this student, uh, sort of looks at, uh, Gemini's response to the question of, uh, India's, uh, emergence as a rising power. And, uh, you know, uh, she also lays out the pros and cons of relying on Gemini. Um, relying is the wrong word. Of course, they weren't relying on it, but of using Gemini.
Um, so for us, the key learning across, uh, this, uh, pilot was that there's a great deal of excitement and nervousness, of course, on the use of AI, uh, amongst both, uh, colleagues as well as students. But I think it's key to avoid, uh, due to all of this is not to make it too daunting. Uh, it's a small part of the, uh, essay and an essay. Something that they are, uh, very familiar with. So it's, um, just important to integrate it, uh, and within any. But it's something that's familiar and not make it too daunting. Um, in this particular case, it was, I think, a useful tool that it was integrated within the module's design and taught components. So we did have a lecture and a session on, uh, the digital politics in India and its neighbourhood, how misinformation was spread, what it did to politics. And I think it sort of, uh, laid the foundation for students to, um, think about AI and think about it critically. We did have, uh, dedicated session on the use of AI in assessments, thanks to our colleagues in the digital education team. Uh, and that was massively appreciated. Uh, and people said students said this was important well beyond the assessment. And finally, the step by step instructions. Uh, I thought, uh, essential. Uh, Rob, uh, was, uh, sort of quite helpful in, uh, but, you know, fine tuning the words. Obviously the responsibilities was mine, but it was just very good to have clear step by step instructions, which students, um, uh, sort of, uh, valued. One of the things we didn't do was to specifically assess the quality of, uh, students analysis of, uh, the AI response. Um, and I think that's okay because we don't want we are not sort of, uh, assessing them, uh, on, on, on just that component. And I think that was part of the whole thing about not making it too daunting. Um, but anyway, that's where we are. It's the first year. And any feedback, questions, comments will be very welcome. Thank you.
Guides related to this case study