Case study: Mentimeter for Q&A and review lectures in Computer Science

Department of Computer Science
Lilian Blot

Lilian describes his use of Mentimeter to support review lectures in a large-group cohort first year introductory programming module. Students come to the module with a wide range of previous experience leading to challenges pitching the module contents and, in particular, engaging students who may have already achieved some of the module learning outcomes.

Lilian has applied a ‘flipped classroom’ approach since the Covid-19 Pandemic, providing content via recorded lectures chunked into 15 minute videos, with in-person sessions taking the form of practical sessions in labs, seminars and, at the end of each week, a ‘review lecture’ focusing on the recorded content. His use of Mentimeter allow him to make the review lectures more interactive and focused on student needs through: - Introductory warmer questions focused on the week’s content and feedback from the teaching sessions - Quiz questions starting with key content before moving on to more challenging areas dealing with common misconceptions. Lilian provides further input on these based on the results before providing similar questions to allow students to check their understanding once again. - ‘What would the result of this algorithm be?’ questions allowing for a focus on application. Lilian described how this leads to a lively buzz of activity in the lecture theatre as students discuss their responses. - Open Q&A sessions allowing students to anonymously ask questions about the module content and assessment.

He has attempted to gauge the effectiveness of this approach by measuring engagement throughout the module and interrogating end of module evaluation data. Student engagement with the Mentimeter quizzes was high throughout and the number of students asking questions increased. The module achieved high satisfaction scores and numerous open comments were received focusing on Mentimeter and the approach in general.
Lilian concluded with observations that the approach has been time consuming to initiate but that the investment is paying off in ongoing efficiencies for staff combined with high satisfaction ratings from students.

Watch their presentation:

Mentimeter for Q&A and review lectures in Computer Science (Panopto viewer) (11 mins 37 secs, UoY log-in required)

Transcript

It's something I took from a presentation I've done for the teaching away day in the Department of Computer Science, because we were looking at how we can create an effective review lecture. So yeah, I've been the one using mostly Mentimeter for the review lecture so it's a large-group cohort. And here's the structure of the teaching we had for the last five years for my module. It's a first year module and the cohort is quite a complicated cohort over here because three quarters of the students already knows the material I'm going to teach them almost because they've done A-level computer science and they've done the first module of the programming. This module is made for beginners who've never done it so three-quarters of the students are going to have a mark above 80% on that exam at the end. So they know almost everything. So it's very challenging for me to keep them entertained or engaged with the different videos.

So since the pandemic I've been able to do what I wanted to. So it's a flipped classroom, so the lectures are recorded, so they've got 15 minute videos, two or three per week and then they've got practical sessions, they've got some seminars, in-person teaching, and then at the end of the week we've got a review lecture which is using Mentimeter So during that lecture I've got about 200 students know that they have to check in it's compulsory, but I've got a large number of students and the idea is to make them participate. Before a lecture and I have again about 200 students and when I'm asking for participation out of 200 students I've got four students answering the questions, so that's not really engagement from the other students and that's something I wanted to change.

I was thinking about using clickers, but it wasn't available and manageable. So when I've written my lecture, I've got the core content I want to use and when I'm thinking about the core content, the first thing I'm going to ask is can I put a quiz in it? Can I put multiple choice question? Can I do interaction that is going to guide the teaching throughout? And then I got some feedback that I've got from practical sessions, I can add introduction or icebreaker or to check how they are doing during the lecture as well. And I've got a period which is very important which is question time that is open ended questions you can ask me during the lecture and I can interact with them about what they learned during the week or what we just covered during the review lecture. And then I've got in case everything goes fine, I've got optional content to be able to continue my lecture if everything has been But rarely I manage to do all of that.

Okay, so maybe the core content. So my lecture is try to answer the core content. So an example of that is I'm starting with something we've seen in the video and I ask them a very simple question, kind of simple question, and they have managed very well for once. And 97% of my cohort was able to answer that question. But on the same principle for the same thing about what we call miscomprehension programming, there is a few details which are very challenging and they need to learn about. So therefore, I'm going to have a follow up quiz with something which is a bit more challenging, a concept they may not have understood properly. And you can see that they've got 99% of the question right for this one. Okay. Most of the cohort went into the misconception. That's why you get quite a widespread error, because that's a common misconception students have on that subject.

So having experience in teaching that makes a quiz much easier to build because I know what kind of answers I need to provide. And I can put that in the question. And then when I've done that, now I can explain what I'm missing and so on, and then I can have a follow up question with a slight variation and see if they answer better. And here now I've got half of the classroom understanding a bit better that concept, still looking at something a bit different, but they start to build on the knowledge they've got. And then I can add a final one where I've 65%, so here I can kind of assess the effectiveness of my teaching on that specific subject.

Okay. And they are interacting. So what is important here, for example, I've got about 69 students who have answered the most difficult question of all. This one is very difficult - it's not because the question is easier is because they start to understand what I try to explain to them in the lecture. So he's very lively during that time. And they can talk together. They've got plenty of time. They've got 1 to 2 minutes to answer the question. To understand the question and to be able to discuss. And then to answer the question. So that's the way I do my teaching.

So the question is, is it effective or not? So what do I mean by effective? The first thing I would mean by effective is are they not asleep? do they participate in my lecture? Are they interested or not? And especially I've got a lot of students which already know most of it okay and some students, probably have answered all my questions correctly from start to beginning, from start to end during the entire semester, because they've done that before, so I want to keep them engaged.

So what did I look for? I look at the feedback at the end of the module, so it's a questionnaire we send to students and I've got about 110 to reply. Okay, so the ranking of my teaching evaluation is about 4.2 to 4.3 out of five for my module being interesting and so on - and again being interesting is kind of a challenge because some of them, already know almost all the material here, so they still need to be engaged. And when you look at that what I expected was to see practical sessions because that's where the learning is going to occur not during the lecture but also they talk about lecture. So when they talk the lecture that means the review lecture and the interaction with Mentimeter we've got review which is a lecture review again.

So there is a lot of things they like about the review lecture. It comes over and over during the feedback of my module - there is other tools they quite like they like the video we got for the teaching before So it's kind of a combination of the two which is essential if you don't have very nice video to start learning, the review lecture will not be as efficient. So you need to learn the material before they can try to attend the review lecture So that's something which is quite interesting and again. Something you mentioned - interactive. What is the most important for me is probably that kind of feedback.

And here that's again taken from the module feedback and you can see that quite often they talk about Mentimeter. review from the live lecture. Okay. And again, what they like during the live lecture is looking at the code. Try to understand what the code is going to do, answer the code, giving the feedback automatically and to see where they sit into the entire classroom. And again, Gradescope, they mention that again, which is a very useful tool for them as well during the practical decision. So Mentimeter is a very important tool for my lecturing and for engaging all my students.

So how can I evaluate that as well? I can evaluate that student participation. So what you can see in red is the percentage of students who logged in in Mentimeter and answered the question on average for quizzes okay And you can see it is pretty high is about 80% you've got more questions about the content of the lecture. Here I get a spike in questions about assessment. And similarly towards the end, more questions about assessment at the end of the semester. But you can see a relation between the questions I've got from students related to the content, which is about 8 to 10, which is twice or four times as much as people asking a question during a proper lecture. So that's the kind of thing I'm looking for because here I can see the students are engaging and try to understand better and they do that because of anonymity, as you mentioned. No anonymity, no question, as simple as that. So anonymity is a key part of Mentimeter that I want to keep using.

So the staff perspective here very quickly, we had a teaching away day and here we can see that some people want to use review / Q&A session. We've got only 8 people doing that and we've got more people are planning to do so. And then we were looking at what kind of tools and I was talking about using Mentimeter. So we may have more and more people looking at a review lecture using Mentimeter.

Whilst I tried to use Mentimeter for seminars in computer science for the one that I was teaching, I didn't find a solution, it didn't work at all. Okay. So probably that's the way I use it. Probably I don't teach the seminars the way it should be for that kind of thing. For seminars, I haven't found a solution yet, so I need to keep looking.

What is the benefit for staff is when I record the lecture of the video recordings, They can pause it and try to answer the question again for themselves. But in addition, when I was off sick last Friday, I was about to publish the one from last year. So they can still have the same content, they can still have a question and try to answer that before they can see the answer. So they can still interact with the lecture in some way using the quizzes. Why we should keep it? The investment I've got as well, when I've got all the list of questions I can do that offline after when I don't have time to answer all the questions for my students. I've got after the lecture to prepare that. And when I've done it once the questions are mostly similar year on year. I can copy and paste from the VLE directly so I don't have to do that again and I can update some of them if need be for an assessment of because there is a set of misconceptions that I haven't seen before, but most of the work is already done and I don't have to do that again. So a lot of gain time in preparing for the following year.

So it's an investment which is costly at the beginning, but it's a reward at the end because you have less work to do to have something which is apparently still very good for the students, very good quality for the students. The feedback from students are very positive and they want to keep the review lectures with Mentimeter. And that's it for me. Thank you so much.