Preventing AI Cheating on Quiz Essay Questions

[MonicaRanes-Gol]

What are best ways to prevent students from copy/pasting AI generated answers for quiz essay questions? Perhaps Canvas software can have a feature to disable copy/paste on the quizzes?

[chriscas]

Hi @MonicaRanes-Gol,

Aside from using an online proctoring or lockdown browser type system, which each have their own drawbacks and are not guaranteed to prevent cheating, my opinion is that this isn't really something that can be effectively blocked for students, teachers, or anyone else.  I know there are varying opinions circulating around the world right now, but I currently agree with those who are saying this may really spell the end of traditional online multiple choice, short answer, or even essay exams.  There are some suggested ways assignments can be redesigned to combat the pure use of generative AI, but they do require thought and work on the part of teachers, and I don't think it's even a "one size fits all" type of situation.

I can pretty confidently say, as a Canvas admin and computer science degree holder, that while your idea of locking out copy/paste sounds pretty simple and seems like it should be doable, the reality is that it would be very easy to get around and would probably have more negative unintended consequences than any actual benefit to preventing copy/paste form AI.

As I said, there are definitely varying opinions on this right now (and they are all subject to change in days or months ahead as things evolve).  I wanted to at least give you my opinion, but there may be others who chime in here and have a viewpoint that is 180 degrees different than mine, and I think that's okay.  We're all trying to figure this generative AI situation out, and I think you're already ahead of the game by thinking about options now!

-Chris

[TheresaHuff]

Should an affordable, effective tool that can detect AI come to be, is there a possibility of that detector being placed in Canvas Quizzes essay and short answer question types? My faculty would find this exceptionally helpful.

[DavidHildebrand]

I suppose if we pasted IMAGES of the questions, that would prevent copy/paste but it would also prevent ADA accessibility.

I teach an 40 person online class. It has quizzes. This semester, the average for Quiz 1 went from 79% to 88% and for Quiz 2 from 82% to 91%. I've never had averages like these or seen jumps like these between semesters. Something is up.

[TanyaMcMillian]

You think. it is very hard to teach with this AI.  The kids are not learning.  This is terrifying for our future.  

[LynzWilmsey]

The increase in copy and paste in our student's quiz answers has also increased the speed at which they complete the quiz. It has only started within the last 6 months, and most have been overseas students with English as a second language. In the absence of disabling "copy & paste" within the quizzes of Canvas, I am instructing our markers to be vigilant, when they find an instance of plagiarism, mark according to our policy and fail the student.  We decided that, if we mark this way and there are constant reminders of our policy, then this will hopefully dissuade the cheaters.  In our Training Institute, we manually cast our eye over the answers, AI is good but at this stage, it is pretty easy to spot, especially when foreign language students are using it. AI can word it but can not replace an individual's human point of view for understanding the subject. We have also been increasing the use of other assessing methods within Canvas quizzes, e.g. matching meanings with words, tables, fill in the blanks and essay answers that need a fuller explanation that comes from understanding rather than looking for direct answers to obvious questions. We are not re-writing all our quiz questions, just a couple that indicate to us that they are not understanding the learning material. Always welcome other ideas to combat this increasing problem. I have included a couple of snippets of obvious copy and paste responses. In pic 3, you can see that the answer does not match what the grading notes requires and this question should have been marked wrong.

[DavidHildebrand]

I have gone ahead and filmed the questions, leaving the answer choices as text. It radically reduced the cheating on the final.

[zelley]

Can you explain what you mean by stating that you "filmed the questions"? 

[DavidHildebrand]

I filmed myself with my webcam reading the question out loud and added captions on the screen as I said the words. So, when a student goes to the question, they get a small video of the question being read to them and then below they choose from the answers on the screen.

[krodda]

Hi, David--I'm going to try this. Do you have a time limit on your quizzes? Has this been impacted at all by your webcam questioning? Thank you!

[DavidHildebrand]

I do have a time limit, and he webcam questioning has worked well. A student who wants to cheat does not have sufficient time to transcribe the video and the answer choices for all the questions in the time allotted. Those who study have sufficient time. Those who want to use AI do not. My grade curve has returned to normal.

[krodda]

Hi, me again. Were you able to use the "record" button in the Quizzes tab when creating your quiz, or did you have to make a separate video and then download it or provide the link? I tried using the "record" function, but I couldn't figure out how to close caption myself reading the question.

Thanks--this is very helpful! 

[DavidHildebrand]

I created everything outside of Canvas -- just easier to do that way. For each question, I had a separate file (Question_1.mov, Question_2.mov, etc.) and the uploaded each one. Big PITA but worked easily and video quality was good.

[krodda]

Much appreciated. I will give this week's quiz this way--I'll be curious to see how different the grades are from when they could just plug them into ChatGPT. 😈

I'm so tired of dealing with rooting out AI. It's like I don't actually teach anymore.

[DavidHildebrand]

I hear you. "Please let me teach you something." Sounds kinda sad.

What if we just had a contract that said, "I hereby notify all that I will cheat on my assignments." Then they sign it and we say, "Fine, you get an A. Don't come to to class. More time for everyone else."

[TanyaMcMillian]

Can they  take a screen shot and put it into AI?

[LynzWilmsey]

I have been chatting to my Boss about this and the students are using AI to translate into their language to be able to understand the question. Then they just carry on using AI to get the answers and we can see it has been done this way as the answers are way more than what the question asked.

Our temporary solution is to give a word limit (which AI can get around) and have different quiz question style types that have - Good, Better, Best & accurate answers, using key words that are found in our learning material.  However we have to get our Education Governing Authority to accept this new way and they are a bit behind the times at best.😉

[DavidHildebrand]

Failure alert --  I have to let folks know here that students seem to be getting around my video-taped questions. I am looking into the Respondus Browser for the next exam.

I thought I was so clever and I put so many hours into video-taping exam questions, editing, uploading. I feel like a fool, honestly. So, at this point, some students seem to be using AI for short papers and online quizzes. They're getting an A (so far) and (apparently) not doing any of the work.

Sigh.

[Robert_Sprouse]

I teach at two separate institutions, one is strictly online and the other is a mix of face-to-face and online. The best tool I have found for social sciences is Perusall. It has been a game changer for many reasons. First and most importantly, it is the first platform where I actually enjoy online teaching because it closely mimics the conversations we would have in face-to-face classes. As a bonus, it also has the option to prevent students from copying and pasting within the LMS. While not 100% foolproof, this requires students to at least type out their questions and responses. Since adding this platform a year ago, the use of AI in my courses has dropped to almost zero.

At the school that is solely online, however, I asked about using Perusall but was told the institution already had licenses with several alternatives. After trying many of them - even those with built-in AI detection and Turnitin - I have found them to be ineffective. For example, I was part of a small group of faculty who piloted a program called Harmonize. During testing, I asked students to both copy and paste from traditional sources and to use AI-generated responses, then submit them. The highest score flagged on any originality report was only 17% plagiarized, despite the fact that I knew 100% of the post was plagiarized. From my experience, there really aren’t any effective built-in apps for Canvas.I  have tried to work around these issues by assigning video and recorded submissions, but I often ended up serving more as tech support than as a teacher. I also experimented with hiding a phrase within the text, but the results were mixed.

I bring this up because I’m currently writing my seventh misconduct referral for AI in just one class.