[ARCHIVED] Best practices for online exams

[everson_50]

Hi Everyone,

Now that we are all diving deep into the online waters, I wanted to find out more about best practices when it comes to setting up online exams.  I teach and coordinate a statistical literacy course at Ohio State, and most students take this course to fulfill a general education requirement.  We have up to 1000 students in the course each semester.  I have one section of the course that is mostly online, but, up until just a couple of weeks ago, we required all students across all sections of the course to take a written midterm exam and a written final exam in a proctored setting.  Most of our students are local and are able to come to campus to take these exams at common times.  We work with students who are not local in order to figure out other options, and most of those students end up making appointments at approved testing centers or having exams proctored at public libraries.

Naturally, we have no choice now but to move everything online for all of our students.  For this semester, I have been given permission not to have a final exam at all in my course and to change the weights of other assignments.  This gives me some breathing room in order to figure out how to restructure the course moving forward.  I am scheduled to teach the online section again this summer, and my assumption is that my midterm and final exams will both need to be online during that time.  Perhaps that won't be the case, but I feel I need to be planning for this contingency.

Given the large number of students we have, in addition to our desire to be able to quickly grade their work and provide them with feedback, we generally tend to create exams that are composed mostly of multiple-choice questions.  We then have a small number of short-answer questions that usually involve computations of some sort, or that require students to explain their reasoning.  I try hard to write multiple-choice questions that go beyond recall and recognition; I want to see that students can analyze, apply, evaluate, and synthesize.  

If my midterm and final exams have to be fully online, I know this will open up the possibility of academic dishonesty, and I know there is only so much I can do to prevent that.  I was hoping I might be able to randomly order exam questions and answer options.  I'm not sure if I can easily set things up so there are multiple versions of the same exam, with different students receiving different versions, but if I can do this, I would definitely want to go that route.   I would probably lower the weights of the midterm and final exams in order to create less of a "high stakes" atmosphere.  To make things easier for us to grade--especially given our large number of students--I might have to resort to an all multiple-choice exam, but I feel comfortable writing questions that will hopefully challenge students (e.g., questions where they cannot simply hunt for answers in their textbook).  I am also thinking of including some kind of "commitment to academic integrity" as a possible test question.

Are there other things I should be thinking about?  I'd love to hear more from those of you who might already be incorporating these kinds of exams in your online courses.  I know we could also go the route of using some kind of proctoring service, but I was hoping to avoid that, if possible.  

Thank you all for your time and support.

Michelle

[paul]

Hello Michelle,

Last semester I did my very first online final exam (due to a pending local transit strike) and have now created a document outlining a lot of the things I learned along the way - hope it might help you too!

Delivering a Final Online Exam - Google Docs 

All the best,

Paul

[afairweather]

Thank you for sharing!

[afairweather]

Hi  @paul ‌,

I used your tips to create a video to share with my faculty to help them transition their F2F exams.

Thank you again!

Angie

[paul]

That's so awesome  @afairweather ‌, thanks so much for sharing! Love what the CC License makes possible

[Shar]

 @afairweather  Nice Work that's a totally awesome video! :smileycool:

[Stef_retired]

 @afairweather ‌, thanks for sharing this wonderful video and for being so helpful to your fellow community members, especially in these difficult times. Please check your direct messages for a small token of our appreciation. :0)

Thanks again for all that you do and keep up the good work!

Sincerely,

Canvas Community Team

[Shar]

Hi Michelle Everson!

Oh my goodness, Wow!! Paul Hibbitts has given you an excellent resource of tips from practical experience!! Everything he covers in that document is important:

1. Write it like an open-book exam
2. Give triple the time you think it would take for the exam
3. Use a question bank to pull random questions
4. Show sample question(s) in advance
5. The exam-time discussion board is genius!!

I think the only other thing(s) to consider are the quiz settings:

• Reveal correct answers at a later date
• Only give 1 attempt (unless you're feeling generous and give 2)

If it's the final exam, I would stick to multiple-choice/-answer questions so that the scores can be auto-generated. One of my favorite question types is True-False-False because it will have the feedback already in the answer choices. Here's a quick example:

Statement: The best day of the week is Friday.

Answers:

True. Friday is the end of the work week and the weekend can begin.

False. Thursday is the best day because the weekend begins on Friday.

False. Saturday is the best day because it's a first full weekend day.

Good luck and hope the advice you get helps,
Cheers - Shar

[everson_50]

Thank you, Sharmaine and Paul!  This is all extremely helpful.

I do have a couple of follow-up questions for you and others who might read this.  First, I too like the idea of an exam discussion board.  However, one concern I have, especially if students might be all over the country or even outside of the country, has to do with how long the exam window should be.  If we take into account the fact that our students might be in different time zones, would we want a 24-hour window?  Of course, once they open the exam, they'd need to finish in a particular amount of time.

Second, I have a lot of experience using question banks and setting up quizzes, but I've never set up a question bank and then had questions randomly chosen from that bank.  Would anyone be able to point me to directions for how to do this?  This might be a whole different discussion unto itself, but one thing I'd want to make sure of is that I can randomly choose questions related to several different topics, especially since our exams tend to cover different chapters.   I'm not sure how easy that would be to accomplish, but hopefully, it's doable in Canvas. 

[paul]

Hi Sharmaine and Michelle,

Glad you found the info helpful, and thanks for sharing additional tips Sharmaine!

All of my students were local university students, so the exam and discussion forum were only available during a fixed period. In your situation where students are geographically dispersed, you could offer the final exam in a much wider window (e.g. a day as you suggest sounds great), however a live question clarification forum would not really work in that scenario. Does anyone here have experience is that situation re: handling question clarifications from students during a much longer possible exam time?

I myself do not have any experience with question banks.

Hope the above helps,

Paul

[snugent]

 @everson_50 ‌

Here are some resources to get you started. 

Quiz Settings to Maximize Security - by Kona Jones

Here is the TL:DR version

• Use large pool of questions (ie question banks) to pull into question groups in the quiz. This will allow you to randomize the questions so student a gets a different test than student b. 
• Don't use result defaults. I tell faculty to select the option Only Once After Each Attempt. This option lets students see feedback but the next time they come to the quiz page, they won't see the results. 
• Turn on to display one question at time. You can also choose to lock answers in so students cannot return to question. 
• Shuffle answers - You have to careful with this option as it will also shuffle answers of question that is design to say all of the above. Rewrite those question so you can use the shuffle answer option. 
• Use some of other question types that are available such multiple drop downs 

Here are some guides that cover setting up a quiz. 

• How do I create a quiz with a question group to randomize quiz questions?
• How do I create a quiz by finding questions in a question bank?
• What options can I set in a quiz?

[everson_50]

Excellent!!!!  Thank you so much, Susan.  This is exactly what I was looking for.  

[ccallicott]

@everson_50 , I gave students an assignment recently, but one student emailed me her submission. She said that even though she tried to submit it at 11:50, Canvas had already closed the assignment (which was due at noon). I asked her to check the time zone on her computer. Turns out her computer was new and was still set to the incorrect time zone. So, I guess when it comes to your students who are working remotely, the Canvas settings apply to their individual computer, not to your time zone.