[ARCHIVED] Quiz Settings Insight Needed

[garciah]

This is a fun one!

Background:

I had an instructor tell me that she set her quiz to "Show Correct answers at" a specific time, but that students were able to see their correct/incorrect answers before that time. This didn't sound right, so I tested it out.

First test:

I tested with one one attempt and the "Only after their last attempt" box unchecked. It worked. I noted that when the "Only after their last attempt" box is checked with one attempt, the "Show correct answers at" option is disabled. That makes sense. However, when you change the quiz to multiple attempts, it gets much more confusing.

Additional tests with multiple quiz attempts:

The first thing I noticed is that with multiple attempts, there are two "Only after their last attempt" boxes:

No combination of checking these boxes disables the "Show correct answers at" option, so one would assume that the "Show correct answers at" setting would work.

I did some tests on various combinations of the "Only after their last attempt" option checked/un-checked, and documented the results.

Both checked:

First quiz attempt: no answers at all, not even my responses

Second quiz attempt: responses and correct/incorrect answers, even though correct/incorrect answers shouldn't be released

First one unchecked, second one checked:

First quiz attempt: responses, but no correct answers

Second quiz attempt: responses and correct/incorrect answers, even though correct/incorrect answers shouldn't be released

First one checked, second one unchecked:

First quiz attempt: no answers at all, not even my responses

Second quiz attempt: responses, but no correct answers, note about when they will be released

Neither checked:

First quiz attempt: responses, but no correct answers

Second quiz attempt: responses, but no correct answers, note about when they will be released

Questions:

1. Any idea why there would be two "Only After Their last Attempt" boxes? This makes absolutely no sense to me.
2. Are you aware of any bugs related to these settings? I'd love to hear about them!
3. If you have the patience to try, were you able to duplicate any of my results?

[kona · Accepted solution]

 @garciah ​,

1. Yes, the first "Only after their last attempt" box is to allow students to view their responses to the quiz only after their last attempt. "Their responses" means what answer the student entered or checked for each question and whether or not the answer they provided was incorrect; it doesn't indicate what the correct answer is, that's what the second box does. For the second "only after last attempt" this box is for showing the correct answers. This box controls when the student gets to see which answer is correct. What this does

2. No bugs, but there are a LOT of things that aren't clear or may seem confusing. I recommend checking out the following document - Quiz Settings to Maximize Security - that goes into more detail about what each setting actually controls/does AND the following webinar - - that shows example settings and how the quiz results would actually be displayed to the students.

3. Here's what I found. The short answer is that I tested the scenarios you provided and everything seems to be functioning as intended. Per the following guide - What options are available for Quizzes? - it specifically states, "Note: If the Only Once After Each Attempt option is selected, it will override any show or hide dates or times. If you want to show or hide correct answers on any specific date or time, the Only Once option should not be selected."

Quiz settings are definitely confusing and I hope I've provided some resources that will help clarify some things!

[garciah]

 @kona 

Thanks for the reply, and thanks for clarifying #1 above. The two options make sense to me now. I've shared Quiz Settings to Maximize Security with numerous faculty here. It's very helpful!

While I think the note ("If the Only Once After Each Attempt option is selected, it will override any show or hide dates or times. If you want to show or hide correct answers on any specific date or time, the Only Once option should not be selected.") is helpful, I find that a lot of people like to figure things out by doing them, as opposed to reading step-by-step instructions. The UI should make it obvious that certain settings override others by disabling them. This would prevent a lot of headache. I mention above that the "Only after their last attempt" box is checked with one attempt, the "Show correct answers at" option is disabled, but this doesn't happen with multiple attempts.

Here's what I think needs to happen: When multiple attempts are allowed and the second "Only after their last attempt" option is selected, the "Show correct answers at" option should be disabled. This would make make the interface more intuitive and it mimic what happens when only a single attempt is allowed.

I guess I'll submit a feature idea, even though I doubt it will get enough attention to reach 100 votes. :smileyplain:

[kona]

I'd vote on it! I think you're correct that some type of error/warning message should pop up or if one is used the other should be disabled. This is drfinitely confusing?

[garciah]

Sweet! Here's the feature idea -

[kona]

Very nice! I've bookmarked it, but it's always good to post reminders (possibly in some of the different Groups in the Community) when the feature idea actually opens for voting!

[lka11]

We just encountered this same issue. Are the questions in your quiz essay questions? If so, your assumptions that apply correctly to other question types may not apply to essay questions.

The problem is that since Canvas does not inherently know the correct answers to essay questions (as it would with multiple choice or T/F questions), it does not show the correct answers. In the case of essay questions, the General Answer Comment is intended for messages such as "this answer will be graded manually", not the actual answers!

If you wish to provide the answers to essay questions, you could use the option of making a file available (which includes both questions and answers) after the due date, or according to other desired settings.

[garciah]

Good point, Laura! I'll definitely keep this in mind for essay questions, but I was testing the settings with multiple choice questions.