Edits made to a copy of a New Quiz in one course are mirrored in other courses

[johnb]

Dear All,

A strange one this so any help greatly appreciated.

I posted this to the Question Forum yesterday but am re-posting here as this was probably where I should have posted to begin with. Apologies for the cross-posting.

A teacher taught a course during summer 2020. She used New Quizzes to deliver the final exam in the course. The teacher is teaching this course again this summer (summer 2021). At the beginning of the summer 2021 course, the teacher copied the summer 2020 course over to the summer 2021 course. Along with all other course content, the final exam was also copied. 

The teacher decided to make a few small edits to the questions for the summer 2021 course final exam. When she did she received the message below. She then contacted me.

I copied her final exam from the summer 2021 course to my Sandbox course. I clicked through the message above and successfully edited a number of questions. However, when I returned to the summer 2021 course, I saw that the edits I had made to the quiz had been mirrored there. I thought that perhaps this has something to do with the fact that the summer 2021 course and my Sandbox course are both open, so I checked the summer 2020 course (the original course) and discovered that the edits I had made had been mirrored there also. 

When I returned to my Sandbox course and removed the edits from the quiz, these edits were, again, mirrored in the summer 2021 course and the summer 2020 course.

This surely is not intended behaviour?

Any thoughts?

Best Wishes,

jb

[anthony_harris]

We just did a quick test and the issue appears to be that questions for New Quizzes are always stored in Question Banks, and you can only edit the question in the bank, i.e. each quiz isn't getting an independent copy of the question.  And, since question banks appear to be per-user not per-course or per-quiz, when you edit the question from another copy of the quiz in another course, you're still editing *the* copy of the question, so that change will show across all courses.

Thank you for pointing this out, as this could represent a significant problem for us.  I wish Classic Quizzes wasn't scheduled to eventually go away. ☹️

[milesl]

Hmm, I was unable to replicate this issue:

I created two course sites: A and B.

I created a new quiz with a few questions in course site A, then imported course site A into course site B.

I edited one of the questions in course site B, and returned to course site A, which was not impacted.

[Steven_S]

I think that if you copy the course using import course content from the course settings, that this will probably not happen.  (If you recreate the quiz by adding questions from the same bank it will almost certainly happen.)  I'm not sure how this works with other course copy methods, but there is good news:  In the prior course, when you check the speedgrader or quiz log for individual students, you should see the original quiz unchanged.

[johnb]

Many thanks @anthony_harris @milesl and @Steven_S 

I have tested this again. The procedure I followed and results below.

I used two Sandbox courses (A and B), each with the same two test students enrolled (X and Y).

In Course A, I created a New Quiz with two 4-answer option MCQs. I created both questions in the quiz and I added both of these questions to an Item Bank. I then acted as Student X and completed the quiz.

In Course B, I imported (not copied) the quiz. I renamed the quiz and then I edited one of the questions to add a 5th answer option.

The messages I received when editing the question below.

(As an aside, something I hadn´t noticed before when importing, the quiz did not appear in the Quizzes folder when Selecting Content, rather in the relevant Assignment Group folder.)

When I returned to Course A, in Teacher view, the edit I had made was mirrored there i.e. one of the MCQs now had a 5th answer option. I then acted as Student Y and started the quiz. Student Y received the updated version of the quiz. As Student Y, I left but did not submit the quiz. 

I returned to Course B and edited the second MCQ, again adding a 5th answer option and receiving the same messages as above.

I then returned to Course A, where my Teacher view had updated to reflect the latest version of the quiz i.e. both questions with a 5th answer option. I acted as Student Y and resumed taking the quiz. Student Y returned to the quiz version presented when she started the quiz i.e the change to the 2nd MCQ was not mirrored. Student Y submitted the quiz.

In SpeedGrader, Student X´s submission reflects the original version of the quiz (2 x 4-answer option MCQs). Student Y´s submission reflects the first edit made to the quiz in Course B (1 x 4-answer option and 1 x 5-answer option).

I then repeated the test above with one difference: when I created the questions in the quiz I did not add them to an Item Bank. I imported etc. to Course B and when I then made edits to the questions, these were not mirrored in Course A.

So, in summary:

It appears that the mirroring is occurring because the questions are linked to an Item Bank as @Steven_S identified in his test. Is this intended behaviour? If it is, it raises a number of worrying issues with regard to using Item Banks in New Quizzes.

In Classic Quizzes, when you create a question directly in a quiz, that question is added to a  ´Unfiled Questions´ question bank. Is there an ´Unfiled Questions´ question bank in New Quizzes? I could not locate one. If there is not, do questions "go" anywhere or do they reside solely in the quiz itself?

Many thanks to all for any further thoughts on this.

Best Wishes,

jb

[Steven_S]

@johnb  In new quizzes, as in classic quizzes, a question created in the quiz is in the quiz.  You can put it into a bank using the item banking settings below each question.

The new quiz exists external to the course and has to be linked to.  If your questions were not in a bank, it sounds like your copy/import method is copying the link to the same quiz, without copying the actual quiz.  It would end up a bit like having cross-linked sections.  I'm not sure what would make this happen, however it has not been my experience with the course import in the right side menu of course settings. used to copy a canvas course. The quiz copies entirely, generally the intention is that item banks do not copy, but rather provide the questions to every linked quiz in the manner you are describing.  However, you would see the notice that the question is in a bank, and need to complete your edits from the bank (usually viewed in a right side slide out.)

Some of the obvious questions raised:

- what if the quiz is copied to two classes taught by two different instructors?  In our tests with live classes, before item bank sharing, the other instructors had access to the questions in the quizzes, so the copy was successful - but they had no need to edit the quiz.  I had updates in two sections and had to re-import the updated quiz into the other section.  I discovered that it matters whether the course I am copying into already has a quiz with the same name (and that changing the name in canvas does not give the actual quiz a new name.)  After some experimenting, the easiest way to move the questions to the other section was to put them in a bank, and then add them from the bank to the quiz of the other section. 

- what if I make changes to the quiz now that questions exist in more than one course through the item bank?  It is good that updating the quiz does not change previous attempts at the quiz, and it makes sense that this applies to in-progress attempts as well.  As the instructor, I can always use speed grader to see which version of questions students responded to and correct the scoring if necessary.

- what if questions in a bank are in a copied quiz?  When we tested this before the shared banks, both instructors had the questions in the quiz for their students, and did not need to update the quiz.  It appears that the banks copied to the instructors.  Now with shared banks, there a way to copy the course with view only access to the banked questions.  If given that access, instructors can duplicate the questions in the quiz, edit the duplicate, and delete the view only copy.  (If the questions were added using the all/random option from the bank, the instructor may first need to add the questions individually using the plus next to each question.)  This will only impact their own course, because they only have access to view the bank.  Multiple instructors can be given access to edit the shared bank, but their changes to the shared bank would apply to all courses.  We will be doing more testing in our fall courses to see how this works with the latest updates.

[AllisonPowell]

I was able to replicate this issue in some of my courses. If I added/changed an item bank in one course after copying that quiz in from another class, that item bank was also changed/updated in the original course (different level). I ended up having to create a brand new quiz for each class and re-assigning. However, I have not been able to replicate this across all my courses/levels, and last week when I was working on these quizzes there were plenty of other issues I experienced with Canvas, so I am wondering if it was a chain reaction. However, since the latest Canvas update, I am pessimistic about how some of the tools work.