Sample draw from question bank -- duplication of questions?

[rkujawa]

I am new to quizzing w CANVAS.  I created some question banks to build an exam.  I created a series of banks and then drew samples from them for various sections of the exam.  I checked it and tested it and it seemed to be working well.  Students have taken the exam and I was surprised to find duplicate questions on the exam.  I thought that I must've duped them in the bank and went back to check -- nope.  So my question is:  When selecting from a bank is it random selection without replacement?  If it is, has anyone else had the experience of getting questions duped?  My banks are small so, for example, I asked for 6 questions out of a bank of 11 and I saw dupes on the exam for several students.  I'd appreciate thoughts from the experts.

[Renee_Carney]

 @rkujawa  

Thanks for posting this to the community.  I think a long time ago I heard of this occurring, but it was when and instructor was drawing more questions than were in the bank.  Would you please go to one of the quizzes you were grading that you saw the issue with and then use the help menu on the left of Canvas to 'report a problem' and report it to our support team?

[rkujawa]

HI Renee, I am not sure how to submit it to the CANVAS support team.

I click on help but there’s no support team choice. The report a problem link is to our (Saint Michael's College) Help Desk.

Maybe it’s one of the other items.

Here’s a screen grab of the list of clickables.

Richard Kujawa

[Stef_retired]

 @rkujawa ‌

Many schools choose to customize their Help menus so that their users follow the support pathway they've defined—so yes, please use the IT Desk link to report this, and the local team can decide after evaluating it how or whether to escalate it to Canvas Support.

[rkujawa]

Done. Thanks.

[James]

 @rkujawa  

I am not sure from your description whether you are meaning that there are duplicate questions appearing on a single delivery of a quiz (questions 3 and 4 are the same for example) or if you are meaning that Tom and Sally saw the same question on their quizzes.

Each delivery of an exam is completely independent of all other deliveries. This includes not only quiz deliveries to different students but also any multiple attempts to the same student. Quizzes have no knowledge of which questions have already been used or how many times they have been used. Sampling is done with replacement for different attempts / deliveries.

Within a single attempt of a quiz for each student, questions from question groups are sampled without replacement. You should not be getting the same question multiple times within a single quiz provided there is no duplication of questions within the question group.

Now, if you are linking to a question bank from more than one question group, then duplication may occur. So if you set up a question group that pulled 1 question from a linked question bank and had another question group that pulled 1 question from the same question bank, each pull from the bank is independent of the others.

The sampling without replacement is only within a single question group.

[rkujawa]

Hi James,

I mean that in the exam I made, which used several question banks, that some questions on some student exams there were duplicates. I mean by that that some students on their exam would see the same question more than once (from within a single test bank). In other words, selection w replacement. I know I didn’t choose more questions than there were available in the banks. So I was surprised. I didn’t document the number of times it happened but just that it did.

I hope that’s clear.

I don’t have time at this point to go back and find actual examples from individual student tests.

Thanks for your interest.

Richard

[ar7662]

I just tested this before making my final exam. I had one group of 5 questions from a pool of 11. Then I added a pool of four questions from a pool of four questions. I tested whether there were any repeats of the questions from the pool of four at the end. (This mimics what I want to do on my exam.) I ran the preview and for five or six samples I never got a repeated question (all four of the last four were presented, and none were duplicated on a single preview of the quiz). Then on about preview seven I got one question twice and one missing question.

I do not think this will be solved before tomorrow, so for this exam I will just have to manually enter each question. This is old quizzes, and I think I need to move to new quizzes next semester, so I do not need an answer to this because the setup may be different, but I thought others might like to know the results of my system test.