For those of you who work with courses that have multiple sections, how do you handle combining quiz/exam data across sections? I'm working with a course that has about 12 different sections. Our quizzes and exams all use question sets, so when I download the student data the ordering of the questions is different in every section. I need to be able to do this in order to compute the overall difficulty and discrimination indices as well as the item-total correlations to identify questions that should be cut for next semester. @gfj100 has posted an idea to fix this, " modifiedtitle="true" title="Consistent Order of Questions in Student Analysis Report. But for now, how do you handle this?
First I would like to apologize for the late response to your question. Have you found an answer to this issue?
I would recommend looking at the APIs to pull the data out of the courses. I am not sure this will give you exactly what you are looking for, but the only other way would be to export the student or question analysis across from each course and then combine them.
Hi @Robbie_Grant ,
No, I have not yet found a solution. I am not familiar with APIs. The exam has about 200 questions, from which each student is given 50 (one from each of 50 sets/groups). It would take days for me to manually align all of the questions across all 12 sections. When the student analysis file is downloaded from Canvas the questions are presented in the order in which the most recent students received them. (I am still trying to figure out why anyone would ever want the data presented in this order.) This means that the questions are in completed different orders for all 12 sections. I was hoping that someone had found a way to relatively easy get all of the questions in a more logical order so that data files could be combined.
You might be able to glean something from this question and following discussion:
While it goes into the same issue you're having (so you're already aware of the problem), it mentions using an R script to combine the results.
No, I have not yet found a way to combine data from multiple sections of the course when question sets are used.
While time consuming, when working with Quizzes (legacy vs. New), I download the Excel spreadsheet with the Item Analysis for each section then use the Outline/Subtotal function on the Data tab to calculate aggregate means for the difficulty index and point biserials, both correct and distractors.
I'm a wee bit perturbed that when faculty use New Quizzes, I can no longer download this spreadsheet. I can get the point biserial data for the correct answer but not for the distractors, which is equally important.