cancel
Showing results for 
Search instead for 
Did you mean: 
carol_kominski
Community Member

Quizzes: question numbers

Jump to solution

In a student analysis report on a graded quiz, what do the question numbers mean?  Can I search item banks by question number?  In an exam, can I tag questions by the item pool they came from?

Is there a way of tagging each question by topic within an exam other than including this information in the item stem?

Basic issue:  I'm dealing with a 50+ item final exam taken from different quiz banks.  Each quiz bank pertains to a different topic and all the questions from the different quiz banks were mixed and presented in randomized order for each student.  When I ask for a quiz statistics report, I get no question numbers at all.  With a student analysis report, I get 5 digit question numbers beside the stem of each question.  I'd like to sort student data by module and easily determine which quiz bank (module) the questions came from.

Thank you.

1 Solution

Accepted Solutions

 @carol_kominski ​,

I've tried to answer all of your questions. Sorry if I missed any.

What do the numbers mean?

The 5 digit number isn't a 5 digit number. Well, yes, it is, for now, but it's a sequential number that grows every time someone adds a new quiz question.

I had to fix a problem with a quiz question just last night and it was quiz question id 57148138.

Every quiz question has an id associated with with it. The numbers are not easily obtainable through the web-interface, which displays the quiz question name and uses the quiz question id in the source code but doesn't make it visible or searchable.

The question id is close to worthless for tracking down a question using the web interface.

Method 1: No Super Powers Required

In the Student Analysis report that you showed, the quiz question id is followed by the question itself. Most of the question is obscured, but if you click on the cell, you'll see that it can be very long. It might even look empty depending on the question.

137833_pastedImage_0.png

Notice (top row) it looks like there is nothing in the cell except for 57148138: and even the colon is cut off in the actual cell.

But if you click in the cell and press F2 to edit it, you'll see this:

137852_pastedImage_1.png

That means that the text of the question is available, it just may not be visible.

The problem with this question is that the text shown was on all 25 versions of a question bank, so it's not helpful for identifying the correct one. However, that's actually the end of the question. Because of the cell alignment, column width, and font sizes I have, it's hard to see the whole thing. You might try changing the width of the column, shrinking the font size, and/or turning off wrap. When I play around with the question, I get this:

137853_pastedImage_4.png

So, now I have some text that uniquely identifies the question. I will search for 394.01 in my Canvas Quiz Questions page to find the right one.

The easiest way for most people to find the question in Canvas quiz or banks is to search for some of the text that they see from the question text in Excel.

Method 2: Familiarity with the Page/DOM Inspector

In both Chrome and Firefox, you can press F12 to bring up a Page or DOM inspector. You can then search for the quiz id number that you found in Excel.

In Firefox, the search is by CSS selector and the way to identify the question is by an id of "question_" followed by the number. So searching for #question_57148138 (the # is for id) would find the question I'm after.

137854_pastedImage_8.png

After pressing F12

  1. Make sure you're on the Inspector
  2. Click in the CSS selector search and type #question_57148138 and press enter
  3. Click on the <div class="header"> section to expand it
  4. The name of the variable is given. In this case, it's ch5k_25

Method 3: Ability to Read JSON

If the URL for the quiz is https://richland.instructure.com/courses/1619254/quizzes/3016652/ and the item from Excel is 57148138, then what you can do is modify the URL by adding /api/v1 before /courses and add the /questions/ and the quiz question ID to the end

Here's what I get for my question (I've replaced the correct answer by 999999 here so my students don't get it)

https://richland.instructure.com/api/v1/courses/1619254/quizzes/3016652/questions/57148138

while(1);{"assessment_question_id":112956800,"id":57148138,"position":25,"quiz_group_id":1566536,"quiz_id":3016652,"question_name":"ch5k_25","question_type":"numerical_question","question_text":"\u003Ctable style=\"text-align: right;\"\u003E\n\u003Cthead\u003E\n\u003Ctr\u003E\n\u003Cth style=\"padding-left: 2em; border-bottom: thin solid #000;\" scope=\"col\"\u003Evariable\u003C/th\u003E\n\u003Cth style=\"padding-left: 2em; border-bottom: thin solid #000;\" scope=\"col\"\u003En\u003C/th\u003E\n\u003Cth style=\"padding-left: 2em; border-bottom: thin solid #000;\" scope=\"col\"\u003Emean\u003C/th\u003E\n\u003Cth style=\"padding-left: 2em; border-bottom: thin solid #000;\" scope=\"col\"\u003Estdev\u003C/th\u003E\n\u003C/tr\u003E\n\u003C/thead\u003E\n\u003Ctbody\u003E\n\u003Ctr\u003E\n\u003Cth scope=\"row\"\u003Epredictor, x\u003C/th\u003E\n\u003Ctd\u003E20\u003C/td\u003E\n\u003Ctd\u003E55.86\u003C/td\u003E\n\u003Ctd\u003E6.14\u003C/td\u003E\n\u003C/tr\u003E\n\u003Ctr\u003E\n\u003Cth scope=\"row\"\u003Eresponse, y\u003C/th\u003E\n\u003Ctd\u003E20\u003C/td\u003E\n\u003Ctd\u003E394.01\u003C/td\u003E\n\u003Ctd\u003E182.44\u003C/td\u003E\n\u003C/tr\u003E\n\u003C/tbody\u003E\n\u003C/table\u003E\n\u003Cp\u003ECorrelation coefficient = 0.424, p-value = 0.062\u003C/p\u003E\n\u003Cp\u003EFind the y-intercept of the \u003Cem\u003Ecalculated\u003C/em\u003E regression line.\u003C/p\u003E\n\u003Cp\u003ECalculate the y-intercept, even if the line should not be used for estimation purposes.\u003C/p\u003E","points_possible":1,"correct_comments":"","incorrect_comments":"","neutral_comments":"","correct_comments_html":"","incorrect_comments_html":"","neutral_comments_html":"","answers":[{"id":1270,"text":"","comments":"","comments_html":"","weight":100,"numerical_answer_type":"exact_answer","exact":999999,"margin":0.005}],"variables":null,"formulas":null,"matches":null,"matching_answer_incorrect_matches":null}

Yikes!

But if you scan, there's a "question_name":"ch5k_25" in the top row. That contains the name you would need to find.

By the way, you can copy that text, except for the while(1); at the top, and paste it into an online JSON decoder to get a more readable form.

Method 4: API Skills

Method 3 calls the API without actually having to mess with authorization tokens and will work as long as you're logged into Canvas. If you are a programmer or know one, you can get the information directly from the API in JSON format and then manipulate it to your heart's content.  See the Quiz Questions API for more information.

Can I search item banks by question number?

Not directly. If you copied the content into the quiz (as opposed to linking to the bank), then you will be able to find the question using any of the methods above.

If you linked it to a bank instead of finding questions and copying it into the quiz, then it's not actually part of the quiz and won't show up. The question id will be in the Student Analysis report, but it will not show up in the quiz itself.

The best most people can hope for is to recognize the question and the bank that it came from, then go to the bank and then search for the question name or some unique text from the question.

Otherwise, it's a programming nightmare through the API. You have to fetch a list of all the quiz submissions, then iterate through each to get the id of the quiz submission. You then use that to get the quiz questions that were used in the submission. Then you can look at the questions to find the one you want.

Furthermore, the bank that it came from isn't identified in the report. There is a quiz_group_id in the quiz submission questions API call, and it should provide a number that can be used in the quiz question group API, but it was coming up NULL for me on the question I linked to a bank.

In an exam, can I tag questions by the item pool they came from?

Not unless you go into the question banks and create a common prefix.

That said, the name of the question doesn't appear in the report, only the quiz id and the text of the question. I'm probably missing it somewhere, but since the quiz_group_id is coming up empty on the quiz submissions API, finding out which question bank they came from to tag them is going to be difficult without exporting the content and pulling it up in something else and that might lose the internal question id (not sure).

Is there a way of tagging each question by topic within an exam other than including this information in the item stem?

Yes!

And here you were beginning to think I wouldn't have any good news, weren't you?

If you import the questions into the quiz rather than linking them to the bank, then you get a whole bunch of benefits including the ability to regrade questions. But for your question, what you get is the quiz_group_id. That can be named with something useful like which bank the data came from, but more importantly, every question from the same bank will have the same quiz_group_id.

If you go back to method 3 above and leave the question id off the end, you'll get a list of all the questions (paginated, so you may need to follow links to get additional pages) from the quiz.

https://richland.instructure.com/api/v1/courses/1619254/quizzes/3016652/questions/57148138?per_page=...

There is a line in each one that says something like: "quiz_group_id":1566536

To find out what that quiz group is, you use the call like https://richland.instructure.com/api/v1/courses/1619254/quizzes/3016652/groups/1566536

And that gives you something like this:

while(1);{"assessment_question_bank_id":null,"id":1566536,"name":"ch5k","pick_count":1,"position":8,"question_points":2,"quiz_id":3016652}

Again, ignore the while(1); at the front. The name of the quiz group here was "ch5k" and that's what you could label to match your bank.

Unfortunately, there's not a single call that will return all quiz question groups for a quiz, so you'll have to do it for each quiz_group_id found in the quiz questions.

Seriously though, all of this should be done through the API where you can automate it, deal with pagination, etc. I just used the browser to get the information because Jive was not displaying my JSON correctly and I didn't want to go to all the trouble of writing something major up if it wasn't going to work for you.

Closing Thoughts

The question banks aren't supported well using the API. This might be a problem that could be tackled using the Canvas Data if you have access to that. I've started to play with it and I know it has a lot of tables on quizzes, but I don't know if you can get exactly what you want out of it.

The quizzing in Canvas is very complex and the API's are marked as Beta and subject to change. There's a good chance I missed something.

Your best bet to accomplish what you want is to import the questions into a quiz and not link them. Then name the quiz question groups for the bank that the data came from. Use the APIs to get the data if you can or look into Canvas Data​ (a lot of people have held it up as a solution to all questions, but it's not unless you already understand big data).

Don't waste time trying to get the quiz statistics page sorted by module.

View solution in original post

4 Replies
chofer
Community Coach
Community Coach

Hi  @carol_kominski ​...

By chance, could you provide a screen shot of where you are seeing the five-digit number?  This is something I've not heard of myself, and I'm wondering if it would help others who may also be reading your question.  Thanks!

Chris,

Below is a screen shot that comes from a downloaded student analysis. The question numbers precede the text of each question.

Carol

2015-12-03_0910.png

 @carol_kominski ​,

I've tried to answer all of your questions. Sorry if I missed any.

What do the numbers mean?

The 5 digit number isn't a 5 digit number. Well, yes, it is, for now, but it's a sequential number that grows every time someone adds a new quiz question.

I had to fix a problem with a quiz question just last night and it was quiz question id 57148138.

Every quiz question has an id associated with with it. The numbers are not easily obtainable through the web-interface, which displays the quiz question name and uses the quiz question id in the source code but doesn't make it visible or searchable.

The question id is close to worthless for tracking down a question using the web interface.

Method 1: No Super Powers Required

In the Student Analysis report that you showed, the quiz question id is followed by the question itself. Most of the question is obscured, but if you click on the cell, you'll see that it can be very long. It might even look empty depending on the question.

137833_pastedImage_0.png

Notice (top row) it looks like there is nothing in the cell except for 57148138: and even the colon is cut off in the actual cell.

But if you click in the cell and press F2 to edit it, you'll see this:

137852_pastedImage_1.png

That means that the text of the question is available, it just may not be visible.

The problem with this question is that the text shown was on all 25 versions of a question bank, so it's not helpful for identifying the correct one. However, that's actually the end of the question. Because of the cell alignment, column width, and font sizes I have, it's hard to see the whole thing. You might try changing the width of the column, shrinking the font size, and/or turning off wrap. When I play around with the question, I get this:

137853_pastedImage_4.png

So, now I have some text that uniquely identifies the question. I will search for 394.01 in my Canvas Quiz Questions page to find the right one.

The easiest way for most people to find the question in Canvas quiz or banks is to search for some of the text that they see from the question text in Excel.

Method 2: Familiarity with the Page/DOM Inspector

In both Chrome and Firefox, you can press F12 to bring up a Page or DOM inspector. You can then search for the quiz id number that you found in Excel.

In Firefox, the search is by CSS selector and the way to identify the question is by an id of "question_" followed by the number. So searching for #question_57148138 (the # is for id) would find the question I'm after.

137854_pastedImage_8.png

After pressing F12

  1. Make sure you're on the Inspector
  2. Click in the CSS selector search and type #question_57148138 and press enter
  3. Click on the <div class="header"> section to expand it
  4. The name of the variable is given. In this case, it's ch5k_25

Method 3: Ability to Read JSON

If the URL for the quiz is https://richland.instructure.com/courses/1619254/quizzes/3016652/ and the item from Excel is 57148138, then what you can do is modify the URL by adding /api/v1 before /courses and add the /questions/ and the quiz question ID to the end

Here's what I get for my question (I've replaced the correct answer by 999999 here so my students don't get it)

https://richland.instructure.com/api/v1/courses/1619254/quizzes/3016652/questions/57148138

while(1);{"assessment_question_id":112956800,"id":57148138,"position":25,"quiz_group_id":1566536,"quiz_id":3016652,"question_name":"ch5k_25","question_type":"numerical_question","question_text":"\u003Ctable style=\"text-align: right;\"\u003E\n\u003Cthead\u003E\n\u003Ctr\u003E\n\u003Cth style=\"padding-left: 2em; border-bottom: thin solid #000;\" scope=\"col\"\u003Evariable\u003C/th\u003E\n\u003Cth style=\"padding-left: 2em; border-bottom: thin solid #000;\" scope=\"col\"\u003En\u003C/th\u003E\n\u003Cth style=\"padding-left: 2em; border-bottom: thin solid #000;\" scope=\"col\"\u003Emean\u003C/th\u003E\n\u003Cth style=\"padding-left: 2em; border-bottom: thin solid #000;\" scope=\"col\"\u003Estdev\u003C/th\u003E\n\u003C/tr\u003E\n\u003C/thead\u003E\n\u003Ctbody\u003E\n\u003Ctr\u003E\n\u003Cth scope=\"row\"\u003Epredictor, x\u003C/th\u003E\n\u003Ctd\u003E20\u003C/td\u003E\n\u003Ctd\u003E55.86\u003C/td\u003E\n\u003Ctd\u003E6.14\u003C/td\u003E\n\u003C/tr\u003E\n\u003Ctr\u003E\n\u003Cth scope=\"row\"\u003Eresponse, y\u003C/th\u003E\n\u003Ctd\u003E20\u003C/td\u003E\n\u003Ctd\u003E394.01\u003C/td\u003E\n\u003Ctd\u003E182.44\u003C/td\u003E\n\u003C/tr\u003E\n\u003C/tbody\u003E\n\u003C/table\u003E\n\u003Cp\u003ECorrelation coefficient = 0.424, p-value = 0.062\u003C/p\u003E\n\u003Cp\u003EFind the y-intercept of the \u003Cem\u003Ecalculated\u003C/em\u003E regression line.\u003C/p\u003E\n\u003Cp\u003ECalculate the y-intercept, even if the line should not be used for estimation purposes.\u003C/p\u003E","points_possible":1,"correct_comments":"","incorrect_comments":"","neutral_comments":"","correct_comments_html":"","incorrect_comments_html":"","neutral_comments_html":"","answers":[{"id":1270,"text":"","comments":"","comments_html":"","weight":100,"numerical_answer_type":"exact_answer","exact":999999,"margin":0.005}],"variables":null,"formulas":null,"matches":null,"matching_answer_incorrect_matches":null}

Yikes!

But if you scan, there's a "question_name":"ch5k_25" in the top row. That contains the name you would need to find.

By the way, you can copy that text, except for the while(1); at the top, and paste it into an online JSON decoder to get a more readable form.

Method 4: API Skills

Method 3 calls the API without actually having to mess with authorization tokens and will work as long as you're logged into Canvas. If you are a programmer or know one, you can get the information directly from the API in JSON format and then manipulate it to your heart's content.  See the Quiz Questions API for more information.

Can I search item banks by question number?

Not directly. If you copied the content into the quiz (as opposed to linking to the bank), then you will be able to find the question using any of the methods above.

If you linked it to a bank instead of finding questions and copying it into the quiz, then it's not actually part of the quiz and won't show up. The question id will be in the Student Analysis report, but it will not show up in the quiz itself.

The best most people can hope for is to recognize the question and the bank that it came from, then go to the bank and then search for the question name or some unique text from the question.

Otherwise, it's a programming nightmare through the API. You have to fetch a list of all the quiz submissions, then iterate through each to get the id of the quiz submission. You then use that to get the quiz questions that were used in the submission. Then you can look at the questions to find the one you want.

Furthermore, the bank that it came from isn't identified in the report. There is a quiz_group_id in the quiz submission questions API call, and it should provide a number that can be used in the quiz question group API, but it was coming up NULL for me on the question I linked to a bank.

In an exam, can I tag questions by the item pool they came from?

Not unless you go into the question banks and create a common prefix.

That said, the name of the question doesn't appear in the report, only the quiz id and the text of the question. I'm probably missing it somewhere, but since the quiz_group_id is coming up empty on the quiz submissions API, finding out which question bank they came from to tag them is going to be difficult without exporting the content and pulling it up in something else and that might lose the internal question id (not sure).

Is there a way of tagging each question by topic within an exam other than including this information in the item stem?

Yes!

And here you were beginning to think I wouldn't have any good news, weren't you?

If you import the questions into the quiz rather than linking them to the bank, then you get a whole bunch of benefits including the ability to regrade questions. But for your question, what you get is the quiz_group_id. That can be named with something useful like which bank the data came from, but more importantly, every question from the same bank will have the same quiz_group_id.

If you go back to method 3 above and leave the question id off the end, you'll get a list of all the questions (paginated, so you may need to follow links to get additional pages) from the quiz.

https://richland.instructure.com/api/v1/courses/1619254/quizzes/3016652/questions/57148138?per_page=...

There is a line in each one that says something like: "quiz_group_id":1566536

To find out what that quiz group is, you use the call like https://richland.instructure.com/api/v1/courses/1619254/quizzes/3016652/groups/1566536

And that gives you something like this:

while(1);{"assessment_question_bank_id":null,"id":1566536,"name":"ch5k","pick_count":1,"position":8,"question_points":2,"quiz_id":3016652}

Again, ignore the while(1); at the front. The name of the quiz group here was "ch5k" and that's what you could label to match your bank.

Unfortunately, there's not a single call that will return all quiz question groups for a quiz, so you'll have to do it for each quiz_group_id found in the quiz questions.

Seriously though, all of this should be done through the API where you can automate it, deal with pagination, etc. I just used the browser to get the information because Jive was not displaying my JSON correctly and I didn't want to go to all the trouble of writing something major up if it wasn't going to work for you.

Closing Thoughts

The question banks aren't supported well using the API. This might be a problem that could be tackled using the Canvas Data if you have access to that. I've started to play with it and I know it has a lot of tables on quizzes, but I don't know if you can get exactly what you want out of it.

The quizzing in Canvas is very complex and the API's are marked as Beta and subject to change. There's a good chance I missed something.

Your best bet to accomplish what you want is to import the questions into a quiz and not link them. Then name the quiz question groups for the bank that the data came from. Use the APIs to get the data if you can or look into Canvas Data​ (a lot of people have held it up as a solution to all questions, but it's not unless you already understand big data).

Don't waste time trying to get the quiz statistics page sorted by module.

View solution in original post

Thanks very much for your extremely thorough answer!

Now in 2021 it's possible to add custom question titles which are supposed to show up in the Item Analysis, but in fact, it seems, do not.   

No question titlesNo question titles