Priority Gathering for the New Quizzes Roadmap

[SuSorensen]

Over the course of the last several weeks, I have been synthesizing data from several sources to understand the concerns impacting priorities for New Quizzes. After examining the suggestions posted in my previous blog post, reviewing prior roadmaps, and chatting with the global product team representatives, the customer success team, and various customers representing various segments, the themes I’m seeing are as follows:

Content is King

One of the learning opportunities which Covid brought to educators across the globe was creating content in an ecosystem. In a time when online learning was new (and intimidating!) to so many, the ability to share and collaborate on quiz content was a wonderful surprise. The reusability of the content provides a value add over traditional course creation—but there are currently limitations in New Quizzes preventing broad reusability. 

Accessibility & Accommodations are Critical

Accessibility is non-negotiable; we cannot leave any students without the opportunity to learn and demonstrate their understanding. Many customers use third party tools to assess if a specific quiz has been designed as a11y. Additionally, the audio tools of RCE and quiz printing are needed to support some students for accessibility/ accommodation.

Don’t make me move Quizzes with a Uhaul

Customers would like to just wake up and see all Classic Quizzes available as a New Quiz they can enable. Legitimate concerns relate back to content and that it would be catastrophic to lose content. Additionally, customers transitioning from Classic to New Quizzes require lead time of approximately 6 months to test any migrations and handle the change management of preparing educators.  Rollout needs to be coordinated to harmonize with their next round of course development while also not impacting current student assignments.

These break down into the following big buckets of items to prioritize:

• Item Banks at Account Level
• Item Bank Permissions
• Quizzes Public APIs (Integrate with third party tools)
• Integrate New RCE
• Migration tooling (Quizzes and Quiz Banks)
• Blueprint: Item Update/ Quiz Locking/ Item Banks
• Print New Quizzes
• Partial Credit: Question Type Expansion
• Analysis CSV: Teacher/ Student
• To Do: Teacher/Student

It can be challenging to meet the needs of such a diverse user set, but it’s actually one of the things that I love the most. 

How do we make sure that we can harmonize the needs of those who are using New Quizzes already, those who want to be using New Quizzes but can’t, and those who don’t particularly care where their content lives but want it all to be there—no matter what! Our costumers are also varied across North America, South America, Europe and the Asia/ Australia markets. Across these regions, customers are also segmented by the needs of K-12 or Higher Education. I’ve been gathering as much feedback as possible to make sure these needs are all considered. If we can’t do everything, this data will help make sure we’re doing the most important things.

If you feel that a critical barrier is preventing you from using New Quizzes and is not represented on this list, I’d like to know about it. Please add a comment below, unless you added on my previous TL;DR post already. (If you’ve left a comment on a prior post, please just point me in the right direction!) 

Someone mentioned that it would be nice to allow the community to provide priority feedback. I completely agree. It would be great to hear your thoughts. I still have to play mix master and figure out timing/capacity and associate that with the most knowledgeable people to perform a particular task. However, it is wonderful to be able to pair that with the community sentiment. Please take a moment to let me know which of the above priorities are most important to you by choosing a selection on this  Poll .

[lisa_wathen]

A critical barrier that is preventing us from using New Quizzes is that the outcomes data does not populate in the Learning Mastery Gradebook. We need the Learning Mastery Gradebook to sync with New Quizzes. Our hope is that at some point, all outcomes information from any source in Canvas will report to one place that is easily accessible for teachers and students.   Thank you!

[chriscas]

@SuSorensen ,

This is fantastic!  I'd love for this to become a model for future development at Instructure (give customers a way to provide feedback on Instructure's ideas before putting in a lot of development time and finding out the things aren't actually what the community wanted.

I previously added some feedback to the other thread, so I know you've seen that.  The list you provided here seems like a good start.

-Chris

[urbansk6]

Thanks so much for this! One absolutely critical barrier for my institution is the inability to export and import entire quizzes—currently, it can only be done by banking questions manually one by one and then "unloading" them manually one by one into the new course. We have dozens of instructors who keep hundreds of quizzes on shared, unpublished Canvas courses so that they can coordinate. They used to be able to move them from course to course with a few clicks; they can't anymore. It's a decrease in efficiency with no major advantages to make up for it.

[dobsocl]

I am unable to use New Quizzes because it does not support older QTI versions. I use TestGen (a Pearson test generator) which exports a QTI file that is not compatible with New Quizzes but IS compatible with Classic Quizzes. Bulk Import of QTI files would also be a helpful feature.

[milesl]

@SuSorensen While it may sound counterintuitive, something I'd like to see as part of the New Quizzes priorities is a clear commitment to maintaining (and maybe making some small improvements to) Classic Quizzes over the next 18 months.

Some institutions may have a few specific boxes to be checked before they can confidently move forward with their campus-wide migration. For others supporting diverse users and needs, there may never be a precise point when New Quizzes meets everyone's needs, so they may choose to keep Classic Quizzes around as long as possible, so as to give instructors as much time and flexibility as possible to plan and adapt. Given that I expect to find myself in the latter camp, seeing a small amount of resources dedicated to the maintenance and upkeep of Classic Quizzes will be critical to the migration.

We've been told for years that New Quizzes is just around the corner, and for years, requests for fixes and improvements to Classic Quizzes have been declined in order to focus on New Quizzes. As New Quizzes' development time stretched on, we've found ourselves at a place where dozens of small issues that might individually be ignored add up to a product with significant issues for many of our users, so successfully bridging them to New Quizzes begins with the false assumption that their needs are already being met.

In summary, I hope that Instructure can find some resources to dedicate to delivering impactful improvements to Classic Quizzes. This will build good faith, and help our instructors successfully make the transition to New Quizzes.

To more directly answer your question, my top 3 picks are:

1. Reporting parity with classic quizzes

2. Ability to regrade question bank questions

3. A complete re-working of availability windows/time extensions. The mental gymnastics required to set up a time constrained quiz are staggering (I spend so much time explaining how availability windows override time extensions via quiz moderation that I dream about it).

[Steven_S]

@SuSorensen  One of the biggest issues I see is with the integration to the gradebook, but I do not see where that fits in your list. I mentioned it in your previous TL:DR so I won't go into great detail about the problem. But to emphasize why it matters:

The existing system (manual grade posting) only identifies new quizzes that are pending manual grading if all students turn in their work before grading begins.  That just isn't what happens in normal semesters.  There are always students with excuses that warrant extensions.  So, grading is complete, and a student sees that a zero has been posted as their grade, and out comes a doctor's note.  I can reprogram the quiz to give them another day or two, but since a zero is already published there is no indication in the gradebook whether or not the quiz has been taken. 

New Quizzes set to do not count towards grade, show up in the gradebook, and are subject to the same late penalty as graded assignments.  That directly interferes with their use to measure student progress using the score at least requirement of modules - which is how we are using practice quizzes.  (It also lowers the perceived retake value for students studying, which is a second purpose for our practice quizzes.)  Showing up in the gradebook is fine (and showing in the student to-do list is essential) but there needs to be a way to exempt practice quizzes from the late policy if they are displayed in the gradebook.

[venitk]

Thank you for acknowledging that we need several months to transition to NQ after it is ready.  This will be a huge project for us, almost as big as when we migrated to Canvas, if we have to recreate every bank.

Without more details, I can't be sure what you mean by

• Item Banks at Account Level
• Item Bank Permissions

but a critical barrier for us is that the banks belong to people. Even with the ability to manually share a quiz with someone else, that will not work with our work model. We need the banks to belong to the course,  so that anyone with permission to edit the course can edit the banks. 

I'm going to guess this is what you meant by Item Bank Permissions, and I'll fill out the poll accordingly, but I wanted to mention it, just in case. 

[MattHanes]

What seems like a quick win for little effort would be adjusting the Moderate screen to only show active students (or add a filter for active/inactive) and allow for sorting by name. I can't figure out what kind of order the names are in but it doesn't seem to be alphabetical. 

*EDIT* Deploy notes for Jan 06, 2021 show the moderate screen will no longer display inactive students! Whoo hoo!

[epulham]

Thank you for posting this! The Blueprint needs are huge for us, so thank you for that. I don't know about the expansion of test settings for displaying the answers. Classic Quizzes lets you show answers one time after each attempt, as well as giving time bounds for viewing quiz results. That is not there yet that I've seen. That is a very important priority for us but doesn't show up on the feature parity lists and I wasn't able to bring it up at the UEN Consortium call. Thank you for your transparency and for collecting feedback in this way!

[yklaw]

@epulham 's post makes a really good point.  A key feature we use is that we allow students to view their old quizzes after the class deadline has passed - a balance between test security and enabling students to learn.  Unfortunately, while it is possible to toggle this manually, it would be very helpful to have this be toggled automatically. (ideally, under settings, the Restrict student result view can have a "restrict until" date field that can be toggled).

[dave-long]

Thanks for posting this @SuSorensen!  I'll echo everything people above said, but was wondering if data from New Quizzes be included in Canvas Data?

Priority wise, I'd say the features and user experience are the most critical parts, but also having New Quizzes data in Canvas Data is important as well.  Focusing on the user experience / functionality parts first makes sense, I could see adding New Quizzes data to Canvas Data taking place after other features are in place. 

[snugent]

Thank you @SuSorensen 

My view of new quizzes is not good at this moment. I am seeing errors with new quizzes after a course import using copy a Canvas course option. If I had know about this was issue, I would never have encouraged faculty to use new quizzes. This needs to fixed ASAP ( I already have a ticket on this). You cannot expect faculty recreate quizzes they have already created. This is going to extremely irritating issue for me as our classes start on Monday. 😬

Other issues I have with new quizzes. 

QTI imports - Still can't import content textbook publishers directly to new quizzes. And yes I know I can run them through the current quiz tool but I should be able to import directly into the new quiz tool. 

Editing Quizzes - This is confusing for faculty. There are two sets of settings in different locations. I have to carefully explain the difference between assignment settings (due dates and points) and quiz settings. I would prefer that new quizzes worked like other LIT tools so that the assignment settings button displays while editing the quiz. 

Integration in a Canvas Course - There are several issues here with quizzes not displaying in instructors' To Do List, module flow for students, adding quizzes to modules (listed as assignments). Also if I duplicate a quiz, the title is not updated in the quiz settings. If I update the title in Canvas, it doesn't update the title in the quiz and vice versa. 

CSV report - Instructors need to be able to download the data as CSV. 

Quiz settings with release option - The options are poor in classic quizzes and are no better in new quizzes. Instructors want the ability to  set a time frame when results are available for students to view. This means the results only displays to students during the time frame selected. Instructors just don't time to remember to change these settings manually. 

Integration with the Canvas Commons - Currently there is no portability of quizzes created in the new quiz tool. Instructor don't have time to recreate quizzes. 

Rich content editor - Most faculty want to have access to content in files and some also want students to be able to record video content using our Studio integration. 

Better control of item banks - I am happy that faculty can share item banks but there needs to be more options. I worry that giving read/write access can be an issue so more granular permissions would be nice. It would nice to have an option to allow people to copy the question bank. As admin I can't see the questions banks unless I masquerade as the person. 

Select All - Please allow questions to selected using a select all option in quizzes and item banks. This is one of the most annoying things about Canvas in general

I understand that COVID-19 has delayed the timeline but new quizzes has been in the works for a very long time and I am exhausted trying to keep updated on the changes. It makes it very hard to prepare faculty for the changes. Please be more transparent with development and make sure our CSM knows what is going on as well!

[yklaw]

The CSV export mentioned by @snugent is critical in my opinion.  Frankly, New Quizzes has promise particularly with creative question types, but I think the integration with the rest of Canvas is sorely lacking.

[DKahre]

Is there a way to get notified on the Dashboard when a student completes a New Quiz? (Especially one that still needs graded.)

Is there a similar way to utilize "Speed Grader" in the New Quiz? Currently it takes exponentially longer to grade a New Quiz than a class full of Classic Quizzes. 

In Classic I would use speed grader from the Dashboard notification to work through each student. Now with the New I go to the quiz, select moderate, grade one student, back out of the quiz after updating the grade, select moderate again, then grade the next student. 

Also our school does not have the grades synced to PowerSchool so when a student does complete a New Quiz, I do not get notified and I may record their grade as a "0" missing in PowerSchool and not know when to go back into the quiz when they have submitted the New Quiz, when the Classic Quiz notified me on the Dashboard. 

[Steven_S]

The course copy errors mentioned by @snugent for new quizzes are a huge issue for anyone creating a master course to be used by other instructors.  Even if the retry button works, the retried import is not necessarily linked to its module, and all new quizzes are imported as unpublished, and even though lockdown browser will still be mentioned in the quiz title the prior settings do not copy with the course the way they did with classic quizzes.  Now instead of clicking the lockdown browser once to reactivate the prior settings, we need to manually turn on lockdown browser for every quiz, and then open advanced settings and turn on chromebook and ipad access.  Individually for every single quiz.  Communicating set-up instructions to new faculty needs to be as simple as possible.

[MTolen]

I would love to be able to switch between question types after I have started the question. This is one feature of old quizzes I love. Sometimes the question type doesn't work as I thought it would.

I teach math and use the dropdown option INSIDE a table often for quizzes. I would love to have this feature in new quizzes.

[jwalz]

A top priority - in association with the QTI - is to be able to IMPORT questions / question banks from OTHER sources.   We use Respondus quite a bit for this and it would be great if that continued, but also was improved.   Why cannot New Quizzes have ADDITIONAL options to import questions?    Many faculty may like to create a quiz in Word first and then import all the questions or some of those questions - so they can keep a separate record of their possible content if they choose to.  It would be nice if there were easier ways to import questions - from a variety of sources.  

[mficzner]

I would like to see the option for students to be able to annotate on the text we include as an image in assignments and quizzes.  We teach how important it is to underline, circle, write connections, etc.  The students would greatly benefit from being able to do this right there on the text image in order to prove their answers by citing text evidence.

[MrsFeliciano101]

It would be extremely helpful to be able to organize the students in the various columns while in the Moderate screen. For example, click on the Student Name header and it alphabetize the student list. 

[staceymojica]

I would like for the students to be able annotate on their computers. I like how the "new" quizzes have the option to have the text on one side, but it would be beneficial if the students had a tool to highlight, circle, or underline the answers from the text. They have learned valuable close reading skills and they should be able to use them when taking tests or completing assignments. Thank you!

[froelich]

Adding my voice to the chorus of those who need to see REPORTING called out as a priority here. Our organization has piloted New Quizzes in one course for a high-profile client of ours and reporting has been a disaster. As I understand it, Instructure rolled this out without proper API. As a result, our Instructure-built custom report, specifically designed a year ago to chart course progress by quiz completion, DOES NOT SHOW ANY QUIZ RESULTS and we have had to routinely dedicate extra staff time to integrate data from other sources for a rolling enrollment of now close to 500 learners. We have seen no progress from Instructure to correct this, or the functionality of our expensive report which we purchased as custom dev to specifically automate this function. 

[tross]

Definitely need to be able to export data in a CSV like you can in Classic.  

First thing I need though is those APIs to release.   We have 3rd party tools we rely on that we cannot use until they have access to the API to develop their tool.  We need those products, like Proctorio, before we can go completely over to New Quizzes.  They need time to develop.

[SBromby]

I would like to see the students names in alphabetical order when moderating a new quiz. I would be a lot easier to cross reference who has completed an assignment.

I would like an option in New Quizzes for the students to make the text on side of stimulus full screen so it is easier to read. Then they can minimize to go back to the questions. Annotation options would be plus as well. 

[and-hu]

Wow, thank you so much to everyone for your feedback. I clicked the option to enable the New Quizzes on my course because I'd read they would be enforced in the summer and figured I'd get a jumpstart on converting (though it appears that has been postponed to next summer). However, with all the shocking issues with them seen in the comments, I will absolutely not be enabling them.

I played around with them a little to see if they were really that bad, particularly the "Modules" integration issue as I will be relying heavily on Modules organization to orient students in online learning. The fact that it essentially takes students to some nigh-unrecognizable version of Canvas, with the course sidebar gone and the previous/next arrows missing, and that it doesn't display number of questions/points on the info header are major showstoppers for me. There are probably more (certainly the many issues presented in this topic and Su's intro post), but these were enough to get me to quit. I'm glad the enforcement date has been postponed a year!

[gcarlucc]

App integration is critical for our organization. We use MathType, a math equation editor, which also works nicely for accessibility concerns. New Quizzes currently only support Mathquill, which is lacking in terms of accessibility (i.e. screen readers). A simple expression like √7 reads "backslash backslash s q r t left brace seven right brace backslash backslash", when it should be "the square root of seven".

[tim_kato]

Hi Susan! Thanks for pointing me here! The following seems like core features that are critical in a modern LMS and for alignment with other features in Canvas:

• Item Bank Permissions
• Quizzes Public APIs (Integrate with third party tools)
• Integrate New RCE
• Print New Quizzes
• Partial Credit: Question Type Expansion
• Analysis CSV: Teacher/ Student
• To Do: Teacher/Student

The following are important but less critical

• Blueprint (it's awkward when some parts of a course are covered and some are not)
• Migration tooling (Quizzes and Quiz Banks)

The following are nice to have:

• Item Banks at Account Level

UBC would also like to see improved reliability and integration with the core components of Canvas.

Thanks!
Tim

[bbennett2]

There are several comments on this original post highlighting a lot of the main issues with New Quizzes, and I wanted to expand on a couple here. (These are in no particular order.)

Building a core function as an LTI is a bad experience. If quizzes are first-class tools within Canvas, then it should be part of the core tooling. Having New Quizzes run as an LTI introduces a host of issues (the teacher to-do list doesn't show new submissions, scores show in the gradebook even if the quiz has unscored questions, getting to the quiz to score in the SpeedGrader is a steeplechase) that make it really unpleasant to use.

Building a core function as an LTI puts us on edge as customers. The main benefit of LTI tools is that they're pluggable. When Instructure started building Quizzes and the new Assignments tools as LTIs, to us it's signalling a revenue source. Will we be hit with upsells to continue to use Quizzes? It's not out of the question because by building these as LTI tools, you can granularly turn them on and off for those who can afford to keep them over those who cannot. It's bad optics and makes us talk more seriously about whether or not we can continue to invest in a platform.

Question banks have been gutted. One of the most powerful tools in Canvas is having course-level question banks that can be used to randomly generate quizzes for students using Question Groups. I know we can pick questions in new quizzes, but by having banks pulled out from the old quiz engine, I've now lost that main feature. Allowing for reassessment through the banks gave me control over which standards to assess and when. I knew that each time a quiz was loaded, whether it was the first or ninth, it was aligned to what I wanted to assess. With new quizzes, that is much harder to do. Which leads to...

Not sending Outcome scores back to the Learning Mastery Gradebook is devastating. Being able to get granular reports on student progress at the standard level is immensely powerful. Since new quizzes doesn't talk to the LMG, it's a dealbreaker. I can do one-off assessments, but not having that body of data with the rest of the student information fragments the system so much that I recommend my staff do not take the time to invest in new assessments. In the end, this is the straw that will break the back of quizzing in general through Canvas.

I'm hoping that since the New Quizzes project was started with Instructure was fragmented and all over the place that some of these issues are symptoms of the larger organizational issues. On the other hand, the Thoma Bravo acquisition is concerning, considering the history of private capital organizations picking up investments only to dismantle and sell off assets for profit. 

Thanks for leaving this open for comments. I'm hoping to see some of these tackled head on over the next year.

[kentmiller]

On the classic quiz you just put in a new editor as of Jan 2021 which superior to the editor you have in new quizzes. Will this new editor be added to the new quizzes?

[Fernsy]

As an admin at my school and being part of a distance education schools network, the biggest issue we have with new quizzes is the lack of ability to export from one Canvas instance to another. We have networks of teachers from different schools that share content and course writing responsibilities which has been a huge positive of all schools using Canvas as their LMS. Currently if one course needs to be exported from one instance to another teachers need to create each new quiz in their own instance. For some courses this results in days of work if there are a large number of quizzes.

As a positive our students all love the look of new quizzes and are happy with the product as a student. Incorporating the RCE, ability to print and partial credit are features I hope start to be enabled as these will make my teachers more happy to use new quizzes.

[urbansk6]

In addition to my above comment I'd like to add that the way the New Quizzes "fill in the blanks" question type is programmed, with no rich content editor and the "return" key being reserved for creating blanks (thus eliminating the ability to create line breaks), has caused my institution many, many problems.

I recently migrated almost a hundred quizzes, and all of the fill-in-multiple-blanks and multiple-dropdowns from Classic Quizzes are now run together in a way that is confusing to students, looks unprofessional, and makes certain basic language test question types impossible (dialogues, conjugation tables, "put the underlined word in past tense," etc.). They are unusable in this format.

I'm now spending hours (I anticipate it taking at least 20 hours, if not more, and I'm in the very rare position of it being my job to do things like this) reformatting all the "fill in the blanks" questions from scratch, because it is in fact faster to rebuild them from scratch than to make adjustments from what's already there, and I'm also spending my time finding creative solutions to my new inability to format dialogues and tables. At this point, New Quizzes have taken more time and capabilities away from us than they've added, and I hope that these issues can be addressed soon.

[SuSorensen]

I am watching your comments and tracking these requests. When I started investigating New Quizzes, one of the first things I heard a lot of concern with are issues raised here by @snugent  and @and-hu  related to the experience of adding and editing New Quizzes. Using the blogs area, the Quizzes team was able to solicit and meet with customers to get feedback on workflow designs. I hope you will all consider supporting us in this way in the future as well. Changes will be coming and I’ll post an update in the coming weeks. 

Some comments also reference buggy behavior with Copy and Grade pass back. These are undoubtedly significant to your users and the engineers are actively researching and resolving issues, so you should see improvement shortly. 

On the reporting front, Instructure as a whole is working on data, reporting and analytics. @dave-long  Canvas Data2 is coming along. There is a User Group here: https://community.canvaslms.com/t5/New-Analytics-Users/gh-p/analytics

Additionally, publicly available and documented APIs will help meet these needs. As @froelich notes, this was helpful in the Classic Quizzes API because you can manipulate the data for reporting. 

@kentmiller   Precisely, the New RCE editor added in Canvas this month. I am interested in hearing from customers how important this is to be done soon versus some of the other critical priorities identified. 

Thank you all for your participation and providing feedback. I understand you have been stretched beyond patience with regard to some of these issues, which makes it even more critical for me to make sure we’re delivering the most valuable improvements first.

I view this as a collaborative effort and we cannot be mutually pleased with the outcome without working together. This specific post will be closed before March. However, to continue our open conversation, I’ll start a new post to let you know which of these priorities discussed will be worked on first and ask for feedback on what the Quizzes team believes should come next. I’ll also be requesting help in discovery phases and again with design feedback to make sure we provide true value to you and your users.

[ScottKing]

Personally, my #1 frustration at the moment is the inability of students to type "math" in their responses.  As the teacher I have access to the rich content editor and can type the question however I like using the equation editor.  But if I want a student to do the same thing in a fill-in-the-blank or numeric answer I am out of luck.  It's so frustrating to want to ask a NUMERIC answer question and not have mixed number or exponents as possibilities.  I understand that I can ask it as short answer then they have the rich content editor available to them, but that increases the grading over head.  

It would also be fantastic to have more control over blank placement in a fill-in-the-blank question like you do in Old Quizzes.  In New Quizzes all you can do is in line blank placement with no rich content editing.  (I can have rich content in the question stem, but not the actual fill-in-the-blank structure).  I would like to be able to have a table and have students fill in blanks in a table.  Not possible in New Quizzes.  (I can build the table in the question stem, but not put editable answer blanks).  I would love to find out I'm wrong on this issue, but if there is a way I don't know it.

One more, make numeric answer an option for fill-in-the-blank.  I can use numbers, but only as text.  Having a numeric option means I don't have to worry about all the different ways a student types a number (0.5 vs .5 vs 0.50, etc...).

[sarawells]

@SuSorensen Thank you for your thoughtful blog on the subject of New Quizzes. I'm an administrator, and I teach and work with other faculty on curriculum development. One of the near deal breakers for using New Quizzes is the absence of partial credit questions. To create an auto-graded quiz without needing to adjust quiz scoring for questions like ordering, matching, and the like, requires use of two basic question types: multiple choice and true/false. My institution is in the process of building several exams in Canvas to function as prerequisite exams, and it would be ideal to have access to partial credit question types in New Quizzes. I find the other types of questions (ordering, matching, etc) impractical due to the all or nothing scoring system in place. At the same time, I am hesitant to continue using Classic Quizzes if it is to be phased out. I do, however, prefer it at present. 

A very lengthy conversation on this subject has been occurring here. I appreciate the interest in hearing about the array of concerns from stakeholders. It seems that a lot of folks would like to see the option for partial credit scoring enabled for New Quizzes as it was for Classic Quizzes. 

Thanks for listening, and thank you for your work on this.

[cford4]

Hello,

One major issue many of my faculty are running into is exporting quizzes from Word Documents to Canvas. This is a feature that is present in so many LMS (Blackboard, Moodle, Brightspace), but not in Canvas. We have been told about the superior quiz making features of Canvas, but many faculty do not wish to copy hundreds of questions and answers to place in Canvas. Their only current option is to use Respondus Test Generator, which is not always reliable. If New Quizzes could incorporate this basic feature that is present in so many contemporary LMSs, that would go a long way to encouraging "buy in" from our faculty. Thank you.

[ddamschen]

I am in agreement with each and every concern @snugent outlined. Thank you @snugent for being so clear and concise in your post. 🙂  

Modules: During the summer (knowing we would be in DL for a while) we instructed our teachers to use Modules. We want to help students direct their workflow for the week. However, using New Quizzes, that flow is severely interrupted. So much so that my coworker no longer creates Quizzes using NQ. He has reverted back to Classic Quizzes.

Item Banks: Spent the first part of the school year migrating the classics quizzes from the year prior. Also worked on creating Item Banks from these quizzes. However, now that we are back to using Classic Quizzes (see Modules above), it makes it that much more time consuming to get those questions into Item Banks.

Also- Need partial credit. Having to "touch" quizzes after they are taken to assign partial credit is tedious and time consuming. 

[BethCrook]

Thank you for this opportunity for collaboration and feedback, @SuSorensen! We absolutely cannot switch to New Quizzes until we have RCE or, at a minimum, the ability for line breaks, in order to accomplish a quiz question with multiple fill-ins such as this:

This is key in all of our new student placement exams, as well as all of our language courses and others.

[jfb42]

Yesssss, inability to format fill-in-the-blanks questions is a HUGE disadvantage compared to old quizzes.

And the partial credit. Those two things have caused me so much headache.

[BethCrook]

Also, New Quizzes must be able to be linked in Modules.  This should be a very simple fix to complete, in my mind.

[kona]

I really wanted to try using New Quizzes this semester, but the very first quiz I tried to create was a bust because you can't directly add files to quiz directions or quiz questions. I teach statistics and it's crucial to be able to attach files for the students to be able to download and use to answer the questions on the quiz. Yes, I could figure out a (not so great) workaround, but I do this so much for my quizzes that the extra frustration didn't make it worth it so for now, I'm sticking with legacy quizzes. #sadpanda

[Steven_S]

@BethCrook  New quizzes can be added to modules, they are just in the assignment list instead of the quiz list where you expect to look for them.  However, new quizzes interrupt the flow of the module for students.  They cannot click next to get to the next assignment, but instead they have to navigate back to the next activity through the modules tab.  For some students this simple step requires a great deal of explaining. 

[BethCrook]

@Steven_S This is good to know, thank you!  However, being able to keep them in Modules is important.  So having that Next button for them to continue on once they have completed/submitted their Quiz to move on to the next item seems to be a simple solution.  

I would be more than happy to show someone at Instructure a walk through of a new student placement exam so they can understand why having the simplicity of being locked into Modules is so important, not only for us, but as an example for many instances.

[gabrielcarey]

Matching questions are graded as all or no credit, meaning that if a question has 10 words to be matched to a definition and a student misses 2 of them, the whole question is counted incorrect.  Can there be an option to give partial credit to the words correctly matched?

[mprogers]

I just want to add my voice to the cries for partial credit questions and properly formatted, multi-line, multiple fill-in-the-blank questions. It beggars belief that such absolutely fundamental design requirements, ones that any teacher, at any level, would insist be there, are not. 

[janna_carney]

In thinking about accessibility and accommodations, it would be great if instructors could reduce the number of distractors for individual students in moderate. This is not currently a feature in classic quizzes, and at least with new quizzes we can duplicate and modify. It would be much better if this could all be done within the same quiz, similar to how we can add time and attempts. I would also like to reiterate the need for printing new quizzes. We haven't even been able to find a workaround for this. 

[megankoontz]

The critical barriers for us are that New Quiz submissions are not showing in faculty tasks, and that resubmits do not show in the grade book.  

In fact, I just went back to the old Quiz because of this and I do not even talk about New Quiz with my faculty due to this.

[DanielleLove]

A critical barrier for us is the interruption of module flow when a student takes a New Quiz. After submitting the quiz, there is no option to continue along in the module--students are required to navigate back to the module list and figure out what module they were on. This is highly problematic for user experience and we're concerned it may cause students to miss content pages. 

[hillarykolodner]

Hello- Our critical barrier is that the new quizzes feature interrupts the workflow of students progressing through a module. They have to click "return" after completing a quiz, then it boots them out of the module back to the module homepage. Students then have to scroll down through all of the modules to find their place, and then this repeats every time they take a quiz. It is confusing and cumbersome for our students to constantly have to find their place again. The Classic Quizzes did not have this problem. And when we were instructed by our Canvas rep to start using the New Quizzes for our modules, we were not informed of this barrier and have already built over 100 quizzes using the New Quizzes features. Can someone please work to resolve this issue ASAP or let us know when it will be resolved? Otherwise, we will need to manually convert all of our New Quizzes back to Classic Quizzes, then migrate them in 2022 when Classic Quizzes are discontinued. Thank you!

[Steven_S]

@hillarykolodner @DanielleLove @BethCrook  It looks like the solution to new quizzes in modules might be coming soon: Modules Support Adding New Quizzes & Release Timing 

[fordhamj]

Fill in the BLANK option doesn't work!  

[rhenri24]

@SuSorensen First, I am happy to see the upcoming changes to the Modules supporting adding New Quizzes. This will be extremely helpful moving forward. Thank you!

For my institution, I still see a few critical barriers slowing our adoption.

1) Missing RCE functionality: Not having access to add course files, record/upload audio and video, and all the other great features that our instructors are utilizing in the new RCE we just adopted is almost a deal-breaker. Without the RCE and its functionality, it's like asking instructors to learn a whole new tool and will change how they have spent the last 5 years creating assessments. Like @BethCrook this will impact placement exams (especially language the depend on audio/video) and other departmental exams.

2) Lack of Third-party integrations (Proctorio, Office 365, etc.): I tend to agree with @bbennet2 above that building this core function as an LTI seems like an interesting choice, to put it mildly. Doing so seems to make it much more difficult to use other LTIs that our institution has invested a lot of time/money with over the past few years. We would need to see a way to be able to utilize these tools as part of our quiz-tool solution. 

3) 'Next' button to continue in Module: I echo the statements above about this. Keeping students going in the module after they complete a quiz is important. If we make them click back to modules to figure out where they left off, instructors have likely lost the students to other things or students think they are done with the module.

4) Multiple places for changing settings: I agree with @snugent above about the confusion having two different places (Assignment settings and quiz settings) to edit settings will cause. This will frustrate our faculty to no end and will be a frequent support request for our support teams. It would be great to have one place to go for all settings. Or at the very least, be able to click on the quiz and access the assignment settings then click on "Edit Quiz" to enter the quiz editor, similar to how the classic quiz has the "Questions" tab at the top. 

Thank you!