Priority Gathering for the New Quizzes Roadmap

[SuSorensen]

Over the course of the last several weeks, I have been synthesizing data from several sources to understand the concerns impacting priorities for New Quizzes. After examining the suggestions posted in my previous blog post, reviewing prior roadmaps, and chatting with the global product team representatives, the customer success team, and various customers representing various segments, the themes I’m seeing are as follows:

Content is King

One of the learning opportunities which Covid brought to educators across the globe was creating content in an ecosystem. In a time when online learning was new (and intimidating!) to so many, the ability to share and collaborate on quiz content was a wonderful surprise. The reusability of the content provides a value add over traditional course creation—but there are currently limitations in New Quizzes preventing broad reusability. 

Accessibility & Accommodations are Critical

Accessibility is non-negotiable; we cannot leave any students without the opportunity to learn and demonstrate their understanding. Many customers use third party tools to assess if a specific quiz has been designed as a11y. Additionally, the audio tools of RCE and quiz printing are needed to support some students for accessibility/ accommodation.

Don’t make me move Quizzes with a Uhaul

Customers would like to just wake up and see all Classic Quizzes available as a New Quiz they can enable. Legitimate concerns relate back to content and that it would be catastrophic to lose content. Additionally, customers transitioning from Classic to New Quizzes require lead time of approximately 6 months to test any migrations and handle the change management of preparing educators.  Rollout needs to be coordinated to harmonize with their next round of course development while also not impacting current student assignments.

These break down into the following big buckets of items to prioritize:

• Item Banks at Account Level
• Item Bank Permissions
• Quizzes Public APIs (Integrate with third party tools)
• Integrate New RCE
• Migration tooling (Quizzes and Quiz Banks)
• Blueprint: Item Update/ Quiz Locking/ Item Banks
• Print New Quizzes
• Partial Credit: Question Type Expansion
• Analysis CSV: Teacher/ Student
• To Do: Teacher/Student

It can be challenging to meet the needs of such a diverse user set, but it’s actually one of the things that I love the most. 

How do we make sure that we can harmonize the needs of those who are using New Quizzes already, those who want to be using New Quizzes but can’t, and those who don’t particularly care where their content lives but want it all to be there—no matter what! Our costumers are also varied across North America, South America, Europe and the Asia/ Australia markets. Across these regions, customers are also segmented by the needs of K-12 or Higher Education. I’ve been gathering as much feedback as possible to make sure these needs are all considered. If we can’t do everything, this data will help make sure we’re doing the most important things.

If you feel that a critical barrier is preventing you from using New Quizzes and is not represented on this list, I’d like to know about it. Please add a comment below, unless you added on my previous TL;DR post already. (If you’ve left a comment on a prior post, please just point me in the right direction!) 

Someone mentioned that it would be nice to allow the community to provide priority feedback. I completely agree. It would be great to hear your thoughts. I still have to play mix master and figure out timing/capacity and associate that with the most knowledgeable people to perform a particular task. However, it is wonderful to be able to pair that with the community sentiment. Please take a moment to let me know which of the above priorities are most important to you by choosing a selection on this  Poll .

[troy_challenger]

Need a Survey Equivalent

1)  In our current LMS, Moodle,  instructors make extensive use of questionnaires/surveys that include questions in arrays such as this.

2)  And the automatic analysis is presented in a similar concise grid format, which makes the results easy to compare between the questions.

3)  And questionnaires/surveys can be set to require students to complete ALL the questions before automatically awarding points for completion.  And branching to sub-questions, is also possible based on student responses. 

We need equivalent functionality in Canvas as a question type. 

[troy_challenger]

Matching Question Type needs way to add Images

1)  Currently the Matching question type allows matching words, but does not appear to have a way to insert images.

2) Also for matching questions, it appears students cannot receive partial credit. If I have 6 things to match and the question is worth 6 points, they get a zero if they miss just one!

[mpfaff1]

Our critical barrier: reliable grade transfer to the Canvas gradebook (with and without Respondus Lockdown Browser). This sounds like it sets the bar very low, but recent failures (whether through high traffic or some failure of communication between RLDB and the New Quizzes LTI) are causing our students to receive inaccurate or missing grades in the Canvas gradebook, with no solution except for students to be paying close enough attention to self-report, or for faculty to manually check every grade for every attempt (and re-attempt) for every student for every quiz one-by-one, which simply isn't acceptable. 

Our college has a very high New Quizzes adoption rate (more than half) because we were new to Canvas and were told Classic was going away in 2021. We have reversed our stance as an institution and officially recommended that no faculty use New Quizzes unless absolutely necessary, particularly with RLDB, but the damage has been done: there's no "migrate to Classic" option, so most instructors are simply stuck unless they have time to redo all their assessments.

[adrienneturner]

We NEED to have new quizzes integrate with the learning mastery gradebook. As a district that uses standards based grading everything needs to be tied to a standards and data needs to pull to the learning mastery gradebook. Thank you!

[SpanishT]

It would be tremendously helpful for many activities to have a set up similar to quizlet.com for entry.  Specifically I want to be able to enter in and edit multiple short answer questions from one page without opening multiple questions up.  Secondly I would love to be able to do this and move from one space to the next simply by use the tab button.  The clicking in and out of each question is extremely time consuming.  Or alternatively, having the ability to import short answer from these type of sites would be a second choice.  Many online games, activities that we use allow for export/import of the information into games/flashcards/practice and with Canvas being our main connector, we need that fluidity here as well.  Being able to create in word and copy and paste into the site which would convert into a question/answer format is extremely helpful.  As a foreign language teacher this often looks like a definition in Spanish/answer in Spanish for example. 

[stephanie_schot]

Expand New Quizzes Fill-in-the-blank question type to allow multi-line text.

Ideally this would be with a RCE like in old quizzes so you could have formatting with a separate blank on multiple lines for lists, steps etc. and the option to make tables that can be filled in by students. 

Even having a 'fill in the table' question type and adding line breaks to fill-in-the-blank questions would be huge. 

Without these options, our Math Department will likely have to completely re-format and manually transfer content for every quiz (we are talking thousands of items) in old quizzes to new quizzes, which has a lot of our users looking to other platforms. 

[danaleeling]

I wanted to echo the comments of @lisa_wathen, @bbennett2 and others above that "not sending outcome scores back to the learning mastery gradebook" is more than problematic. Assessing student learning through outcomes is an institutional priority and an accreditation requirement. That results for outcomes attached to new quizzes are not reported in the learning mastery gradebook renders new quizzes all but unusable. As noted above, perhaps running a core function as an external tool is simply not practical at this time. Notifications of submissions to the To Do list are missing, learning outcomes reporting back to the core platform is still not in place - things that should have been baked into the very first alpha release.

[MaryHofferberth]

The most important feature that I have found lacking in New Quizzes is the RCE and not having the ability to record media directly to the question. This is a fantastic benefit to students to be able to hear audio or watch a video demonstration of an instruction. Making a separate file and uploading it for each question is time consuming and prone to error ( locating the right file! ) It seems unnecessary when the feature is available. Although new quizzes has the aesthetic appeal. it is lacking in this key function that I need to work effectively with my ELL learners and students on IEPs. 

I have also not found a way to effectively import QTI files into New Quizzes as I could with Classic Quizzes. I have hundreds of Exam View tests that are modified for different levels. Importing them was a simple step before. Now it appears that Quizzes.next only accepts a certain version of QTI. Is there a way to convert from one QTI type to another? 

[hesspe]

I've read most of the comments, and it seems like most of the concerns touch us too.  After 9 years with Canvas and tens of thousands of course sites, that is not surprising.  As one of the commenters said, this switchover is going to be almost like switching to a new LMS.

I worry along with the commenter who said that using an LTI for a core function like quizzes was a strategic mistake that assures that there will never be full feature parity with existing quizzes, and that will leaves some heavily used features in the dust.

I made the following suggestion to our CSM, which was not made tongue-in-cheek: That Classic Quizzes and New Quizzes co-exist in perpetuity - New Coke alongside old Coke, with, as another commenter suggests, bug fixes and improvements ongoing for both versions.  Canvas now has a large enough customer base that this doesn't seem like it would be unreasonably burdensome to take on, especially after the New Quizzes development is deemed to be mostly completed.  I am dead certain it would save support people like me massive headaches come 2022.  I think it will cut down on the Tylenol consumption for Canvas CSMs and support staff too.

[mwistey]

My top priorities, in decreasing order of priority:

• Export student submissions to CSV for (re)grading! I'm looking at 30-40 hours to clean up a disaster of an exam. At least if I could export student submissions to a spreadsheet or Python, I could put all the answers in front of me and do *some* automatic grading. 
• Ability to fix mistakes in the original answers and regrade, including formula questions. Right now, if there was a mistake in the original formula, there is no way to fix it and regrade. 
• SpeedGrader or equivalent.
• Student math and rich text entry.
• Extra freeform box on every question: "Other (explain): _____" or "If you think this question is ambiguous, explain here: ____"
• MathType for screen readers: major accessibility issue. 
•  

[joe_fahs]

Converting Question Banks to Item Banks. We have had some faculty start with New Quizzes and really like the UX but when they had Question Banks to import opted to use Classic Quizzes instead.

Having a New Quizzes survey as a question type is crucial for assessing student feedback. 

Many publishers (McGraw-Hill, Cengage, and others) provide QTI formatted test banks that do not meet the New Quizzes import standard (QTI 1.2 or 2.1). The alternative right now is to import each publisher chapter bank into Classic Quizzes, export the quiz bank as a Canvas QTI, and import the standard QTI into a New Quizzes Item Bank.  Asking publishers to convert their legacy QTI test banks to a standard QTI is like pulling teeth. 

There should be an option to move/copy/delete multiple questions based on filtering. For example, some publisher banks include a mix of MC and short essay questions. When randomly choosing questions, instructors want to filter and move the short essay questions to another Item Bank. The alternative is time-consuming. 

[sbevier]

Here are the barriers in my building:

1) Lack of partial credit in many of the question types.**

2) New quizzes not being an option when creating from a Module. (I think this is being solved 3.22.21)

3) Lack of fill-in-the-blank formatting options**

4) Old RCE**

5) Not being able to record audio (where the teacher reads the question aloud and the student can listen if needed)

6) New Quiz submissions/resubmissions showing on To-Do list/Gradebook

7)Only being able to access speedgrader from the gradebook rather than from the Edit Quiz page.

😎 Co-mingling of assignments and quizzes** (I think this is being solved 3.22.21)

9) Inability to add a document to the question 

10) Inability to click "Next" after the quiz to progress through the Module.

11) Inability to create a survey for students. I have many teachers who would like to have exit tickets for the end of class that doesn't need to be scored...their either done or not done.\

12) Poor options for math teachers and those using equations and variables.

13) Inability to import question banks from the test gen software we already own.

**Things making my faculty angry on a regular!

[sbevier]

I forgot one!!
14) The inability to throw out a question for an entire quiz and have the quiz "rescore" everyone automatically.** 

[wendy4]

New Quiz Resubmissions not showing in Gradebook

When students resubmit their work there is no indication of this in the gradebook when working with New Quizzes. This has resulted in trainers asking students to resubmit work when they already have. I have advised trainers to work from their email notifications as a work around instead or working directly from the gradebook.

Is this currently being looked into? 

[SuSorensen]

Thank you so much everyone for your feedback. You can find our path forward for the year here:  Quizzes Planned Priorities and Roadmap 2021

All of your comments have been very useful and I'll see if additional requests, which are smaller, can be accommodated within the scope of our plan for the year.