Feedback Discussion • Canvas AI Authoring Usability and Workflows

[MarcusVu]

As you use the Canvas AI Item Authoring Assistant, please use this space to share your feedback/comments around the usability of the features and workflows. 

[MarcusVu]

As you use the Canvas AI Item Authoring Assistant, please use this space to share your feedback/comments around the usability of the features and workflows. 

[drobbins]

Thanks for the demo and introduction today. Looking forward to using this tool!

• Like I mentioned on the call, it might be nice to add a little UI hint that the "Nutrition Facts" are available to view in the Item Authoring Assistance modal. I only brought it up because I spent a few minutes looking for it unsuccessfully! If the IgniteAI double-star logo is indeed emerging as a standard affordance in the ecosystem, it might be worth guiding new users to that, at least in the near term.
• I would be surprised if it's not near the top of the roadmap, but selecting course files as the source material for question generation would be a great addition. The need to upload, again, files that are already included in the course (articles, slides, etc.) is a pretty awkward UX.

[turnermz]

I agree on the workflow for using files. I was under the impression initially that only course pages could be used to generate questions, but since files can be uploaded at this stage it's an odd workflow not to be able to access files directly from the course.

[turnermz]

I find the page selection tool confusing. In some courses pages are grouped by module block, in others some pages are presented in their module blocks, and some separately (possibly because some pages are not in the module blocks). In addition, in a course with a larger number of module blocks and pages, not all are displayed in the selection list.

Could there be an option to select how to view the pages (by Modules or Pages) and a search field to more easily locate pages in courses with lots of content?

[ayork5]

I would like to be able to attach powerpoints. I am converting my PP to a PDF, and it does seem to work great, though, for lecture content checkpoint type quizzes. 

[em200261]

Some initial feedback: 

• When I’m prompted to select content to create the quiz from, I can only choose items within a module. Can I select the whole module? That’d be an easier way to quickly create a quiz with content from multiple modules.
• How much content can it generate from? I tried to generate a quiz using content from just 1 page (it has 3,400 words), and the tool says “Request too large. Please reduce the size of the input text, course contents or attached document.” What is the limit here? That seems a bit too small. If I’m going to have to generate questions going one page at a time (and some individual pages are even too big), that’s not very time-saving.
• Right now, the AI only generates MCQ questions? In the future, will it generate a mix of multiple correct answers, T/F, fill in the blank?
• I would like to edit the questions in the Generated Items Preview screen. They are all here for easy access. I like to see them all there and edit, not have to edit them one at a time in the next screen.
• I like the ability to add my own answer when editing and to reorder the answer choices.
• I’d like to be able to add my own feedback in the Review Items screen. Instead, I have to add feedback by clicking to edit the question once it’s in the quiz. So I have to edit it in two places at two times.  It’d be nice to have the other editing features in the Review Items screen too (align to outcomes, options, item banking).

[MarcusVu]

Hello @em200261 , 

Happy to try to troubleshoot these bullet points: 

• When I’m prompted to select content to create the quiz from, I can only choose items within a module. Can I select the whole module? That’d be an easier way to quickly create a quiz with content from multiple modules. At the moment, you can't select a whole module, but this is good feedback that bulk selection would be desired. 
• How much content can it generate from? I tried to generate a quiz using content from just 1 page (it has 3,400 words), and the tool says “Request too large. Please reduce the size of the input text, course contents or attached document.” What is the limit here? That seems a bit too small. If I’m going to have to generate questions going one page at a time (and some individual pages are even too big), that’s not very time-saving. The limit is 20,000 characters, so your 3,400 words may have exceeded that limit. We are exploring increasing the character limit prior to going to general availability. 
• Right now, the AI only generates MCQ questions? In the future, will it generate a mix of multiple correct answers, T/F, fill in the blank? Yes, in the future we're exploring essay questions and fill-in-the-blank questions next. We hope to expand the tool to more question types. 
• I would like to edit the questions in the Generated Items Preview screen. They are all here for easy access. I like to see them all there and edit, not have to edit them one at a time in the next screen. Thanks for sharing this feedback. We'll take it into consideration. 
• I like the ability to add my own answer when editing and to reorder the answer choices. 
• I’d like to be able to add my own feedback in the Review Items screen. Instead, I have to add feedback by clicking to edit the question once it’s in the quiz. So I have to edit it in two places at two times.  It’d be nice to have the other editing features in the Review Items screen too (align to outcomes, options, item banking). This is also great feedback that we'll take into consideration. 

Marcus

[atbarbary]

On the file front, whilst maximum is configured, I'd also like to see minimum. Why? In the current state, images are not interpreted and text in PDFs may be ignored if the text is unable to be read, is part of images etc. So, perhaps less than say 100 words could trigger a prompt, insufficient information - must be more than 100 words, so the user is alerted to the fact that their reference file is having no impact on the question generation. (100 words was just a suggestion). 

[em200261]

A few more points of feedback: 

My main issue with uploading files is that I get an error message (even with a doc of 1,098 character) or the file is too large. An 18-slide PPT file is too large. But it did work with two uploaded Word documents.

For the first doc, it generated standard comprehension questions. Why do all questions start with “according to the information?” (See screenshot.) I’m going to have to edit all of them.

For the second one, I asked it to write questions about tone, audience, genre, and structure/organization of the text. It did a pretty good job! I have to edit it, but it’s a great foundation to work from.