Can we talk? Usability testing of Canvas Discussions Redesign with instructors and students

[englu061]

Day 2: July 10, 2024 at 10:00am

Presenters

• @englu061 
• @SamGarza1 
• @froye001 
• @rgeorgeb 

Session Overview

The session outlined findings from a usability study conducted on the Canvas Discussions tool, focusing on collaboration with Instructure to improve its functionality. Based on four key findings, the study aimed to understand user experiences to implement changes and/or improve training. 

Usability partnership & study process

Since 2015, the University of Minnesota academic technology teams have been conducting usability studies to understand how instructors and students use different learning technologies. This work is carried out with internal and external stakeholders such as University of Minnesota academic technology leadership, the academic technology community, and our learning technology vendors.

For this study, the Canvas Discussions tool was selected for usability evaluation for two reasons: Instructure was redesigning the tool, and it’s used in approximately 25% of University of Minnesota academic courses. Discussions are heavily used in those courses: as often as 14-15 times during a 15-week semester-long course.

In conjunction with the University of Minnesota Usability Services team, our project team developed a focus question to guide our study. We wanted to know: What prevents people from engaging with one another using the new Discussion tool in a satisfying way? We then determined the criteria for participant recruitment and tasks that participants would complete during the usability testing sessions. While the participants were participating in the usability testing sessions, project team members took notes that captured their observations about what the participants did/said and points of confusion. After completing all usability testing sessions, the project team met to debrief and review issues that the observers captured in their notes.

Summary of Findings

1. Mentions Feature:
• The "Mentions" feature allows users to tag others in discussion posts using the "@" symbol, similar to social media platforms.
• Students immediately recognized this feature, while instructors were unsure of its functionality and notification settings.
• Based on these findings, UMN plans to educate both user groups about the feature and ensure they know how to turn on notifications through Instructure's knowledge articles.
2. Anonymous Discussions:
• New settings include options for partial or full anonymity in discussions.
• Instructors found setting up anonymous discussions problematic, as options were only available during initial setup.
• Instructure will modify the tool to allow editing these settings post-creation, aligning with the functionality of Assignments and Quizzes.
3. SpeedGrader Confusion:
• Instructors confused discussion replies with feedback comments in SpeedGrader, leading to miscommunication.
• The plan to help mitigate this confusion includes extra communications to clarify this distinction and training new instructors and TAs about proper feedback channels.
4. Group Discussions Feedback:
• Instructors expressed dissatisfaction with the Groups feature, describing it as difficult and cumbersome.
• Although not the primary focus, this feedback was documented, and Instructure is beginning to explore improvements.

Future Directions

The collaboration between UMN and Instructure in UX studies is still developing. Plans are to invite Instructure to future studies and use the findings to inform communication and training for new features.

Resources

Presentation slides