We're Listening: How Your Feedback Shapes Our Future

[shirenv]

At Instructure, we believe the best way to improve is by listening to the people who use our tools every day—you.

That’s why on June 21st US customers will begin seeing a quick question pop up while using Canvas or Mastery asking for your feedback. It’s part of something called the Product Net Promoter Score (NPS)—a simple, powerful way for us to hear your feedback and act on it.

What is NPS?

NPS asks how likely you are to recommend Instructure Products to a colleague or friend. It might seem simple, but it tells us a lot. Your response—and the reasons behind it—helps us understand what’s working well, where we can do better, and how we can better support your goals.

Why Do We Ask?

Because your voice matters. We want to do more than just provide software—we want to be your partner in creating amazing learning experiences. NPS gives us a window into how you’re feeling, what you value most, and how we can evolve to better meet your needs.

Where Will You See It?

We we start deploying NPS surveys in a few ways:

• Where:
  • Canvas NPS surveys will be located in the Canvas Dashboard 
  • Mastery NPS surveys will be located in the homepage, admin analytics page and tracker page. 

• Who: 
  • All users EXCEPT for students. If a user is a student in ANY context within the app they are being surveyed in, they will NOT get a in-app NPS survey
  • All US Customers only, (International customers will begin seeing NPS at a later date when the survey is compliant with international standards)

We promise the survey will be concise, respectful of your time, and—most importantly—meaningful.

What Happens Next?

Every response you share is reviewed and considered by our teams. It helps shape our roadmap, improve our services, better your customer experience, and ensure we’re aligned with what matters most to you and how we can best support your needs. Your insights don’t just sit in a spreadsheet—they help us make real changes. 

I hope you take the time to share your feedback with us. Thank you for your partnership.

For more questions, see our Canvas FAQ and Mastery Connect FAQ

[jenn_stevens]

Sounds interesting!

• I would love to see a mockup of what this popup would look like, where it would be located on the dashboard, and what the user experience would be.
• Is it numerical answers only, or are short answers allowed?
• Are there any plans to share the data back to Canvas admins?

Thanks!

[rake_9]

Everything @jenn_stevens asked, plus -

Can you say more about "...asks how likely you are to recommend Instructure Products to a colleague or friend"? Will the survey in Canvas differentiate what products are available to users at a particular school?

How often will instructors see this survey?

[cheryltice]

Thank you for sharing information about the upcoming NPS surveys. While I appreciate Instructure's stated commitment to gathering user feedback, I'd like to respectfully express some concerns shared by many in the teaching community.

The Canvas Community already serves as a dedicated feedback channel where many educators invest significant time providing detailed insights. However, there seems to be a growing perception that while feedback is collected across multiple platforms, it doesn't consistently translate into development decisions prioritizing educator needs and concerns. Adding another feedback mechanism raises questions about how all these input channels are synthesized and prioritized.

Institutions face challenges with the current pace of Canvas updates -- I know ours does. Changes, like Discussions Redesign and Checkpoints, and others, require faculty training, documentation updates, and workflow adjustments that impact teaching time and effectiveness. Some of the changes occurred before major issues were addressed, even though they may have been brought to Canvas' attention in the community. Has Instructure considered:

• Implementing an opt-in model for updates, allowing institutions to adopt changes on timelines that align with their academic calendars and offer time to roll updates out over a reasonable length of time?
• Maintaining feature parity with legacy systems before deprecating functioning tools that educators rely on to avoid major workflow issues (discussion checkpoints come to mind)?
• Creating a more transparent roadmap showing how Community feedback directly influences development priorities?

I value Canvas as a teaching platform and appreciate efforts to improve it. However, the most meaningful partnership would involve not just collecting our feedback through various channels but demonstrating how that feedback is important and shapes development in ways that respect the core teaching and learning mission of educational institutions.

I look forward to seeing how the NPS data will be integrated with existing feedback channels to create a more responsive development process that balances innovation with stability.

[JamesSekcienski]

@shirenv 

The following is stated above about who will receive the survey: "All users EXCEPT for students. If a user is a student in ANY context within the app they are being surveyed in, they will NOT get a in-app NPS survey"

How do users who are primarily Teachers or Admins get access to the survey if they are also enrolled as a Student in a course?  Teachers who also take classes will be enrolled in courses as a Student too.  In addition, at our school, faculty and staff are enrolled as a Student in training courses, so we would anticipate most, if not all, employees are enrolled as a Student in at least one course.

Also, are users that are only enrolled as Observer included in the survey?

[dspiel]

I don't think there is a single college that you will get responses from. We do not have a single employee or faculty that isn't a student in at least one course. I would be shocked if you end up with a 100 responses from around the world.

[alexchapdelaine]

@jenn_stevens 

Thanks for the questions, Jenn! 

1. We have videos of this experience and more details linked in the FAQ. 
2. The In-App NPS Survey is two parts, first numerical response and second short answer response.

Yes, per the FAQ customers will be able to request data responses from your CSM on a quarterly frequency.

[alexchapdelaine]

@rake_9 

Thanks for the questions, Valerie!

1. Great question, at this moment we do not have that level of granularity. The survey will measure holistically the product brand (Canvas or Mastery) that it is hosted in. You can read in more detail in our FAQ.
2. Per our FAQ
1. Sampling Methodology: Instructure employs a randomized sampling methodology to ensure a representative sample of users is surveyed. Approximately 1/365th of the eligible user population (all non-student users) are presented with the survey each day.
2. Survey Frequency: a user is only presented with the in-app NPS survey once every 365 days within a given application (i.e. once a year in Canvas and once a year in Mastery Connect).

[dbrace]

Add my instittution to the list of institutions that very rarely has a Canvas user account that is not enrolled in at least one course as a "student".

-Doug

[ScottHahn]

All of our instructors, TAs, staff, and admins are enrolled as students in our training course, Growing with Canvas. This would disqualify our entire campus. 

- Scott

[rake_9]

While I don't have hard numbers / percentages, many of our instructors also have student enrollments for one reason or another.  That's gotta skew the results a bit.

[csato]

Thank you @rake_9 @dbrace @ScottHahn and @dspielfor your concern on this point! In an effort to be as conservative as possible in our survey methodology, we elected to remove anyone with a student role to eliminate any risk of a user who is primarily a student from receiving the survey. This is a primary concern for many of our customers in our community.  We felt it best to exclude them from receiving the survey until we have a viable solution that ensures no users who are students receive the survey. 

[cms_hickss]

@shirenv 

Like many others, UCF's faculty and TAs are often students in other courses. Even as an Admin, I'm in test courses as a student.

Is there a reason you are not targeting the dashboard announcement toward the roles of Teacher, Designer, and TA?

[csato]

Thanks for feedback and thoughts @cheryltice 

We hear you and several of your colleagues have expressed these thoughts. The Canvas Community will continue to remain a valuable space for sharing insights. We're also actively improving how we gather and utilize feedback from the broader community, and the In-App NPS Survey is an important part of this evolution. Our goal is to better understand our customer's priorities and ensure Canvas delivers the solutions you need.  Our success is defined by the value Canvas delivers to you and NPS is one of the best metrics to hold ourselves accountable.

Addressing the challenges you mentioned, Shiren posted a blog earlier this March that touches on a few of the things you’ve highlighted here and how we are addressing them. However our commitment to easing the burden of change management for release and deploys with institutions, maintaining parity across tools and features, and making a transparent roadmap are all being tackled through our Customer Connectivity Program we’ve been operating since the start of this year. While we are not where we want to be, I want to assure you that our Product, Engineering and Customer Experience teams are all focused on deepening trust back through this program.

[csato]

Hi @cms_hickss, great to hear from you. To eliminate any risk of a user who is primarily a student receiving the survey, we opted to exclude anyone with a student role. Ensuring no students received the survey is critically important for several of our customers, especially with our K12 customers. When we have a plan in place on how to address this issue, we will update you all through our communication channels. 

[JamesSekcienski]

@csato 

I certainly understand the challenge of trying to identify users that are primarily a certain role since users are able to have mixed roles.  This is a challenge that admins face with things like global announcements and developers/admins face when building customizations that should only target certain users.  It would be nice if there was a way to set a user's primarily role in Canvas.

However, the current restriction seems too extreme to eliminate everyone that has a Student role anywhere within the system and prevents a lot of users that should be eligible to participate from giving their feedback.  As @cms_hickss shared, even admins may be enrolled as a student in a course and based on the current restrictions they would also be excluded from the survey.  In my opinion, it seems more concerning that a user who is primarily a student and shouldn't complete the survey would also have access as an admin.

Have you analyzed how many schools/universities would have little to no eligible users for the survey due to this restriction on student enrollments?  Is there a way to provide institutions with an option for who can receive surveys based on certain criteria that you can use for filtering users?  I look forward to hearing your plans to address these concerns so that our faculty/staff would be eligible to give their feedback too.

[jenn_stevens]

I definitely like this better than the previous system of doing the surveys via announcements - that has caused me trouble before when the president wanted their announcement front and center and there was !!this other announcement!! This seems like a much more distinct difference.

Also everyone who is concerned about faculty not getting survey requests because they have student roles, ssh don't look a gift horse in the mouth! I am okay with this as well - it means faculty aren't survey-fatigued when I want to survey them. 😜

[csato]

Hi @JamesSekcienski, thanks for the additional context. As we relaunch In-App NPS we recognize the conservative nature of our approach. The survey data collected will be incredibly value to Product and Engineering teams and we are taking the approach of progress over perfection with our criteria of eligible survey participants. We will continue to work on refining our strategy with eligibility requirements while also maintaining our fundamental policy that no student should receive a survey ever. More to come when we have a solution available! 

Additionally regarding a question you posed earlier about observers, they will be included in the survey. 

[cms_hickss]

Hi @csato 🙂

This is going to be a little blunt and a little harsh sounding, but.... One of the first things this blog post said was that our voices matter, but if you are not going to receive any responses from our faculty (because you have eliminated them all), then do our voices count?

You also mentioned one of the reasons that you were omitting teachers, TAs, and designers who also have the student role is K12. How many students in K12 have teacher, TA, designer, or admin roles?

[hollands]

I'm sure others have mentioned this but I think it would be beneficial for Instructure to put out a URL that can be sent to other constituents at our institutions. Many of our instructional designers, faculty and other staff have student enrollments and many would likely want to participate. 

[leward]

@shirenv et al.

Dear Instructure,

Apparently you are NOT listening.  Your customers are telling you that your proposed methodology will not work.  At most institutions, the number of users who have no student enrollments is extremely low.  There must be a better way.  Why not, for example, look at the ratio of non-student enrollments to student enrollments for each user (information easily extracted from the Canvas API).  Users who have a overwhelming majority of non-student enrollments are likely not students.  Or you could exclude users who have ONLY student enrollments.  Even if an occasional student gets the survey, wouldn't that be better than excluding most of the eligible respondents?

Also, you already have an incredibly rich source of feedback in the form of customer ideas. In addition to the ideas themselves, in the past you allowed customers to vote ideas up or down, which empowered individuals and institutions to throw their support behind the ideas they deemed important.  Customers (and you)  were able to sort ideas by vote count and filter by status, offering a level of transparency and accountability that long since disappeared.  Today it's impossible for users to easily ascertain, for example, that the idea of adding rich text support to Messages has been around since the early days of Canvas and garnered a huge number of votes over the years.   I'm not necessarily opposed to the survey (if you can resolve the eligibility issues), but what I really want to know is whether ideas important to my institution have gained significant traction in the community, and, if so, whether, when, and how Instructure plans to respond to them.

As someone who has been using Canvas since 2012, I have watched Instructure evolve from an extremely open, transparent, and customer-centric company with an amazing product to a company that is no longer listening to its customers, making poorly designed and inadequately tested changes, and removing functionality without understanding the adverse effects.  The NPS is yet another example of a major initiative designed without customer input that doesn't work for your customers.  Please show your commitment to listening by going back to the drawing board on the NPS and announcing your plans for better handling ideas submitted by customers.

I apologize for the snarky tone of this message.  I really want to fall in love with Instructure and Canvas again, but this latest announcement, like many moves over the last few years, doesn't inspire confidence.

[cheryltice]

I'm listed as a student in some courses. Would that mean I won't be able to participate in these surveys? I'm an account admin and also teach classes. All of our instructors are also listed as students in some courses for professional development and faculty senate communication, for example. 

Am I understanding this correctly? It appears many others in this thread have the same concerns.

Cheryl

[RetaChaffee]

I am thankful that you will not be including students in these surveys.  I am however very concerned that you will be including faculty. I work at a college where our instructional design team works in conjunction with the academic directors and SMEs to develop the online courses.  The faculty teaching the courses are focused on teaching and engaging students not evaluating the LMS.

I would see this pop-up as an annoyance and a commercial intrusion on the academic nature of our online programs. The company should not be surveying our academic community. With that said, if a faculty or staff member is interested in being engaged in the Canvas Community, then perhaps they can choose to opt-into surveys as a 1-time invitation. 

Regardless, the institutions should have the option to determine whether or not they want to enable the NPS.

Reta Chaffee

[JenniferJWhite]

I appreciate the impulse and need to collect feedback from users, but I have concerns about the approach and selection of participants, as there are a few blind spots.

I am particularly wary of this statement: "If a user is a student in ANY context within the app they are being surveyed in, they will NOT get a in-app NPS survey." I'm afraid that NO ONE at my institution (including root admins like myself) will receive the survey request for Canvas or Catalog, as every single person at my institution is enrolled as a student in at least one course due to the professional training or certifications that we all must take via our instance of Catalog, which of course links to Canvas courses (e.g., mandatory FERPA training, security training, institution orientation, travel card training, etc). We also have many admins enrolled as students in test courses (or have test enrollments in courses). 

Is there not a better way to implement the survey? Maybe you could choose to offer the survey based on ACCOUNT role (instead of course role, as seems to be the current plan)? If this survey is meant to be meaningful, as you say, it seems you may be missing valuable input from many users by using such a crude selection approach.

[DrTerriC]

@JenniferJWhite is making a good point.. 

We have had problems with an LTI vendor on roles, they are doing it as binary much like you are.. instead they are doing the opposite - anyone who was a teacher was always a teacher and never a student. 

It seems both that experience and this survey selection might be providing valuable feedback to Instructure about the nuances of how roles are used..  and that the programming around assigned roles needs to be revisited. Since putting peers in a course to test user experience is a best practice, this will likely continue to be a problematic approach. 

Terri 

[jmeierdierks]

I would like to echo what many have stated that you will likely get zero responses from our institution based on the logic "If a user is a student in ANY context within the app they are being surveyed in, they will NOT get a in-app NPS survey." We have training courses that admin, faculty, and staff are enrolled in as students. I will be very curious to know how many responses you receive to the survey since it's targeted to an extremely small percentage of the Canvas population (which will skew results).

[TrinaAltman]

@JenniferJWhite The account role would not work for us if the goal is to collect feedback from instructors, TAs, and Designers. We do not use account roles for instructors, TAs, or Designers at our institution - they all have course roles.

[JenniferJWhite]

Fair point, @TrinaAltman - The selection doesn't have to be exclusively by account role or exclusively by course role, but a selection of both: make the survey available to specified account roles OR teacher/designer course roles. But some institutions would rather not have their teachers give feedback, so there are institutional preferences, too.

As @DrTerriC says above, finding a more nuanced way of selecting populations for the survey would be helpful, and we're giving Instructure food for thought on that point!

[nwilson7]

I was going to echo the sentiments by others that I am pretty positive none of our users are going to get the survey based on the student role elimination but then started thinking through it more and it could be even worse than just that.  If we do have an Admin/Teacher/TA that does not also have a student role in a least one course that would mean they are a rare user of Canvas and basically have no affiliation with our university.  Basically some of our contract employees (such as our OPMs) would be the only ones "representing" us.  That seems even worse to me than just no one even getting it.

-Nick

[cms_hickss]

@RetaChaffee - You can opt out of the NPS survey. It's not mentioned on this blog post, but over in their "In-App Canvas NPS Survey FAQ" there's a bullet point for that. You basically have to ask your CSM to opt your school out.

[RetaChaffee]

Thank you so much, @cms_hickss for pointing that out.  I missed that FAQ and it's good to know.

[ArohiDesai]

This has been a very constructive conversation - thank you for the candor. 

When we began this process, we asked many of you how we could improve Product NPS collection. The clearest feedback we received was: "Do not put this out to students." We wholeheartedly agreed. Protecting student boundaries is a core priority in every decision we make. 

What we're sharing now is an initial approach to collecting Product NPS in a way that fully respects the student experience. As @RetaChaffee aptly put it, "I would see this pop-up as an annoyance and a commercial intrusion on the academic nature of our online programs."

We have incorporated your feedback into ongoing discussions about roles and how we can better gather feedback from non-student users. We don't have any changes to announce now, but will keep you updated as those conversations evolve into concrete, iterative changes.

[LukePratherCUB]

I notice in the screenshots and preview video the survey will allow users to see the Informed Consent Policy which is good. However, I anticipate additional questions and concerns from our instructors that are more clearly answered in the FAQ. Please consider including a link to the FAQ (or a customer-facing version of the FAQ) on the survey widget so our Canvas admin and support teams do not have to re-share information when questions are raised.