We made a mistake in Ideas & Themes decisions

shirenv
Instructure
Instructure
10
4171

Instructure.png

We won’t mince our words here – we tried out a new way to help prioritize things from the community at large, and we made some progress. But if we’re being honest, we were really only tackling the small fixes, ones that were easy to fit into a full roadmap, or ones that were adjacent to larger initiatives already planned. We’d hoped that some progress here would be better than none at all, but the reality is that those small wins haven’t made a big enough dent in the long list of requests. 

And we definitely can do a better job of being transparent about the things we are doing, why they are super important, why they might beat out other things, and showing you progress as we go. We also learned that we need to provide the right amount of context on the blog posts announcing what we were (or were not prioritizing). If we’re not prioritizing the top 5 most-voted items, it deserves some explanation.

So where do we go from here?

The reality is that we’ve got enough signal from the votes, comments, and suggestions on ideas & themes to know what’s top of mind for you. What’s also true is that we’ve got a lot to do if we were to take on all the things that have been elevated as important. And we’re also balancing the fact that some things are universally desired, and others, there’s some debate over.

Here’s what we’re going to do about it as we evolve our process.

  • Small pain-points – for things that are truly ‘easy’ to do, we just need to move a lot more quickly. We have assembled a small team of folks deeply versed in Canvas to help accelerate through as many of these quick things as we can. Expect an update on these before the end of March.
  • Larger needs – for the things that are harder to do (even if they seem like they ought to be easy), we need more help from you. Help to slice up the most valuable pieces, help to prioritize how we incrementally create value, and help as we’re working on it to provide real-time feedback. To do this, we’ve extended an invitation to some of our most prolific community contributors to spend regular time with the team planning, co-creating, doing discoveries, and prioritizing incremental delivery of value. We’re starting these sessions on March 19th. Look for more blog posts on this, following the 1st session.
  • Major overhaul items – there are some things that are so intricately connected to the core architecture of how Canvas was built, that they aren’t straightforward. The right way to address them is to re-architect some of the underlying infrastructure to enable more rapid delivery of value. On top of that new foundation, we’ll then be able to rapidly iterate. Our new CTO, Michael Lysaght, will be sharing more on this in an upcoming blog post.
  • Items on our roadmap – we need to be more transparent and visible about the great user inputs that drive the things we have prioritized on our roadmap. Not everyone will agree, but you should be able to understand the rationale and input that drove how we prioritized. Expect a blog post from the Canvas team following every sprint (approx every 2-3 weeks), starting on April 1, 2024.
  • Ideas & Themes – we heard your feedback that it’s unclear how this process informs the priorities and we want to take time to improve it. We’ve identified areas that are working and areas that are not, and while we’re gathering more feedback from you, we’re going to pause the next voting window, expected in March/April. In the meantime, we will continue to review ideas as they come in. Expect more information by June. 

It took us a bit to respond to the original post and some of the thoughtful comments on there. It sparked a lot of discussion internally, and was paired with a desire to respond, not just with words, but with a thoughtful plan of action. I recognize this meant that it took a bit to respond.

I hope this post helps clarify our position, and our continued commitment to make a great product for educators and learners everywhere. And we greatly appreciate your patience and understanding as we continue to get better at this.

 

10 Comments
JamesSekcienski
Community Coach
Community Coach

@shirenv 

Thank you for taking the time to make this blog post and the time you and the various teams have spent discussing the concerns from the Community, especially after the most recent prioritization results were shared.

That sounds like a good idea to put a hold on the next round of voting while you work to make improvements to the process, especially since we are still waiting for development work to finish on the prioritized ideas from the first round of voting.  For clarification, are the prioritized ideas from the last round of voting still moving forward with development or are they being put on hold too?  If they are still moving forward, when should we expect to receive an explanation on each of the other ideas for why they weren't prioritized?

I am looking forward to seeing more regular updates from the teams sharing their progress.  Some teams have been very transparent about what they are working on and the product managers have been very engaged with the Community and responding to users.  Even if we don't agree, this level of transparency and engagement is greatly appreciated.  I have been watching New Quizzes and Discussion Re-Design posts closely, especially since they were prioritized, and I have really appreciated the regular updates and engagement from these product managers.  However, the Rubric Re-Design is an example of a high impact project that would benefit greatly from more transparency and engagement with the Community.  This was the top voted theme in the first round of voting and yet limited information has been shared and questions from the Community are left unanswered.

Getting community contributors involved in the process sounds like a great way to involve key stakeholders on projects and provides the Community with more representation in the development process.  As you re-evaluate how the process will work, I would like to suggest again bringing back the ability to comment on individual ideas for a few reasons:

  • Allowing the community to comment on individual ideas year-round provides opportunities for all Community members to share their feedback while it is fresh.  Needing to wait until an arbitrary fixed window of time to comment makes it more likely to forget something that you wanted to share.  It is even worse when an idea you want to share information about isn't a part of one of the themes that is open for voting.
  • Some ideas may seem universally accepted, but there may be some edge cases that most users don't have to deal with that may be overlooked.  This would provide an opportunity for Community members to share potential concerns about a suggested Idea.  This would be especially important if it is considered a small idea that may be prioritized for quicker development.
  • Some users submit ideas that others already have a solution to handle or a workaround to address the issue.  Allowing others to comment they can share these solutions and/or workarounds that could be beneficial while waiting for the Idea to be addressed.
  • Some users also have more to share about an idea and want to build on to it and/or contribute alternative solutions.  Allowing them to comment on the idea directly rather than making a new idea that gets merged can also help reduce the number of ideas to review.  This would also help address the current problem where when your idea gets merged with someone else's you no longer have access to comment on it.
  • The user discussions on an idea also provides more insight into what various users find most important, which can help guide community contributors and product teams when discussing these larger ideas.

Also, since you have mentioned three different levels of ideas (small pain-points, larger ideas, and major overhaul items), it would be nice if ideas were tagged with the level they appear to be after they are reviewed.  This can help the Community better understand how much work you perceive the idea as potentially needing.  This would help us to be more understanding of an idea that needs a major overhaul taking significantly longer to be prioritized/implemented compared to small pain-points.  It would also help to clarify items that may be perceived as small pain-points, but are actually larger ideas and/or major overhauls based on the solution that would be needed.

mbmacdonald
Community Participant

Thanks so much for this post. It's incredibly rare these days to hear anyone unequivocally admit to a mistake.

completely agree with everything @JamesSekcienski mentioned. I have been submitting a huge number of "ideas" recently which are really just minor things, when what I really want to do is *comment on existing ideas.* Because often what I'm suggesting is either an expansion, counterpoint or different perspective on an existing idea.

When my users come to me with a question or issue, I like to be able to include in my follow-up a record of whether the issue has already been submitted as idea to Instructure, and if not, I submit it myself. So often, there is an idea very similar to mine but not exactly, so I am forced to submit a new idea.

nuscitpkg
Community Participant

This is timely. We are a relatively new Canvas customer. When we joined, I enthusiastically sent a few suggestions via the previous Canvas Ideas system. I noticed that a lot of popular ideas were never implemented. I became quite skeptical about the whole process.

Then Canvas changed to Ideas & Themes. By this time, I was wary of suggesting to our teaching staff that they should get involved. I noticed the same things were happening, just with a different dressing - popular ideas and themes were getting ignored.

We have had a couple of feedback sessions with our CSM, but for many of the items we brought up, we were redirected to Ideas & Themes.

I hesitate to encourage our colleagues to do this as Ideas & Themes is seemingly a black hole. (Not to mention that before you can post an idea, you need to earn the right to give feedback. A bit ridiculous, if you ask me. Not sure if this is still the case, but I had to jump through a few hoops before posting an idea in Canvas Ideas.)

I think providing concrete reasons why specific ideas and themes are passed over will be an improvement over being told that this is not in Canvas' list of priorities (or however else Canvas chooses to phrase it).

Thank you for acknowledging these issues. However, I will wait and see how things improve before I attempt to encourage our teaching staff to be active participants in this.

John__Lane
Community Explorer

Hello Community

Also coming from a recent transition from a different LMS, I found the 'Theme" concept to be a bit convoluted and confusing.  Just wrapping my head around what a 'theme' was, instead of something more descriptive and straight forward.  I'm also thankful how the transparency of the process will be embraced.   Looking forward to big positive changes!  

dave-long
Community Participant

Thank you for acknowledging the mistakes here Shiren, and the plans to address them going forward. 

In general, I appreciate transparency in how decisions get made.  While I may not agree with the outcome, being able to understand the rationale behind how the decision gets made helps.  I'd encourage this transparency to continue as Instructure works through these changes.

 

 

DerekStevens
Community Member

@shirenv 

Introducing "organizational" level votes could be helpful to balance / enhance the voting landscape. Organizations roll-up faculty feedback, spend resources on training, and aggregate support tickets. 

You could:

  • assign 1 organizational account to each company
  • place limits on how many votes they get in a calendar year
  • weighting the votes based on the user volume could be considered

 

dkpst5
Community Participant

I'm glad to hear of your small pain points "strike team". Here's a perfect example of what's been missed entirely by both the previous and current Ideas solutions, but is an easy target for such a team:

https://community.canvaslms.com/t5/Canvas-Ideas/Syllabus-Add-Syllabus-Page-History/idi-p/359508

  • "Syllabus" is essentially a page already.
  • Canvas already has revision history for pages.
  • This special pre-fab page with the extra feature (course schedule) does not have revision history.

It's not a sexy feature. It's easily overlooked in the mass of bigger, sexier features. It's not necessarily high-impact (until it is needed), and, for implementation, it is largely just a slight evolution of what already exists.

I don't post this necessarily to evangelize this specific update (though that would be great!), but just to highlight the kind of things my school and I are hoping to see out of the process: bugfixes and polish.

JamesSekcienski
Community Coach
Community Coach

@dkpst5 

I agree that it is great to hear about the "strike team" and look forward to find out what they will be targeting.  

I agree that adding page history to the Syllabus would be extremely beneficial and would be very beneficial for Assignments too.  Unfortunately, I'm not sure if they will consider this in the small pain points since Syllabus isn't currently a Page.  If you look at the API support for accessing the Syllabus, it seems to be limited to a part of the Course details.  Since there isn't API support for version history as a part of this endpoint, it would require an update to present this in a new way too.  Also looking at the URL for the Syllabus it looks like it may be more closely related to an Assignment than a Page for some reason since the URL ends with "/assignments/syllabus".  Since Assignments don't currently have version history this may be part of why the Syllabus doesn't.  Just to add on to this, it would also be nice to be able to link to the Syllabus from Modules without having to use the external URL option which doesn't work well for course copies.

emily-mckeag
Community Member

Thank you so much for this thoughtful response on how to approach themes and ideas. The points highlighted above make a lot of sense and I really appreciate the time and care you all took to address the feedback and concerns around themes and ideas. 

One thing I didn't see addressed here is the feedback around having separate themes/ideas/voting for the different Instructure products like Catalog, Studio, Credentials, Mastery Paths, etc. Each of these Instructure products seem to get lumped in with Canvas even though they are their own separate products with their own needs. I can only speak on my experience with Catalog but I think that Catalog could benefit from having its own themes and ideas and voting to address needs and concerns on that platform.

dave-long
Community Participant

@shirenv Will the "Items on our roadmap" blog posts be tagged with the "Roadmap Processes" tag? 

Looking at  https://community.canvaslms.com/t5/The-Product-Blog/bg-p/tldr I see the following posts today (April 10th, 2024):

Title Author Tags
Canvas Rubric Redesign: Progress, Challenges, and Next Steps Ravi Koll Canvas LMS
Provide Feedback on the New Sample College Prep Item Bank! Chris Nguyen Mastery Connect, Mastery Item Bank
Welcoming Feedback: Commons is Back in Action! Kata Keri Canvas Commons
Studio Automatically Captions Any Uploaded Media From Today! Akos Farago Canvas Studio
Releasing Modules to Students and Sections Allison Howell Canvas LMS
Measure Mastery with any Canvas Assignment Cason Mastery Connect
Data Access Platform (DAP) CLI and Client Library v1.1.0 is now available Bob O'Dell Admin Analytics, Canvas Data, Developers, Release Communications
Introducing Enhanced Staff Management APIs for Canvas Credentials Dora Jekkel Canvas Credentials

 

Which, while having a lot of useful information about the specific topics, I wasn't sure if these blog posts were the ones you mentioned that would be from the Canvas team following every sprint.