New Ideas & Themes Now Live

[TaraGoldman]

Our team has been anxiously awaiting the official release of the reimagined process formerly known as “Idea Conversations” (and we hope you have too!). I am happy to announce that Ideas & Themes is now live. You can see this new process in action in this video. 

Get to Know Ideas & Themes 

Over the past 60 days, our team analyzed the historical open ideas and identified about 75 themes. A “Theme” is a grouping of ideas that outlines two things:

1. the higher-level problems to be solved
2. what value could be provided to users 

As you begin reviewing the Themes, you’ll see that Ideas are grouped together in the way that makes most sense for the product development process. As Community members submit new Ideas, we'll connect with the submitters via the Community to ask clarifying questions and facilitate a dialogue to ensure understanding and alignment around the need. That’s the beauty of this process: it’s not intended to be static, but a dynamic way to continuously identify and prioritize the needs of our Community. 

In addition to the video, our team has created in-depth user guides to walk you through each step of the process. 

As you review Themes, we encourage you to subscribe to the themes that are most interesting to you so you can get updates throughout the process. 

What Comes Next?

Mapping all of the open ideas to themes was no easy feat and, to be honest, took us longer than expected, so we will be rolling this process out in phases:

Referenced video and how-to-guides

Like any new process, we know we may need to iterate and evolve it. Please use this survey to provide any initial feedback you may have as you dive into the process. 

We are really looking forward to this improved way of collaborating together. We know this will reignite a critical–and very fun–part of the Community in a meaningful and proactive way. 

[christopher_gil]

I like and appreciate the effort to make this process streamline and efficient!! 

If I can make a suggestion? What about a window to the right/left that shows Tags that may relate to the Theme Category I am looking at. 

For ex. I am looking at Theme: Open for Voting but I don't know what to type in the search, but a keyword list of Tags may help me click for more. 

Also, a filter for K12 or HE may be helpful. Thanks! 

[Renee_Carney]

Hey @christopher_gil 
This is a great suggestion and we'll chat about it more as a team, but it may be a while before we could implement anything with tags.  Tags actually do not function as one would expect them to in Khoros, so we strategically do not use them.  

[JohnElsbree]

When should we expect to see all the previously submitted ideas from the old "Canvas Ideas Conversations" mapped into the new "Themes"? Is that the February 8 date in your timeline?

The specific one I'm looking for is this oldie-but-goodie, originally posted 4+ years ago:

Stop Canvas Modules from Auto-Publishing All Conte... - Instructure Community (canvaslms.com)

[KristinL]

Hi @JohnElsbree - The existing Themes are nearly complete, but we plan to add many more themes in the next week. It is our plan to have all ideas that we found mapped to a theme addressed and moved into the new Ideas & Themes space by February 8. You will likely see Identified Themes a few days earlier.

[canvas_admin]

I think it would be helpful on the "Ideas and Themes" page to be able to see a banner or message as to when the next voting window will open as well as to when the next submission window will open rather than having to go to another page to see some timeline, click through links on the page, etc. 

[kmallen]

Hi @TaraGoldman,

Can you clarify the role of the referenced questions that are linked to each theme? Is your team planning to develop all of the referenced ideas under the theme?

Thanks!

[KristinL]

Hi @canvas_admin - Starting February 13, the submission window will always be open. We have plans in place to add a banner in the entire Idea & Theme space when a voting window approaches so everyone knows when that will take place. Thanks for sharing your thoughts with us! 🙂

[Renee_Carney]

Hi @kmallen 
That is a great question and one that we will cross-check with the documentation of the process to ensure the answer is clear.  Once a theme has been developed and ideas are linked to it, the process of developing a solution is based on the theme and not individual ideas.  It is possible that when a theme is completed that some referenced ideas will be marked complete, but not all.  The ideas that are not completed will be revisited to see if they map to inform another theme, or if they should be archived.  

Let me know if you still have questions. 

[Charles_Barbour]

I understand that this is a complex problem and the concept of themes seems sensible but I'm a little unclear on a couple of areas:

1. If there is an idea which may require relatively low effort to implement, but it resides within a theme which is not selected by Instructure for voting (let alone receiving enough votes to get prioritized), does that mean there is no chance for that idea to be worked on until at least the next round of voting?
2. I presume not all developers at Instructure work on the same things.
   1. How will you balance developer time across groups and ensure maximum ROI in a situation where the themes which have been prioritized may not contain any/many ideas for improvements in all developer areas?
   2. Can ideas from other themes be selected for development despite the parent theme not being selected? If so, how will those be selected?
3. In addition to community submitted themes and ideas, Instructure obviously has their own (internal) themes and ideas.
   
   1. Will those be shared and included in the themes and voting process?
   2. If not, how will Instructure balance the two sets of priorities?
      
      I recently saw this internal conflict play out with a VCMS vendor when I was serving on their I2 advisory board. The board was asked to organize our requests, narrow them down, rank them, and they were added to their roadmap. The vendor then proceeded to release a bunch of features which were not on our lists, had never been discussed with us, and were intended to attract and fulfill requests primarily from their corporate customers.

I understand this may be an iterative process so it may change as you implement it (at least I hope so), but my gut tells me that 75 themes may be too many. Having so many themes may make it hard for many great ideas to get the attention they deserve. Even if 5 themes are selected for voting twice a year, it would still take 7.5 years before all of them are voted upon in the best case scenario. And that's just voting on themes, not the actual results of the votes or the implementation of the ideas.

While prioritization and organization are obviously important, it feels like more/faster development is what's really needed. Hopefully you can figure out a way to accomplish both, but neither one is a replacement for the other.

[dave-long]

Excited to see this new process in action! 

[leward]

Once again, I fear that this process will result in ideas with hundreds of votes over many years that would address major shortcomings in Canvas will continue to be ignored because they happen to have landed in a lower priority theme.  With customers voting for themes rather than individual ideas, the evidence that certain ideas are hugely important to customers will be lost.  I think it's important to retain idea-level voting data and continue to allow customers to vote for ideas.  This would allow theme-level voting and prioritization to be the primary driver of development while at the same time not losing track of ideas that have a huge amount of community support regardless of whether they get assigned to a theme and/or the status of their assigned theme.  Also, I've submitted many ideas of the years for changes that I (and many others) regard as bug fixes because they implemented solution is either counter-intuitive or results in internal inconsistencies.  Some of these suggestions would require minimal effort and deliver a huge return in terms of user experience.  To ingnor suggestions like this simply because they don't happen to land in a high-priority theme would be very unfortunate.

It would also be helpful if a direct link were provided to the main ideas page, so users can easily search for ideas and determine whether and which themes they are linked to.  Oh wait, I now see that the Submit and Idea button on the main themes page already does this.  Maybe change the label to View or Submit Ideas to make it clear that the button goes to a top-level idea-centric page.

[Charles_Barbour]

After a casual review of a few dozen ideas, some of them are obviously bugs or trivially easy changes to the text or UI.

It shouldn't require so much effort to get items like these fixed as they're just basic common sense or usability issues.

• [Speedgrader] Grade Validation Error in Speedgrader
• [Files] Student user files alert when space is full
• [Speedgrader] Audio or Media Comment upload failure confirmation
• [Course Settings] Do not allow course end date prior to start date
• [Notifications] Use error message to give helpful information
• [SpeedGrader] media comments: warning to stay on page until comment fully uploads/processes
• [Pages] Fix needed - unaware 'prerequisite' text is clickable
• [Speedgrader] Move the emoji image out of the comments textbox

It feels like under the new system, issues like the ones above have roughly the same chance of being fixed as something like [User Settings] Automatic clean up of discarded/inactive users seems has of being implemented. That seems wrong to me.

Stuff which is broken or should be fairly easy to fix should get fixed first (and quickly).

Then you prioritize the other enhancements.

But figure out a way to stack up some quick wins while you work on some of the larger pieces.

[christopher_gil]

So, again, thanks for working to make this process better!  

after reading what  @Charles_Barbour  wrote (below) Wow, it might be suggested that an easier way to identify obviously bugs or trivially easy changes to the text or UI VERSUS feature requests. 

Maybe this does exist already, maybe I have missed it.  

===

After a casual review of a few dozen ideas, some of them are obviously bugs or trivially easy changes to the text or UI.

It shouldn't require so much effort to get items like these fixed as they're just basic common sense or usability issues.

• [Speedgrader] Grade Validation Error in Speedgrader
• [Files] Student user files alert when space is full
• [Speedgrader] Audio or Media Comment upload failure confirmation
• [Course Settings] Do not allow course end date prior to start date
• [Notifications] Use error message to give helpful information
• [SpeedGrader] media comments: warning to stay on page until comment fully uploads/processes
• [Pages] Fix needed - unaware 'prerequisite' text is clickable
• [Speedgrader] Move the emoji image out of the comments textbox

It feels like under the new system, issues like the ones above have roughly the same chance of being fixed as something like [User Settings] Automatic clean up of discarded/inactive users seems has of being implemented. That seems wrong to me.

Stuff which is broken or should be fairly easy to fix should get fixed first (and quickly).

Then you prioritize the other enhancements.

But figure out a way to stack up some quick wins while you work on some of the larger pieces.

[TaraGoldman]

@Charles_Barbour and @leward : Thank you for your feedback and I’m going to do my best to address your concerns and questions as we’re noticing a couple of themes. My goal is to give a window into our product development process and its many nuances and complexities. 

1. Quick Wins & Level of effort: As power users of Canvas, I can understand that there may be perception of what may be considered a quick win with high impact and low product development effort.  I think it would be helpful to share the criteria with which we assess effort at Instructure which includes complexity, tech stack, interdependencies on other systems and features, skillets, end to end quality assurance testing as well as user experience. Something that may appear low effort such as [Speedgrader] Audio or Media Comment upload failure confirmation involves multiple services that need conversion, backend changes to Canvas and background processing for “real-time waiting” which is net new functionality in addition to end-to-end quality testing. Addressing individual ideas without considering the end to end experience can result in a frankensteined product that hampers our ability to support you and your colleague’s end to end needs. Our intention is always to be thoughtful which requires understanding complexity and dependencies that may not be visible from the front end. Additionally, our commitment to quality requires that every change be tested and validated to ensure that both the change and any impacted features in that workflow continue to function as expected.
2. Definition of a bug versus feature idea / improvement: A bug is defined as an error or defect in the system that was not caught during quality assurance testing (e.g. broken link, course copy not working with our updated RCE, etc). Ideas that identify areas that aren’t working as desired and would require new workflows are net new investments that result in new features and/or systems needing to be built. I can understand how it could be interpreted that a theme such as [User Settings] Automatic clean up of discarded/inactive users seems is a bug; however, this requires net new functionality to be built which is a new idea/feature. 
3. Feedback channels and prioritization: Our team leverages multiple channels of feedback in order to drive our prioritization which include but are not limited to market and industry trends, ideas & themes, user focus groups, and customer feedback (direct and indirect). We are continuously reviewing feedback across all these lanes to understand trends and unique needs based on elements such as user role, institution level, product area, and more. This is how we develop our 12-month outcomes roadmap. @Charles_Barbour you asked how we share our internal priorities: our roadmap provides that visibility and we are openly sharing the overarching outcomes we plan to deliver. We make it clear what areas of the product we are improving, and we are consistently reviewing and selecting those “lower effort” items that we can incorporate to continuously be enhancing the user experience. 

As mentioned in our initial blog post about ideas & themes, the intention behind grouping ideas into themes allows us to better understand the areas of the product that have the most opportunity for improvement and address larger outcomes versus individual solutions so that we build a holistic end to end product. I also think it’s important to reiterate what @Renee_Carney mentioned above, which is that it is possible that when a theme is completed that some referenced ideas will be marked complete, but not all, which does allow us to deliver on smaller areas versus an entire theme.  The ideas that are not completed will be revisited to see if they map to inform another theme, or if they should be archived. Lastly, it’s important to note that our team is constantly reviewing the ideas & themes our members are submitting as part of our strategic planning process that results in our roadmap.

[brian_mullins]

@leward Agree! I've written elsewhere that I'm not a fan of only working on ideas/themes that get votes and for many of the reasons you've mentioned. 

@Charles_Barbour Also agree! I think if they stick with the Themes we need a Theme solely for the bugs that they plan to fix in the next sprint. And overall, I don't think the frequency of votes on themes is often enough. It implies that we're only going to get the fixes and changes we need a few times a year, and with as many ideas as are out there, as someone said, it will take years to get through them all. On the other hand, I can see the need from Instructure's point of view to group 'similar' ideas together and do them at the same time, but those need to be small chunks, iterated on very frequently, and very often with. 

[Charles_Barbour]

Thanks for your response @TaraGoldman . A couple of points of clarification and suggestions:

Fixing the underlying issues in the [Speedgrader] Audio or Media Comment upload failure confirmation or [SpeedGrader] media comments: warning to stay on page until comment fully uploads/processes are obviously complex fixes which might require significant time to resolve and test.

What wouldn't take long is changing the text displayed to Instructors. "Your media is being uploaded/converted. Do not leave this page or your work may be lost!" I think fixing the issue is important but until it's fixed, how many faculty are being alienated by an unreliable tool with unclear messaging? How many will never try it again?

Regarding bugs, I don't think the definition you shared matches the expectations of many of your customers. As an example: Why on earth is it acceptable for Speedgrader to not accept a non-numeric grade, but not notify the grader the grade they entered was rejected?

That's a bug. It doesn't matter if that wasn't written in the scope. Either don't let them enter a non-numeric grade in the first place or notify them of the failure as the gradebook would.

Instructors or graders should never lose work because they weren't notified that their work wasn't saved yet and might be lost. If that happens, it's a bug and should be a cardinal sin. That should be handled as a break/fix, never a feature request. Even if the only viable quick fix is increased messaging to lessen the likelihood of data loss.

If I was unclear, my apologies. I don't feel that [User Settings] Automatic clean up of discarded/inactive users is a bug. I think the other items I mentioned are bugs. This functionality would obviously be new and require significant development.

And while I understand what you're saying regarding the roadmap, there are dozens of idea with hundreds of votes. Some of them go back 5 years. Yet somehow, none of those great ideas with obvious requests have gotten time allocated to resolve them. Instead, Instructure shipped an Icon Maker (which lets users create icons which aren't accessible).

Why was creation of an Icon editor prioritized over any of the other features? I can't find any record of it being a community idea. Was it ever on the roadmap? I'm sure it fit into some broader theme, but undoubtedly so do many of the ideas which haven't been selected. I feel pretty confident most community members would have chosen another feature over that if given the chance to do so, yet here we are.

Finally, regarding the plan that ideas in a theme which are not completed will either be moved to another theme or archived seems odd. I'm not sure ideas and suggestions from your customers should be "archived". Certainly not without a clear explanation as to why. If you archive them without implementing them, I suspect you'll see lots of them re-created in a few months when someone creates the idea again.

But your response does makes me think about the issue more. It sounds like when you select a theme and work on the ideas within, at the end of development the plan is to remove the theme. While that might make sense for some transitory areas of focus, some of themes seem like they will be permanent. Unless you think that you can fix the ideas in Make grading more efficient and effective or Save Admins Time and Clicks once and then be done with those themes?

Obviously none of this is easy. You're trying to improve the situation which is great. I would just hope to see Instructure challenge more of your processes as well as their underlying logic and decisions.

I'm not seeing how this categorization and ranking inherently speeds up the pace of development or gets more ideas shipped. I'm honestly not even sure I see how it translates into better ideas being selected for development.

If a great idea with lots of votes can't be selected on its own merit, how does grouping it with other (less popular) ideas makes it more likely it will be chosen for development?

[JamesSekcienski]

I'm confused why the voting process will be based on voting for a particular theme rather than ideas. 

Please correct me if I am wrong, but based on the information provided so far, it seems like if there is an idea that I support and would like to see implemented, I will need to vote for that theme.  By voting for the theme, I will increase the chances of that theme being selected as a focus for the next development cycle. If the theme is selected, then development will begin around that theme (not necessarily the ideas associated with the theme).  Once development is done, the idea that I was really trying to show my support for (and possibly the only reason I voted for the theme) could still be left unresolved.  If the idea that I wanted to support is left unresolved, then my vote was wasted, especially if the idea is just archived at that point.

If I am voting for a theme, then the ideas associated with that theme should be implemented.  If there are too many ideas associated with a theme, then perhaps the theme should be broken into smaller themes before voting to ensure all associated ideas would be addressed.  If it something that won't be developed, I would rather know that before voting so I can vote for things that would be developed.

[Renee_Carney]

Hey @Charles_Barbour 

I see the time and passion you've invested in this discussion and I wanted to pause for a moment to thank you for your involvement in our Community and this new process.  One observation i wanted to share after reading this thread is - anytime there is change it takes time to really shake out what works and what doesn't. Change that leads to success for all parties also takes a bit of grace and assumptions of positive intent. 

I've been a part of this Community for almost 13 years now, first as a customer, then as a Community Manager, and now as the Director. I've experienced so many different iterations of the ideation process and been a part of strategizing many of them. Every iteration has had a different level of Product Team involvement. I share that with you to hopefully build a little credibility when I say - This iteration far and above surpasses any and all Product team involvement we've had in the past.  The Product Team is owning this process from the moment of idea moderation to the final release notes, and that will only benefit you and anyone else engaged in it. 

Over the past two months I've observed groups of Product Team members [including the Chief Product Officer] gather to read every single idea in the Community, debate what's within the realm of possibility, verbally express how much they love an idea and appreciate the customers that submit them, and develop themes to help bucket them and better collect your input. Every working session was filled with excitement and desire to better hear your (the customer's) voice, and better communicate when development is and is not possible - because in most cases a quick 'no, we're not developing this and here's why' is just as important as 'yes, we're going to scope this'. 

So I ask, please, stay engaged, stay passionate, and stay kind.  We're all in this together and we're all seeking the same positive outcomes; our visions just may differ at times on the necessary path(s) to get there. 

[Renee_Carney]

Hi @JamesSekcienski 

From your comment I can see that you recognize just how complex it can be to implement an ideation process that truly informs Product development. For a long time this Community has only focused on ideas as individuals, and it often gave the perception that Instructure had gamified their Product Roadmap. To some degree, the total votes of an individual idea would get it a bit more attention, but it never guaranteed development.

Instead, most of the requests from The Product Team to the Community were for ideas related to problems they were solving or features they were improving - they were requests around themes. So this new process was set up to be much more honest and transparent about this approach, better identify the points at which you (the customer) could engage in that process, and help reduce the perception that individual idea votes equaled development. 

When it comes to ideas that aren't developed as part of the final release, I relate this to a research paper - great researchers take in as many resources with differing perspectives, opinions, and thoughts, but not all make the cut on the final paper. The difference in this process is that our Product Team is committed to communicating when and why an idea isn't developed, and if it will be considered in the future. 

Please do reach out if you have further questions and we look forward to engaging with you in this new process

[james_whalley]

This process now means I will almost never recommend to faculty that they engage with the ideation process. The idea of telling faculty they should put in an idea, wait for it to be organized into a theme, wait for the voting window to open, and then to vote on a theme that includes their idea is convoluted and unrealistic. We wanted a clearer idea of how decisions are made and how we could actually get popular ideas into production. Instead we have been gifted a complicated system that provides more communication and less actual clarity.

[nwilson7]

First, thank you for taking the time to rethink the idea process.  I have always felt like it was a double edged sword, where it was great anyone could create ideas (and feel heard) and have them potentially developed but that did mean there were the thousands of ideas and even the best ones could get lost in there.  I know some faculty felt that this became the land of forgotten children as they would run into an issue, look there for an idea and find it has been created 5+ years ago with no movement.

The themes path initially sounds like a great idea.  If you are going in and editing code, why not address multiple requests at the same time to save the developers from going back and forth.  I also understand the complexity of what many people think is an easy change and the ripple effects of that change in other systems.  That said, I am confused on how to "vote" for things I would like to see developed in this new format.  Based on the response to @kmallen it sounds like voting for a theme does not always mean the ideas under it will go anywhere.  I also worry that voting for a theme could wrongfully indicate support for all the ideas contained within as @JamesSekcienski mentions.  There are ideas that personally I would NOT like developed as I think most users prefer it the way it works currently.  If those are contained in a theme that has ideas I do want developed, how do we indicate support for just the ones I support?

I know there is no way to make everyone happy with an LMS, especially people that have experience with other LMS's as usage of the other may have created habits that now they want to see work the same way in Canvas.

-Nick

[Renee_Carney]

@james_whalley 

I can see your very valid concerns. I worked with and mentored faculty for a few years and I know just how valuable their time is. My opinion is that this new process will provide more clarity for when (scheduled voting windows) they should focus their time, and where (themes that contain solutions they are seeking). Once they share an idea they won't need to keep checking back, they will receive notifications along the way to let them know where things are in the process. 

[Renee_Carney]

Hey @nwilson7 

Great feedback - Thank you! 
For themes, there will be two voting windows per year.  You can find the schedule in this guide.

Once voting has ended and the top themes are 'Prioritized' is when the discussions will begin regarding what solutions make the most sense, and when you'd have the opportunity to debate individual ideas that are referenced on that theme.  

[JamesWhalley]

@Renee_Carney I think @nwilson7 makes a really good point above. You're asking folks to vote for a theme (rather than a specific idea they support) that may include ideas they very much do not support. You mention that the new process provides instructors a when (most likely not when they are already here engaging in the community by putting in their initial idea or finding a similar idea that already exists) and a where (a theme that contains the solutions they are seeking, but provides no promise that selection of the theme means their specific issue will be dealt with or that they might end up getting other features changed that they really don't want changed).

For instance, let's look at Improved UI Controls and Usability - Added or clearer alerts and confirmations and notifications. For the sake of this argument, let's say I really support including a "quiz saved at" notification on New Quizzes for students, but I absolutely do not support adding an indicator in grades cells in the Gradebook to show when there is a comment on a submission. Well, now it is time to vote and I vote for this theme because it has something I support, but it also has something I do not support, so I cross my fingers that my feature is the one they work on and not the other one that I really do not want.

We can make an even clearer example using food. My kids ask what's for dinner. I say, "We can have pizza or kale and mutton." Well, my kids love pizza and hate kale. But, if I tell my kids that choosing pizza means they might get pepperoni pizza or they might get anchovy pizza or they might get both, it might make them reluctant to select pizza for dinner.

[Renee_Carney]

@JamesWhalley 
I love the pizza analogy - pizza is the best!  
I think the one element missing from the analogy is the discussion.  The themes allow users to say they'd rather time be invested in making pizza than kale (although I'm a big fan of mutton too), or to put it in Canvas terms, quizzes work would be more valuable to them than discussions work in this season.  Then, once the pizza is prioritized the discussions open up to evaluate, expand, or narrow down the suggestions (referenced ideas) to the best pizza (pepperoni+pineapple, of course). Ultimately, you as the parent make the decision on the final toppings based on your elevated knowledge and perspective of how everything works together. 

[marpayne]

When grading short answer or essay questions in Quizzes, I have to post the score at the top, but, then click 'Update' at the bottom. Please either put the score box at the bottom or put the 'Update' button at the top. Put the fudge point area wherever the other two are.

[Renee_Carney]

@marpayne, it sounds like you'll want to subscribe to the following theme - there is a related idea that is referenced as informing it. 
Make grading more efficient and effective - Instructure Community   

[kmallen]

@Renee_Carney 

Thanks for pointing us to the schedule in this guide. It was helpful for me to do a deeper read of that doc now that this conversation is going. I'm still trying to wrap my head around what happens once the themes are selected. Here's what the doc says:

Prioritized

Prioritized themes have gone through the voting process and have been selected by Instructure to be delivered within the next six months. As Instructure works through the development process for this theme, Instructure Product Teams will provide updates and engage the community for feedback on the potential solutions. When we deliver on prioritized themes, our focus is on solving the problem with the best solution we have identified. Due to this, the end solution and deliverable may be different from solutions or specific asks in the original ideas. If the problem of an idea mapped to a prioritized theme is not solved in our end solution, we will re-map the idea to a different theme (existing, new or extension of the current theme if the scope was deemed too big). The theme would go back through the process. 

It is important to note that some themes will extend past six months of development depending on the size and complexity of the theme. 

I'm muddy on what that engagement looks like and how your team will decide what moves forward. From your comments on this thread, it sounds like a discussion. How will the outcome be determined from the discussion? As @JamesWhalley and @nwilson7 pointed out, community members have different ideas about what we want, so it seems unlikely that there will be a consensus. Will your team perform a qualitative analysis on the comments to see which ideas have the most support (e.g., lots of folks chime in and surprise us by wanting sausage and mushroom pizza)? Or will there be more voting on the ideas under the theme?

I think we're (or at least I'm) trying to figure out what "engage the community for feedback on potential solutions" will look like.

[Charles_Barbour]

@Renee_Carney Is this a fair interpretation of the process @marpayne should expect?

1. They follow the theme which contains an already existing idea. In this case the theme is Make grading more efficient and effective and the idea is [New Quizzes] Move the Update Scores button for quizzes.
2. Voting should open this month so Instructure will select some themes upon which we get to vote. (How many themes might get chosen for voting?)
3. marpayne and others get to vote and comment for a month.
4. Based (in part) on the results of the voting, Instructure will prioritize some of the themes.
   1. If Make grading more efficient and effective is not prioritized, no work will happen on the idea until (at least) the next round of voting.
   2. If Make grading more efficient and effective is one of the themes which are prioritized, Instructure will then select ideas from all the themes which were selected. These decisions may be based on comments and discussions on the ideas.
      1. If [New Quizzes] Move the Update Scores button for quizzes is not selected, it may either be moved to another theme or marked as being rejected. (I understand this is aspect not fully fleshed out yet.)
      2. If [New Quizzes] Move the Update Scores button for quizzes is chosen to be developed, it will be worked on and should be released in the next 6 months (with the goal being to have it released before Spring of 2024).

Is that accurate?

If so, I believe there are 5 possible outcomes.

• The theme is selected but the idea is rejected and shelved.
• The theme is not selected for voting by Instructure. It may be eligible for voting again in October.
• The theme is selected for voting by Instructure but does not get prioritized. It may be eligible for voting again in October.
• The theme is selected for voting and prioritized by Instructure, but the idea is not chosen for development. It gets re-categorized in a new theme. Its new theme may be selected for voting again in October.
• If the theme and idea are both selected, the idea gets developed and shipped in 6-9 months.

The idea was created in 2017 but was discussed as early as 2015 (possibly earlier) and QuizWiz solved this (and other issues) in 2016.

So in the best of the 5 possible scenarios, a fairly obvious usability issue which has existed for 8 years may finally be addressed.

In 3 of the other scenarios the idea might still get resolved but the timeline is pushed back 6 months.

The idea may also be rejected entirely.

Regarding the selection of ideas from themes, will they be announced so people will know what is being worked on for the next 6 months? Will all ideas be selected up front or will ideas be chosen throughout the development cycle?

I hope you can understand that from the customer perspective (at least mine), while this new process provides some much needed clarity and visibility it doesn't exactly inspire hope that ideas are going to get developed. The process helps show focus but doesn't appear to show if (or how) Instructure plans to increase the pace of development.

Hopefully I'm wrong on that and you can start pumping these ideas out. You've certainly got enough of them to choose from!

[hesspe]

I see a basic tension here.  Canvas developers (understandably) are looking for an integrated set of priorities that can be "projectified."  User interest (as reflected in the comments here) centers on  much more granular concerns.  I KNOW improvements of a granular nature constitute the desires, requests, and gripes I hear for the users I deal with on a day-to-day basis.  If ever I hear from them about a "theme" it's usually articulated along the lines of "the attendance tool stinks," which, although true, is not I think what Canvas developers are looking for.  (Sorry if that seems "unkind.")

[Renee_Carney]

@Charles_Barbour 

You have a lot of great questions in here, and I'm going to try and catch them all.  

• How many themes might get chosen for voting? We don't have a concrete answer on that yet - each round could vary.  It will depend on the potential size of themes and available resources.
• The possibilities you lined out on what could happen to an idea and/or theme throughout the process read correct to me. 
• Regarding the selection of ideas from themes, will they be announced so people will know what is being worked on for the next 6 months? Prioritized themes will have discussions where solutions are shared and discussed. 
• Will all ideas be selected up front or will ideas be chosen throughout the development cycle? The goal is that all ideas are aligned up front, but the truth of the process is it is very manual and human centric. It is possible ideas could be added as discovery occurs and discussions develop. 

The other thing I want to be sure and address is the focus on "what happens to my individual idea".  This focus has been the central driving force for all ideation processes in our past and I think we've proven it's not productive. Having a pool of 5,000 ideas with 20-50 more rolling in each week is like trying to dig a hole in the surf. 

[TaraGoldman]

Great question @kmallen. I know you asked Renee, but I'm going to go ahead and take this one. 

Once prioritized themes are selected, we will convert those themes into a discussion forum. The forum will allow for threaded discussion around all topics and referenced ideas of that theme. The Product team will also be asking questions within the thread and connecting with select representative community members for discovery validation as needed. We'll be taking into account different qualitative and quantitative data along the way and communicating our assumptions and solutions within the theme discussion.

[JamesSekcienski]

I understand that trying to developing a process for managing ideas is complex and you are trying to improve the communication with ideas that users submit.  You are also now having to manage and balance the needs for a lot more users and support more of the needs for K-12 due to the boom in online/blended education with the pandemic.  While I don't agree that this process is the best approach, I also understand that a lot of time has already been invested in this change and we should give it a try since the most recent process was not working.

I agree with @nwilson7's  comparison to the most recent process before this being like the land of forgotten toys.  There have been a lot of great ideas submitted with a lot of conversation and yet they have felt ignored.  Some users have even built their own user scripts or theme customizations to address the issue and they have gained popularity among users, but still they felt ignored.  Even when some ideas started to be archived, many of those years old ideas still remained active which made it seem like they were still being considered, but still no information from Canvas about them.  While it was stated that individual ideas have been the central driving force in the past, it hasn't felt that way based on this.

Here are some thoughts and hopes for this new process:

• The themes seem like a great idea to group related ideas.
  • This helps to make it easier to find and review ideas that are related.
  • This also makes sense when planning development to try to consider similar ideas to evaluate their impact on the system and there may be overlap in changes that could be done at one time.
  • While some themes seem too broad and will clearly take longer than 6 months to complete, it is nice to see a bit more organization to the ideas.
• Themes may be the focus, but I hope small ideas are still considered for quicker development
  • In the Canvas Release Notes there are often minor fixes/improvements that get released.
  • If a Product Manager when reviewing ideas notices an idea that could be a quick win, hopefully they would consider seeing if it could be developed sooner in an upcoming release.
  • One of the things that is currently frustrating is sometimes when reporting an issue to Canvas Support, the conversation ends with being told it needs to be submitted as an Idea and that the issue is "expected" behavior and not a bug.  This would be even more frustrating now as individual ideas are not considered an only the overarching theme is considered when planning development.
  • For small improvements to be left in limbo for years simply because its theme isn't voted up seems like a waste of time to go through the process of submitting minor ideas.
• Improved and open communication is very much appreciated
  • Recognizing that communication needs be improved around ideas and incorporating it into the plan is greatly appreciated.  I do hope to see more engagement from Canvas/Instructure employees in the themes and ideas.
  • Keeping the communication open helps us understand more about the decision making process and also helps us to better understand where ideas stand and why some aren't being developed.
  • However, I am curious what is going to happen with ideas in themes that aren't prioritized. While there be at least periodic updates or comments from employees about those ideas that have been waiting for a long time or will they linger in a void until they are part of a prioritized theme?
  • Also, is feedback going to be provided on why certain themes aren't included in a vote?  I noticed that not all themes will necessarily be included when voting is open.
• Voting and commenting
  • Has the ability to comment on individual ideas been taken away with this new process? I've tried looking at a few ideas and notice that there doesn't appear to be a comment option.  This seems like a lost opportunity for the community to share their voice, especially outside of the limited voting windows. I hope this is just temporarily disabled during the transition, but I'm not sure after reading the updated documentation more.
  • In addition, is there no opportunity to comment on themes before the themes have been prioritized?  Discussion among users should occur before voting.  This would be part of making an informed decision when voting for a theme.
  • How much do the community votes influence the themes that are prioritized? Even though there is a voting period and even though only themes that are deemed potentially possible at that time are included in the vote, there is still no guarantee that the most voted for theme would be included as a prioritized theme since it says there are other factors considered too.

[Charles_Barbour]

@JamesSekcienski's makes several great points but the one about Canvas Support is especially noteworthy. Hopefully they are being included in these discussions.

I would also like to see some acknowledgement that small, easy fixes (like changing working or improving error messaging) shouldn't be required to run through the same process as all other ideas and enhancements.

I'm sure typos don't need to be submitted as ideas. Improved messaging should follow a similar resolution path. Especially if the underlying issue may not be resolved for 6 months or more.

[Charles_Barbour]

@Renee_Carneythanks for your response!

I have a follow up WRT this part of your response: "The other thing I want to be sure and address is the focus on "what happens to my individual idea". This focus has been the central driving force for all ideation processes in our past and I think we've proven it's not productive. Having a pool of 5,000 ideas with 20-50 more rolling in each week is like trying to dig a hole in the surf."

So what's the solution here? I assume some of the outstanding ideas (as in not implemented as opposed to terrific) are being closed and archived or merged with other similar ideas. Are you also trying to reduce the number of incoming ideas? If so, how do you plan to accomplish that?

[james_whalley]

@Renee_Carney Up above you mentioned:

"Once voting has ended and the top themes are 'Prioritized' is when the discussions will begin regarding what solutions make the most sense, and when you'd have the opportunity to debate individual ideas that are referenced on that theme."

In the latest Daily Digest, it says:

"Will all referenced ideas be addressed in a Prioritized Theme?

No, not all ideas will be addressed. What we are committing to next for selected themes is that our teams will conduct the necessary discovery to determine the best solutions to solve for the value outlined in the theme. As we further assess and define those solutions, we will keep the Community informed on any ideas that may no longer work into that solution."

Does this mean the community will not be involved in the discussions of which ideas to focus on? I don't see discussions mentioned at all.

[Charles_Barbour]

@james_whalley you are correct. Instructure will select which ideas are selected from the themes.

I suspect in the larger themes (some of which have 20 - 40 ideas) it's doubtful that most of the ideas will be selected. Many of them will probably still remain at the conclusion.

Some themes which have only a few ideas with less specificity might have only 1 or 2 ideas worked on, but that might represent a greater % age of the ideas in the theme.

I suspect part of the challenge is for them to figure out how the pieces fit together.

It's like a home-remodeling project:

I want to redo some stuff in the bathroom, but I also want to refinish my wood floors. Since I'm pulling out the tile in the bathroom, do I also want to update some of the electrical in the bathroom? It shares a wall with the kitchen, so I can I add some outlets there too? I don't want to refinish the wood floors until we're done with everything else so they don't get damaged so maybe that doesn't make sense now. Instead, since I'm doing some wiring in the kitchen already, maybe I can add some can lights in there? Maybe also tack on repainting the dining room since that's really only a few hours of work?
(If that all sounds oddly specific, that's because I just went through it a few months ago.)

[james_whalley]

@Charles_Barbour Thanks for the example, it is very clear to me. It does not help the frustration I have at the fact that we were promised we could be part of the process via discussion of what ideas to focus on. Some of these themes have ideas in them that I would want, but also have ideas I am less fond of. A discussion process would help Instructure to understand why people voted for these themes and what from these themes really matter to them. 

Maybe I voted to redo the bathroom because the toilet is just awful. Instructure might see that there were also ideas to redo the flooring with carpeting. Well, if I and most the people who voted for the theme really want the toilet fixed and think that is the priority, and most of us don't want carpeting, we will now not get our way and will be less happy with the bathroom than before.

Instructure can't take a vote for a theme as a blanket support of every idea in that theme. That just doesn't make sense. @Renee_Carney clearly stated above that, "Once voting has ended and the top themes are 'Prioritized' is when the discussions will begin regarding what solutions make the most sense, and when you'd have the opportunity to debate individual ideas that are referenced on that theme." What's happened to this promise?

[james_whalley]

@Renee_Carney I'm still hoping for an answer about what's going on as far as discussing the ideas under the prioritizes theme(s). You very clearly laid out that there would be discussion and debate over individual ideas.

@TaraGoldman Can you verify if the plan is still:

"Once prioritized themes are selected, we will convert those themes into a discussion forum. The forum will allow for threaded discussion around all topics and referenced ideas of that theme. The Product team will also be asking questions within the thread and connecting with select representative community members for discovery validation as needed. We'll be taking into account different qualitative and quantitative data along the way and communicating our assumptions and solutions within the theme discussion."

[Renee_Carney]

@james_whalley 

My apologies - I started a response on Friday and did not complete it. I appreciate you calling my attention to your question, again.

The quotes you referenced still hold true and the most recent blog outlines what you can expect to happen next. There will be discussion about ideas, which will potentially include the product team sharing ideas that will not work in a solution. 

Our number one goal in this process is to focus and magnify the customer voice in our prioritization process. We appreciate your engagement and understanding as we take the first round of voting for a spin.

[james_whalley]

@Renee_Carney I appreciate you getting back to me and I am excited that discussion will be part of the process.

I took a look at the most recent blog and still don't feel like it clearly communicates that decisions about ideas will be rooted in discussion. In fact, the language implies the opposite.

"[O]ur teams will conduct the necessary discovery to determine the best solutions to solve for the value outlined in the theme."

"[W]e will keep the Community informed on any ideas that may no longer work."

I'm crossing my fingers that everything works out and that the process truly captures what the majority of the community values.

[cvalle]

I need clarification on how to actually vote for themes. When I click on “Themes Open for Voting” the page is empty. When I look at “Identified Themes” none of them seem to have a button for voting.

Thanks

[KristinL]

Hi @cvalle - The themes that were open for voting have been moved to the stage Post Voting Review. The voting window was open between April 3 and April 28, 2023. I'm sorry you missed the opportunity to vote, but when the identified themes move into Prioritized, we hope you're able to join the conversation!

[cvalle]

Thanks @KristinL. Is there a regular schedule for when voting is open? If not do you have any estimate for when the next voting window will be?

Thanks again.

[KristinL]

Hi @cvalle - Right now, there isn't a specific date publicly available, but we can share that there will be another window sometime in Q4. Tara included this timeline in her latest blog as well.