Last November we posted a proposal for Canvas release changes in 2019. Our proposal intends to improve the quality of Canvas releases, resolve bugs more quickly, and create more predictable releases with a more manageable pace. Community feedback proved highly supportive, and our engineering teams have continued to make progress with necessary adjustments. Thanks for your patience as we’ve been working out the specifics to share with you.
- Release and Deploy Schedule
- Schedule Adjustments
- Timeline Implementation
- Release Strategies
- Production Releases
- Beta Releases
- Beta Release Terminology
- Release Notes
- Deploy Strategies
- Production Deploys
- Canary Deploys
- Bug Fixes
- Beta Deploys
- Deploy Notes
- Previous Comment Concerns
Release and Deploy Schedule
Since our initial proposal summary, our engineering teams have explored necessary adjustments for the feature release schedule, non-feature code deploy schedule, and the treatment of Canvas feature options.
To review terminology:
- A deploy means making code available in a specific environment; code may or may not be visible, but it is code that does not change customer workflows.
- A release represents a scheduled event when customer-affecting code is intended to be visible.
Schedule Adjustments
The original proposal suggested that releases be moved from every three weeks to every month. The proposal also suggested that deploys would take place every two weeks.
After thorough research among all engineering teams, releases and deploys has been adjusted to the following dates:
- Feature releases in the production environment take place on the third Saturday of every month (originally noted as the first Saturday of every month).
- Deploys of the main Canvas codebase to the production environment take place on Wednesdays, and beta deploys take place on Thursdays (originally shown in the calendar example with production on Saturdays and beta on Mondays).
- Deploys of supporting Canvas services are and will continue to be deployed frequently to both the beta and production environment. This behavior is not a change from current practice.
Timeline Implementation
The current date timeline for implementing these changes is as follows:
- July 13: Last regular Canvas production release with deploy/release on the same day
- July 15: New feature changes released to beta environment as August preview
- July 22: August release notes published
- July 31: First weekday production deploy (recurring every two weeks following)
- August 1: First weekday beta deploy (recurring every two weeks following)
- August 17: First monthly production release with new process (recurring every third Saturday of the month)
- August 17: New feature changes released to beta environment as September preview (recurring monthly on the same day as the production release)
- August 26: September release notes published
- September 21: September production release date
The process would then continue according to the established cadence.
The image representation below displays a visual of the deploys and releases for August and September.
Caveats:
- The beta environment data refresh and test environment data refresh will continue to take place every Saturday and with every production release, respectively, although not represented in the image.
- The Upcoming Canvas Changes page may display an enforcement date that does not align with a feature release.
Release Strategies
Production Releases
Feature releases will be managed by a feature flag enabled by engineers. The feature flag contains all previously deployed customer-affecting code that is not made visible in the production environment until the feature flag is enabled. This functionality is like a feature option, except that all customer-affecting changes are made available to all customers at the same time.
Feature flags also protect the production environment as the feature flag can quickly be disabled by Canvas engineers if needed.
Beta Releases
On the same day as the current month’s production release, a separate feature flag will also be enabled within the beta environment containing all customer-affecting code that will be released in the following month. This change ensures that you have a full month to experience all upcoming changes in the beta environment and will include all Canvas services, such as Quizzes.Next.
The beta environment data refresh will still take place refreshed every Saturday, which means the updates available in the beta preview will not be available until the refresh is complete.
Beta Release Terminology
To better define the intention of a beta release, terminology is being renamed as a monthly preview.
This change will also align with adjustments to Feature Option terminology, which is being renamed as Feature Previews. More information about Feature Previews will be made available shortly in a separate blog post.
Release Notes
As with our current process, the first week of the beta release is still considered unstable and subject to change as engineers continue to verify the newly available code within the beta environment. Therefore, feature release notes will not be available until the following week. As our new processes become more reliable, release notes may also be able to adjust appropriately in the future.
Deploy Strategies
Non-feature deploys contains code changes that are intended to fix bugs, improve performance, and prepare for new features. We know through experience that smaller, more frequent deploys leads to a higher quality product. And when there are problems, our teams can fix them faster.
Production Deploys
Our overall deploy process has matured enough to move the primary Canvas deploys to a weekday. Deploys of supporting Canvas services are and will continue to be deployed even more frequently. To ensure that deploys are non-disruptive to customers, our teams have been planning and are already implementing several new safety precautions, including tightening our processes on how we characterize a change in Canvas (e.g. feature vs bug).
Canary Deploys
In the current release process, deploys and releases take place at the same time. Code is incrementally deployed around the world by region, and the first regions have always been the first to experience and report unintended behaviors.
The new process will slightly modify this behavior. Deploying code to a small group of end users is called a canary—inspired from historical reference of placing canaries in coal mines as a warning system. Canary deployments slowly distribute the changes to this small subset of users before they are made available to the entire infrastructure, and its impact is usually minimal. Code can be reversed or corrected before resuming deployment.
The scheduled production deploy takes place every other Wednesday, but the Canary deployment begins a day earlier on Tuesday under close monitoring. Deployment only proceeds if no problems are detected.
All customer accounts will go through a rotation as the canary for the release. Being part of the canary deployment should occur infrequently. Customers will not be notified when they are placed in the Canary deployment. Any unintended behaviors noted by an institution should be treated as part of that institution’s support protocol.
Bug Fixes
Only changes characterized as a bug fix can be deployed to the production environment at any time. Our tightened processes ensure that bug fixes are only restoring intended behavior. If a bug needs to be addressed by adjusting functionality or the user interface, the bug will be changed to a feature-type categorization and placed behind the monthly release feature flag for future visibility.
Beta Deploys
Deploys of the main Canvas codebase to the beta environment take place the day after a production deploy. Deploying to the beta environment allows our engineers to confirm the newly deployed code is not causing unintended behaviors and ensures the production deploy will be as accurate as possible. Engineers may continue adjusting code in the beta environment up to the production release.
Deploy Notes
Because engineering is creating smaller teams with quicker releases, up-to-date deploy notes would become difficult for our teams to maintain—as well as customers to reference. Notable entries for the main Canvas deploy will be available as deploy notes on the same day as the production deploys. Additionally, customers with a greater interest in the full deploy can access our open-sourced canvas-lms Github repository deploy list.
When the new production deploy process goes into effect beginning July 31, deploy notes will still be created manually and added as part of the Canvas Release Notes page in the Canvas Community. However, our goal is to make deploy notes more automated while ensuring that notable bug fixes, for instance, are properly highlighted. Automating the notes may affect the future location of the deploy notes. Discussions about automation are still ongoing, and more information will be posted as we have additional updates.
Previous Comment Concerns
We had several topics weaving through the comments we received in our first blog post. This second post expands on many of the received questions, but we want to ensure all underlying questions have been addressed. Presented in a question-and-answer-type format, the remaining content summarize the topics that concerned you:
Will this change place more focus on fixing issues discovered in the beta environment?
Our teams are redefining the process for how features are being deployed; some that are designed as feature options will undergo beta testing with a small group of customers before being available to all customers.
Will this change place more focus on fixing bugs?
Quicker deploys will allow our teams to be able to fix bugs more timely. Bugs are prioritized as part of each planning meeting depending on severity. We obviously do not want to release features or changes prematurely and we anticipate that this new process will also help us improve our quality in our feature deliveries.
Will some features still deploy straight to production?
All features will eventually be available in the beta environment, although their use may include some caveats.
- Canvas Catalog introduced beta environment support as part of the Catalog Release Notes (2019-04-22) and adjusting the release notes schedule to allow for future notice.
- Quizzes.Next is now available in the Canvas beta environment as included in the Canvas Release Notes (2019-05-11).
Other services are currently planned and will be announced at a later date.
Will this change increase development of Community feature ideas?
Engineering's goal with this release change is to improve product quality and be more efficient, which may come back as a benefit to allow them to work on more ideas! The feature process itself won't change, and our product team always considers ideas against existing priorities and timelines. Stay engaged in the feature idea process so our product team is aware of the workflows and functionality you’d like to see incorporated into Canvas.
Can same-day deploy notes be posted in advance?
As noted, we’re tightening our process on how we characterize a change in Canvas so that deploy notes are not a concern. Official deploy notes will most commonly include notable fixed bugs, considering the majority of Canvas code is deployed for future functionality and infrastructure improvements. If there is any question as to whether a change will affect existing functionality, that change will be changed to a feature-type categorization to be available as part of an upcoming monthly release. And if a code change slips through the cracks, our teams always review the prior release to identify what needs to be improved to prevent unintended future situations from occurring.
_____
If you’ll be joining us at InstructureCon, we’ll be hosting a session that will review these upcoming changes as well.
As always, thank you for your interest in upcoming Canvas changes. Let us know if you have further questions that we can address.
The content in this blog is over six months old, and the comments are closed. For the most recent product updates and discussions, you're encouraged to explore newer posts from Instructure's Product Managers.