Canvas Data 2 updates

[Edina_Tipter]

It is time to share progress on the Canvas Data 2 product development.

First and foremost, I am happy to announce that we have achieved an important milestone -  the first version of the Canvas Data 2 was released to a closed alpha customer group with 13 participants from one region on Monday, June 13. This means that these 13 customers can query their institutional data via our new data platform and start kicking the tires and giving feedback.

Choosing the participants wasn’t easy for this initial release, while product development is still on-going, we needed customers with vast data experience and prior CD1 knowledge. Furthermore, we are limited for now to one region (multi region functionality will be rolled out incrementally) and autoscaling will be a focus in the coming months. Last but not least, availability for quick feedback and interviews was also something we requested from the alpha group.

While the closed alpha program is running, the product team continues to work towards the open beta to onboard additional customers. We are looking for feedback during the alpha period to steer when we're ready for more institutions to use CD2. Our target estimate for opening CD2 to beta participants is around the end of October 2022, stay tuned for updates. If you have interest please talk to your CSMs. 

If everything goes as planned, we are looking to release the product in GA (General Availability) at the end of this year. To achieve this, we are working on the scalability, maintainability, monitoring and improving the quality of our data pipeline.

Since the Canvas Data 2 will replace Canvas Data 1, there will be a maximum of 12 months given to customers to transition before we sunset CD1 (timer starts once GA is live). To support users in transitioning to Canvas Data 1, some direction and  documentation will be provided.  You can also share your thoughts on what information would be most helpful while transitioning to CD2 in the thread below this blog post.

Finally, one might wonder why it takes this much time to build this product. Well, besides the very old truth that good things take time 😉 I just want to repeat that the Canvas Data 2 is not simply an improved version of the CD1 but a completely new product which enables customers to get their institutional LMS data more efficiently. Canvas Data 2 is not an analytics or reporting tool, but it is built to share high fidelity source data to power schools' analytics and custom reporting initiatives: two stand-out features being the lower data latency (fresher data) and incremental queries besides the table snapshots. 

It is also worth noting that Canvas Data 2 as a product doesn’t provide one with a custom data request tooling. In other words, there is no reporting engine on top of the data to produce custom data extracts. Canvas Data 2 has a defined relational schema as opposed to the Canvas Data 1 star schema which dictates what is available in each file. 

Customers will be able to pull a single file (e.g. users, assignments, courses) and store it locally on their server to use in an excel sheet or locally run database like postgres. We are also planning to extend the datasets incrementally and following the General Availability, to encompass the entire Instructure Learning Platform, including New Quizzes, Catalog, Studio etc. In the initial release, Canvas data 2 will be accessible via API (no UI planned).

In my next blog post, I will share a comparison table between CD1 and CD2 to help grasp the major differences between the two products and some more detailed descriptions about stand-out features. However, the API specification draft (subject to change) is available for all to review, so feel free to inspect it and share your thoughts below this blog post. As we consider this document being the base of product development and source of truth, the earlier google doc with the CD2 schema will be removed in a few days and this specification will replace it. You can find the schema under the Tables menu or if you download the OpenAPI specification, you'll need to scroll down and find the schema in JSON.