Canvas Data 2 updates

This blog from the Instructure Product Team is no longer considered current. While the resource still provides value to the product development timeline, it is available only as a historical reference.

Edina_Tipter
Instructure Alumni
Instructure Alumni
13
10428

3.png

 

It is time to share progress on the Canvas Data 2 product development.

First and foremost, I am happy to announce that we have achieved an important milestone -  the first version of the Canvas Data 2 was released to a closed alpha customer group with 13 participants from one region on Monday, June 13. This means that these 13 customers can query their institutional data via our new data platform and start kicking the tires and giving feedback.

Choosing the participants wasn’t easy for this initial release, while product development is still on-going, we needed customers with vast data experience and prior CD1 knowledge. Furthermore, we are limited for now to one region (multi region functionality will be rolled out incrementally) and autoscaling will be a focus in the coming months. Last but not least, availability for quick feedback and interviews was also something we requested from the alpha group.

While the closed alpha program is running, the product team continues to work towards the open beta to onboard additional customers. We are looking for feedback during the alpha period to steer when we're ready for more institutions to use CD2. Our target estimate for opening CD2 to beta participants is around the end of October 2022, stay tuned for updates. If you have interest please talk to your CSMs. 

If everything goes as planned, we are looking to release the product in GA (General Availability) at the end of this year. To achieve this, we are working on the scalability, maintainability, monitoring and improving the quality of our data pipeline.

Since the Canvas Data 2 will replace Canvas Data 1, there will be a maximum of 12 months given to customers to transition before we sunset CD1 (timer starts once GA is live). To support users in transitioning to Canvas Data 1, some direction and  documentation will be provided.  You can also share your thoughts on what information would be most helpful while transitioning to CD2 in the thread below this blog post.

Finally, one might wonder why it takes this much time to build this product. Well, besides the very old truth that good things take time 😉 I just want to repeat that the Canvas Data 2 is not simply an improved version of the CD1 but a completely new product which enables customers to get their institutional LMS data more efficiently. Canvas Data 2 is not an analytics or reporting tool, but it is built to share high fidelity source data to power schools' analytics and custom reporting initiatives: two stand-out features being the lower data latency (fresher data) and incremental queries besides the table snapshots. 

It is also worth noting that Canvas Data 2 as a product doesn’t provide one with a custom data request tooling. In other words, there is no reporting engine on top of the data to produce custom data extracts. Canvas Data 2 has a defined relational schema as opposed to the Canvas Data 1 star schema which dictates what is available in each file. 

Customers will be able to pull a single file (e.g. users, assignments, courses) and store it locally on their server to use in an excel sheet or locally run database like postgres. We are also planning to extend the datasets incrementally and following the General Availability, to encompass the entire Instructure Learning Platform, including New Quizzes, Catalog, Studio etc. In the initial release, Canvas data 2 will be accessible via API (no UI planned).

In my next blog post, I will share a comparison table between CD1 and CD2 to help grasp the major differences between the two products and some more detailed descriptions about stand-out features. However, the API specification draft (subject to change) is available for all to review, so feel free to inspect it and share your thoughts below this blog post. As we consider this document being the base of product development and source of truth, the earlier google doc with the CD2 schema will be removed in a few days and this specification will replace it. You can find the schema under the Tables menu or if you download the OpenAPI specification, you'll need to scroll down and find the schema in JSON.

Tags (3)

This blog from the Instructure Product Team is no longer considered current. While the resource still provides value to the product development timeline, it is available only as a historical reference.

13 Comments
Jeff_F
Community Champion

@Edina_Tipter 

To date I've referenced the CD1 schema several hundred times and found the page to be beyond invaluable.  https://portal.inshosteddata.com/docs 

Noting the post above states the current Google Doc for CD2 will go away and that this information will appear with the 'Schema types menu' on the api specification draft, I wish to point out that I cannot find the schema where it is noted.  I need a url that I can bookmark that takes me to the sheet/ page/ etc. that lists the data tables and fields. Thanks.

Edina_Tipter
Instructure Alumni
Instructure Alumni

Hi @Jeff_F , Good point as we have relabelled the menu 😕 and it is under 'Tables' of this document.

Please note that it's an OpenAPI specification, so if you download it, you'll have the JSON schema included (you'll need to scroll down a bit).

Does this help?

Jeff_F
Community Champion

@Edina_Tipter  - thanks for clarifying and I really like the Redocly interface. The search feature will be useful and I can bookmark the first named anchor.

https://data-access-platform-api.s3.eu-central-1.amazonaws.com/index.html#tag/access_tokens 

fdavenport
Community Member

We are working on a new data warehouse and analysis project in MS Azure using Snowflake.  We have discussed pulling in Canvas data as part of this campus wide project.  But now I'm wondering if we need to wait.  I don't want to redo work that we just finished.  I'm not familiar with the differences between CD1 and CD2. 

When you say CD2 is only API based and not in a star schema... I hope it will provide the tools for automating the update of institutional data models from CD2.  

Any advice?  

Jeff_F
Community Champion

@fdavenport  - if I were faced with that decision I would likely opt to identify the top 3-4 dashboards that faculty/ administrators/ advisors needed as a priority and I would set up the ETA to only bring down the tables/ fields into the DW necessary to support these dashboards. This way the dashboards could be used in Fall 2022 to support teaching/learning, academic assessment, and advising (i.e. an early warning report for at risk students).

Then of course schedule time to explore and build out CD2 during SPR 2023.

a1222679
Community Contributor

@Edina_Tipter are there any updates on time lines or any announcements to come?

jason_hill
Community Contributor

@Edina_Tipter Is there a more recent update to status of Canvas Data 2.0?

@mloble 

hmcneill
Community Participant

I am about to start looking at using Canvas Data for our institution so an update on the time line for Canvas Data 2 would be helpful @Edina_Tipter 

hansardc
Community Participant

Hi, @Edina_Tipter .

We are also looking for an update and hoping for good news! Are the timelines you referenced above still accurate?

stelpstra
Community Champion

Hi @Edina_Tipter, could you provide an update on the release of CD2 in Europe (the Netherlands in our case)?

stelpstra
Community Champion

Good to see that the conversation continues in another blog post: https://community.canvaslms.com/t5/The-Product-Blog/Canvas-Data-2-is-coming/ba-p/552312

dgarciat
Community Member

@Edina_Tipter can you share the equivalent link to this one, but for CD2? https://portal.inshosteddata.com/docs

Edina_Tipter
Instructure Alumni
Instructure Alumni

@dgarciat For this specific one it would be the Open API spec. On the top of the doc you can find entity relationship diagram link, PostgresSchema link, DML and JSON schema.

However, it could be useful that we have more guides and resources on Community such Postman collection for quick start, CLI and Python Client Library that you might also want to check.