What is the least expensive way to get the data from the Canvas Data Portal?

[LindaDessert]

Hello, 

I am trying to get files from the Canvas Data Portal that we can read and use. We have used the 7Zip file reader to open the gz files as recommended and we can get a couple of fields to show, but the rest of the fields are errors and the fields that are showing are not helpful. There aren't any headers and the data is too large for Excel to read. I have read the FAQ's and through some of the forum discussions, however the last ones I could find were 2020. Is there any way to get the data in a readable form with out purchasing software like Redshift and then using Tableau?

Thank you. -Linda

[ben_hudson]

Hi Linda,

If you're a MSSQL environment we use the Eastern Michigan University Canvas Data Viewer project: https://github.com/EMU-CFE/CanvasDataViewer I had to make some minor tweaks when I did the initial setup (2019ish) and then a couple customs along the way to get things the way we need, but the core of the process still seems to work well. The GitHub issues thread has has some fixes in their comments.

Also keep in mind that Canvas Data 2 is on the way and current Canvas Data will only be available for a limited time after 2 is available to everyone. https://community.canvaslms.com/t5/The-Product-Blog/Canvas-Data-2-updates/ba-p/529377

Thanks,

Ben

[LindaDessert]

Hi Ben,

Thank you so much for this information, it is very helpful and I am excited for the Canvas Data 2. I am new to Canvas (1 month) and I am the analyst and administrator entering course and people data. I am not so much the builder of getting the data as I am the person that clicks on something and it gives me the data I need. While I understand some of what you are telling me and what we need, I don't know what my new company is capable of or what exactly I should be asking for. Would you be able to assist me in a quick blurb as to what I should be asking our BI team and if there are any costs to using the GitHub data viewer? I will refer them to all the information you provided, the Data FAQ's and any other information I found, but I need to get them interested first. I am hoping you can help me with that part. 

Thank you,

Linda

[LindaDessert]

Hi Ben,

Thank you so much for this information, it is very helpful and I am excited for the Canvas Data 2. I am new to Canvas (1 month) and I am the analyst and administrator entering course and people data. I am not so much the builder of getting the data as I am the person that clicks on something and it gives me the data I need. While I understand some of what you are telling me and what we need, I don't know what my new company is capable of or what exactly I should be asking for. Would you be able to assist me in a quick blurb as to what I should be asking our BI team and if there are any costs to using the GitHub data viewer? I will refer them to all the information you provided, the Data FAQ's and any other information I found, but I need to get them interested first. I am hoping you can help me with that part. 

Thank you,

Linda

[ben_hudson]

Hi Linda,

Sorry I forgot to come back here last week and respond.

The Eastern Michigan University data viewer is free, we run a virtual Microsoft SQL Server for the setup. So you'd be asking the BI team or your IT team to setup a server and the data viewer process. Our setup currently requires 1.2TB of storage, 12GB of RAM, 2 Logical Processors running at 2.3Ghz. Other than that just the time to get things working.

Let me know if you need anything else.

Thanks,

Ben

[LindaDessert]

Hi Ben,
I appreciate your help with this. :).
Have a great day!
-Linda

[Jeff_F]

@LindaDessert  -- also consider the staffing requirement. While the initial build out of the system and dashboards can easily consume a few people for several weeks, the ongoing care and feeding of reports and dashboards using Canvas Data does take some time. This is especially true if you have Canvas Data setup so that the tables link up with data from your SIS in a data warehouse.

[LindaDessert]

Jeff,

This is definitely the conversations we are having right now. My past experience with LMS's was that we received a nightly data dump and we were able to download reports into Excel from the LMS.  What are your thoughts on Canvas Data 2? Do you think this will eliminate the need for this type of tool when the Data 2 is released or will there be a need for a different tool or an additional purchase from Canvas?

Thanks,

Linda

[Jeff_F]

Hello @LindaDessert  - I've been using CD1 to create reports, dashboards, etc. for over four years now and have 991 related project files in that project directory. I mention that only to point out I have some experience with CD1. As for CD2, the new data schema uses data sets which is notably different than the current CD1 star schema. So everything I've created will need to be updated to the new data structure. This is both exciting but also a little daunting at the same time. But I like to be challenged.

As far as creating reports, dashboards, etc., if you desire an efficient system, I do recommend contracting with Canvas for their Redshift service. With this you can generate reports that pull data from the live connection or you can schedule data extracts every evening, etc. using a service such as Tableau Online. 

I'd be glad to join a call to demo some of what we have done using Tableau but also how we ended up downloading CD1 into our institutional data warehouse. We are now using OBIEE for dashboard delivery. Use of our organizations OBIEE tool enabled merging data between Canvas and the SIS. This is quite useful for academic assessment, program assessment, etc.

[LindaDessert]

Hi Jeff,

I didn't realize you had responded and I am sorry for not getting back to you sooner. This is very helpful and I have shared it with my team. We are still working out who in our organization can assist us in getting the conversation started. I will use all the information you and others have provided with them.

I will get back to you once we get some information for our side. 

Thank you,

Linda