Canvas Data 2 - Automation

[sor1]

I can now retrieve data from Canvas Data 2. My DAP version is 1.0.18.  Since it is so different from canvasdatacli, I was curious if anyone was working on processes to retrieve data and load it into AWS S3.

[robotcars]

I have not tried CD2 yet, but there's a CD1 example for AWS/S3 by @ColinMurtaugh you might be able to pick apart and build a working process.

https://community.canvaslms.com/t5/Canvas-Developers-Group/Build-a-Canvas-Data-Warehouse-on-AWS-in-3...

[nada-e]

How do we get the data from Data 2? I too need to get the data into AWS but none of the solutions I've found so far seem to work

[LeventeHunyadi]

I highly recommend the official Python client library if you haven't check it out yet. You could set up a PostgreSQL database in AWS Aurora, run the client on an AWS EC2 instance, and have the client library replicate the data from DAP API to AWS Aurora by means of commands initdb and syncdb. This is similar to how we test the client library internally.

[nada-e]

Is there a way to maybe fetch the data as is and upload it to S3 first? Kind of similar to what ColinMurtaugh did here.