[ARCHIVED] Obtaining Assignment Names for all courses in a term

[james_umphres1]

Hi all,

My school has canvas data set up but our leadership wants a way around the two day delay. I have very minor experience with the API but am trying to find a way to pull all assignment data on a daily basis. Specifically I'm looking at obtaining all assignments with the word "final" in the title.

I found a post on the community that provided an excel sheet coded to do this. It works, but my school has 10K + classes per semester and the sheet crashes or times out with anything over a few hundred.

Has anyone come across a script that does this? I've checked out github but haven't really found anything.

Thanks

[robotcars]

Hi  @james_umphres1 ,

Are the courses creating new assignments every day? If the assignments are already part of the course, they would be in CD, even if unpublished. Anything created today would be delayed. You could use Live Events assignment_created to append the daily data to CD and fill the latency gap.

Canvas Data Services Overview

Same with submissions*

We use both CD and LE. CD is loaded nightly to refresh everything. Then we continuously import all LE in real time to create views with both, returning up to date submission activity.

There some resources for LE in the community and here https://community.canvaslms.com/docs/DOC-17354-awesome-canvaslms#CanvasLiveEvents 

Check out  @millerjm ‌'s presentation Combining canvas data and live events to have timely data 

* Of course as you do this you continually find yourself appending LE events onto CD views to make sure you have all the updated enrollments, users, and anything else you need. I favor Canvas Raw format events over Caliper format as they align with CD much better and there are more events to fill your needs.

Happy to share more details if you like.

[millerjm]

If you have a database set up locally, you have two options:  

1.  Use the API.  Iterate through every single assignment in every single course for the term.  You'd then need to check the title, if it has final in the title then pop it into database table.  Obviously that is going to be a mess with as many courses you have.  

2.  Use Live Events - Canvas Raw Format is going to look just like API data for you.  You would need to work with your CSM to set it up and set up an Amazon Web Services account for SQS. Then you have to set up scripts to download the SQS messages, add them to your database, and then delete from the SQS queue.  I use a separate table for the SQS data and then combine it in a union query and take the MAX updated_at.  I do not do anything to update the original canvas data because I want to know where the data came from.  My database is MySQL and all my scripts are in Python and Bash.  

We are using it for assignments, submissions, users and enrollments.  We have around 12-13K FTE to give you an idea of our size.  We have paid Amazon Web Services less than $5 over the past year we've been using Live Events.  Our first bill was 15 cents and our IT department thought it was a mistake.