How should I check a text file to see if it has a new course for my Canvas project?

[sam_ofloinn]

Hi, I hope this is the right place to ask.

I have a "courses.txt" which contains the courses for my Canvas community. This will have new classes added to it as time goes on, and I want a way to update my courses on the regular. In this case, I have a "courses.txt" that contains data in a fixed pattern about all the available courses, including name and ID. This will have new values added regularly to it.

Now I want to check Canvas, and using some detail there, compare it to this text file to see which courses 

What I want is a way to check which of these courses are new ones - that the database doesn't recognise them already. But how do I check this? What value from Canvas should I check?

Right now, I'm checking the most recent sis_import_id to the ID values in courses.txt; if they don't match, the course is considered new. But am I anywhere near the right track? Is there a better metric for what I want to do?

Any help on this would be greatly appreciated.

[robotcars]

Have you seen the various account reports?

Many course reports exist, including Storage, Unpublished, and Unused, and can be run by Term.

How do I view reports for an account? 

How you compare two files is probably going to be a matter of skills and preference.

If you use the same report from Canvas each time you download it, you could use something like BBEdit ($) or Text Wrangler(-$), and compare 2 files.

I'm sure there's a way to do this in Excel, but I don't excel at Excel and I don't do Office. I bet  @James ‌ does. :smileygrin:

If you have Linux/Unix/Mac, you can use diff

https://unix.stackexchange.com/questions/429310/compare-2-csv-files-and-output-only-difference-into-...

https://stackoverflow.com/questions/38950802/how-to-display-line-numbers-in-side-by-side-diff-in-uni...

A simple script could be written to open 2 text files (CSV) and compare a column and then diff them by what doesn't exist. ie, https://ruby-doc.org/stdlib-2.0.0/libdoc/csv/rdoc/CSV.html

ruby - How to compare data in two CSV files - Stack Overflow 

Depending on your permissions/admin level, you can use Canvas Data, which would provide all the courses for your Canvas and comes in a text file. Canvas Data Portal 

Have you checked the API? Courses - Canvas LMS REST API Documentation 

[James]

I might know how to find differences with Excel. It wouldn't be my first choice for things like this, but if you didn't have other tools available to you, it can be done. A Google search for comparing two lists with Excel would turn up about the same instructions I would come up with.

Based off what Lee has described, I would take the provisioning or SIS report from Canvas and compare it to the other list. Things that don't have a matching sis_course_id or a matching course_id are new. I wouldn't use sis_import_id as that's a meaningless number for us.

That's not what I do in reality, of course. My database is the authoritative list that links our SIS with Canvas. I originally programmed it to look for courses that were created in Canvas that didn't come from me, but in the 6 years we've been using it, it's never happened. I just didn't know that would be the case when I set it up -- it was a case of over building.

[jsavage2]

HI Lee,

You haven't told us *why* you want to compare your list to Canvas. Do you need to then do an SIS import to create the courses in Canvas? If so, then you don't really need to do the comparison yourself; you're on the right track with the sis import: just turn on diffing mode for the import (as described here: https://canvas.instructure.com/doc/api/file.sis_csv.html) and let Canvas do the diff for you. 

If you want to compare your file to Canvas for some other reason, as James suggest, the Provisioning report is probably your friend. You can also generate a list of courses with sis_course_id directly from the courses endpoint of the api.

[sam_ofloinn]

Jay, thanks for your reply.

To clarify, the goal is to spot any new courses so I can run a separate script that updates their template - to be precise, it makes their template the same as a pre-existing one. I realise that part is a separate issue, so I thought it unnecessary to bring up.

I'll be trying both of those solutions, thank you very much.

[jsavage2]

Ah, that is a little more complicated. 

I'm still not clear, though: is your text file is being crated by an external process, and then you want to query Canvas and see if the courses in the text file exist in Canvas and have the correct template? Or are you querying Canvas to see if there are new courses, there, that *don't* exist in the text file, which you will then add to the text file and push updated templates to?

If the former, the most effective thing is probably to query the courses API endpoint (Courses - Canvas LMS REST API Documentation ). That will return "resource does not exist" if the course hasn't been created in Canvas, and a Course object with the the "created_at" timestamp if it does. 

If the latter, you can query the accounts endpoint to get a list of all the courses in your sub account: Accounts - Canvas LMS REST API Documentation 

[sam_ofloinn]

Jay, thanks again for the reply.

Everything relating to the "courses" text/csv file's existence comes to be through processes unrelated to my current process - in fact, I don't believe my current project alters the file at all. So, to clarify, the former.

I'll read through that documentation you linked now and see my options then, thank you

EDIT: was able to get the "created_at" value from the list, thank you very much. I believe that'll be enough, going forward