How can I search for URLs or specific content across all courses?

[labrewster]

Is there an API or other useful method that can be used for mass exporting all courses from a term, semester, or year? I'll mention the Canvas Community as well to see if anyone out there might know the best solution.

Thanks!

[kbickell]

I've been asking for such a feature for a while...

[Stef_retired]

 @labrewster , this presentation from InstructureCon 2015 describes how to push content to multiple course, but I wonder if Postman can be deployed to do the reverse. Check out  

[kmeeusen]

Hi  @lbrewster-c ‌

Can I ask why?

the total storage space would be huge, the download time would be long, and all of your courses are maintained on Amazon servers until the end of time.

I prefer to let Canvas/Amazon take care of my storage, but you may have a valid reason and that is why I asked.

Kelley

[labrewster]

Sure!  The reason we wanted to do this was to be able to write scripts that would be able to pull out and discern URLs that teachers might need to be white-listed or unblocked.  Currently, all teachers have to email us and we have to add specific URLs to our filter one-by-one.  However, if we were able to pull their courses and just have a script that ran through the course and picked up this specific info, we could white list specific sites from this point.  Since we use Canvas as a one stop shop for lesson planning, etc., this would be a much easier process to pull from here for that info.  Let me know if you need further clarification.  Thanks!

[kmeeusen]

Okay, now that I have the "why",  @lbrewster-c I think I might be able to find you some help that can provide an easier route for you. I suspect this can be done through the Canvas API, so I am going to share your question with the https://community.canvaslms.com/groups/canvas-developers?sr=search&searchId=a48edcc2-bb59-46ab-8433-...‌ group where there are a whole bunch of folks who are much smarter than I am - being smarter than me is easy, but being smarter in a way helpful to you is what I am aiming for.

Kelley

[mcarruth · Accepted solution]

 @labrewster  

This information is also available in Canvas Data.  You can query inside the assignment_dim, discussion_entry_dim, discussion_topic_dim, wiki_page_dim, and for ExternalUrl in module_item_dim.  Searching CanvasData is much faster that crawling through every course with the API

below are some the queries I used to fine all the lynda.com URLs in our instance of Canvas.

##assigments
select cd.canvas_id, cd.name, ad.title, ad.description from assignment_dim ad
join course_dim cd on cd.id=ad.course_id
where ad.description like '%lynda.com%' and (cd.name like 'MASTER%' or cd.name like'WORKING%');

##pages
select cd.canvas_id, cd.name, wpd.title, wpd.body from wiki_page_dim wpd
join wiki_page_fact wpf on wpf.wiki_page_id=wpd.id
join course_dim cd on cd.id=wpf.parent_course_id
WHERE wpd.body like '%lynda.com%' and (cd.name like 'MASTER%' or cd.name like'WORKING%');

##Discussion
SELECT cd.canvas_id, cd.name, dtd.title, dtd.message FROM discussion_topic_dim dtd
JOIN discussion_topic_fact dtf on dtf.discussion_topic_id=dtd.id
join course_dim cd on cd.id=dtf.course_id
where message like '%www.lynda.com%' and (cd.name like 'MASTER%' or cd.name like'WORKING%') ;

##ExternalURL
SELECT cd.canvas_id, cd.name, mid.title, mid.url FROM module_item_dim mid
JOIN course_dim cd on cd.id=mid.course_id
where mid.url like '%www.lynda.com%' and (cd.name like 'MASTER%' or cd.name like'WORKING%') ;

[labrewster]

Thanks Martin!  Extremely helpful!

[mcarruth]

You are welcome.

[awilliams]

Hey  @labrewster ‌. I believe mcarruth‌ has given you some great advice on using the raw data that is available to institutions to find specific types of content within courses. With the explanation you've provided here, I was wondering if you would mind changing ,or allowing us to change, the title and description? In the event Martin's answer ends up being marked correct, changing the title/description will make it easier for people with the same question to find his answer in the future.

[labrewster]

Yes.  The title and description can definitely can be changed.  That pulled exactly the info I needed.  Thanks again!