Find all Public Courses using API

[ysmalls]

Hello,

My disclaimer:  I’m not a programmer but wanting to learn.

 @dwillmore   had posted created a post regarding course’s that were set to Public visibility and needed update them to private and I’m at a point where I need this done.

I started by running the Public Courses report by Term for this school year only.

Thanks the response from  @James   I was able to create API calls using Postman but some data is quirky.

I first ran the Public Courses report by Term.

I created a Get API that list all my public courses( again thanks James Jones):

GET /api/v1/search/all_courses?public_only=1&per_page=100

However, my results only list 47 courses and I know there are more since I was cross-referencing them again the report I had generated but I moved on to the next API call.

I then created a PUT API to update the course to private (again thanks  @James  😞

PUT /api/v1/courses/:course_id?course[is_public]=0

Now when I run this for just one course it does work but funny thing is that this will then select “Customize” and set the Syllabus to “Public”?  Weird right or am I missing something?

Therefore, to eliminate that issue I change the API call to:

PUT /api/v1/courses/:course_id?course[is_public]=0&course[public_syllabus]=0

Finally, I got what I wanted and was able to update one course to private and keep the syllabus as private.

So I wanted to be able to update all courses through API call using Postman but realized that’s not possible?  Well it is kind of but I would need to create something to iterate or a collection that will run one course at a time?  Again this is using Postman and way above my head and I’m really liking API’s!

I am getting help from one of our programmers  @bbisbee  and we are working on a PowerShell script but have noticed that the search API does not appear to be returning the results from all courses, but rather just the ones that public in the index. We were able to get all of the course by running GET /api/v1/accounts/1/courses?per_page=100. It doesn’t seem like we are able to filter on that through the API call for is_public so we are downloading all of the courses and working through them with a powershell script to get just the public ones. Is there a better way to get this info through the API?

We also use the Canvas Data API which in the courses table does include which of our courses are publicly visible, but that only updates once a day, so we are trying to find a more “on-demand” method that is faster. 

We have the powershell script set up to iterate through all of the records and flip them to private, but just wanting to find a faster way to get the courses that are public

[bbisbee]

One thing we forgot to add in the above is that it looks like "public_syllabus" is flipped. From what I could tell, when public_syllabus = True, it's actually disabled, but when public_syllabus = False, it's actually enabled. Am I misunderstanding something with this?

[James · Accepted solution]

ysmalls,

I'm not sure that I correctly separated question from information, but I'll take a stab at what I think some of the questions are.

There are several fields in the course record that have to do with visibility. First, let's look at Canvas and then what happens within the data from the API.

• The Visibility setting can be be: course, institution, or public
• The Customize option allows you to make the syllabus available to just the course, the institution, or public, but never at a lower level than the course visibility. By that, I mean you cannot restrict the syllabus to a smaller group than the course, but you can open the syllabus up to a larger group than the course. This is also why setting a course to public forces the syllabus to be public; that is the same behavior that happens with the web UI and is not specific to the API call. The reverse process is not automatic -- you can have the syllabus public while the course is not, so setting the is_public to false without also setting the public_syllabus to false won't disable the public syllabus.

In the API, there are is_public, is_public_to_auth_users, public_syllabus, and public_syllabus_to_auth. The first two are for the course visibility and the second two are for the syllabus visibility. The list all courses endpoint of the Search API only matches courses where the is_public is true -- that is, where the course visibility is set to public. It does not match course visibility = institution or places where the syllabus is publicly available but the course is not. Another thing is that it requires that the course be published (technically it's workflow_state='available'), so there may be some unpublished courses that have a course visibility of public, but this API call won't catch them.

To get a list of the full settings from the API, you will need to iterate through all of the courses. The fastest way I know to get these from the API is by using the list active courses in an account endpoint of the Accounts API. That can take some time to process if you have a lot of courses, so you may want to limit it with query parameters to courses that are published and have the current enrollment_term_id. I use completed=false, enrollment_type[]=student, published=true, state[]=available, state[]=deleted, state[]=completed, with_enrollments[]=true, but that's for the purpose I'm working with, which is not to find public course. You can use Canvas Data to find other courses, but if they're not current, they're unlikely to change as often, so this might be a workable solution if you don't have time to let it process all of the courses. When I fetch the list of courses, I use state[] = 

I cannot help at all with Postman. I use Advanced Rest Client but only for debugging purposes and I don't know what variable support or logic Postman has or allows. If I need to interact with an API like you're wanting to do, then I write a script to do it. That might be in PERL, PHP, or JavaScript for me, but Python seems popular and I have to admit that the more I look at it, I'm impressed with how much it can do in so little code. I wrote some JavaScript code for Node JS using the Bottleneck library that allows me to make multiple calls to the accounts API in parallel while limiting it to keep from hitting the limit. I allow for 30 concurrent requests and set them to make sure there is a minimum of 50 milliseconds between requests to keep the initial flood down. I limit the per_page to 50 instead of 100. What I do is fetch the first one to see how many are available then I do the rest of the requests in parallel. We have just under 500 courses for this term and it took about 1.5 seconds to fetch them last night. If I tried to fetch the serially, it would have taken much longer. This isn't something that I had to do fast, but I wanted the results to be as close to a snapshot as I could. This was for analytics in another software that ran at midnight and I wanted to be able to wait as late as I could before gathering the data so it was the most current information I could feed the other program. You shouldn't have that issue with your situation.

[ysmalls]

Hey  @James  thank you for all the information and link to resources!   @bbisbee  has created a script to run in Powershell which we tested out in the test environment and did exactly what we needed.  My head still hurts from all the information and coding but his amazing and figured it all out.

[richard10]

 @bbisbee ‌ ysmalls‌ Hi

We would like to export the course catalogue and publish to an external site.  Would it be possible for you to post a copy of the powershell script here? We use PHP but the logic should be easily transferable.

Many thanks

[bbisbee]

Hey  @richard10 ‌, 

It sounds like you just want to download a list of all of your courses? 

What ysmalls‌ and I are doing is downloading a list of all of our courses and storing them in a MS SQL Table and identifying which ones are set to public, and then turning that flag off.

What you could do to get started is run an api call for something like:

"http s://$instructureurl/api/v1/accounts/1/courses?per_page=100"

This will return the first 100 courses, but using a series of splits such as:

$maxpage = $link.split(",")[3].split("?")[1].split("&")[0].split("=")[1]

you can get the max number of pages to start iterating through each page (I'm using a while loop for this), and start storing the data in a database table of some sort.

Hopefully that will help you out some, if you need help with something more specific, let me know.