TLDR;  I wrote a script that creates a Catalog Session and then downloads all Users' Transcripts. There are some subtle authentication tricks happening so if the code does not make sense refer to the following text.    Why Did I want to download all Catalog User Transcripts? Recently we have been looking into institution-branded storefront services other than Catalog. In this process, we have had to look into how to download every transcript before possibly sunsetting our Catalog instance. The simplest way to preserve the user data is to download a user's transcript. Unfortunately, the transcript PDFs are not accessible by the API. However, I have found a workaround for this and I wanted to share it with others who are interested in creating a local store of transcript PDFs.    What do I need before I do this?  You will need to have admin rights to both the Catalog instance and the Canvas instance that is linked to the Catalog instance.    Why Can't I Use My API Token to Authenticate? To access transcripts seems relatively trivial; one could simply iterate through the Catalog User_Ids and make GET requests to '<CATALOG_DOMAIN>/transcripts/transcript.pdf?user_id=<USER_ID>'. Since this is not accessible with the API, you must simulate a 'login' or create a session ( python documentation ). Without creating a session any requests will be rerouted to the /login page.  This login page contains information that is passed to the login POST request that is hidden to the user. To obtain that info I use the lxml (https://lxml.de/lxmlhtml.html ) package to parse the HTML for these hidden values and then add my username and password to the form before sending it in a POST request to log in.   Why Can't I Make a Basic Request Now That I am 'Logged In'? After simulating a login, I found that making a GET request to /transcripts/transcript.pdf?user_id=<USER_ID> would redirect me to /transcripts/transcript.pdf which is my own transcript. When I looked at the history of the redirect ( see the function history() ) I noticed that the parameter user_id was lost in the first redirect which was to /login?force_login=0&target_uri=%2Ftranscripts%2Ftranscript.pdf .  However, if I requested the page /login with params force_login=0 and target_uri= %2Ftranscripts%2Ftranscript.pdf%3Fuser_id%3D<USER_ID> , the request would ultimately redirect to the desired page! I am not fully sure why this worked and if anyone has any idea why I would love to know.       Python Script   some information has been omitted for privacy and clarity. Please post a comment if there is anything unclear    # Fill in your details here to be posted to the login form. username = config.get('catalog','username') password = config.get('catalog','password') canvas_catalog_domain = config.get('instance','canvas_catalog') catalog_domain = config.get('instance', 'catalog') catalog_headers = { 'Authorization': 'Token token="%s"' % (config.get('auth_token','catalog')) , } catalog_ids = ## A LIST OF CATALOG USER IDS ## # Use 'with' to ensure the session context is closed after use. with requests.Session() as s: login = s.get(catalog_domain+'/login/canvas') # print the html returned or something more intelligent to see if it's a successful login page. #print(login.text) login_html = lxml.html.fromstring(login.text) hidden_inputs = login_html.xpath(r'//form//input[@type="hidden"]') form = {x.attrib["name"]: x.attrib["value"] for x in hidden_inputs} #print("form: ",form) form['pseudonym_session[unique_id]']= username form['pseudonym_session[password]']= password response = s.post(catalog_domain+'/login/canvas',data=form) #print(response.url, response.status_code) # gets <domain>?login_success=1 200 #pp(response.headers) # An authorised request. if int(response.code) != 200: raise Exception("Login failed with :", response.code ) for user_id in catalog_ids: #print('user_id: ',user_id) # getting transcript pdf r = s.get(catalog_domain+'/login?force_login=0&target_uri=%2Ftranscripts%2Ftranscript.pdf%3Fuser_id%3D' + user_id, headers=catalog_headers) history(r) if int(r.status_code) != 200: # possible error error_log.write('%s -- %s ERROR \n' % (r.url, r.status_code)) else: # lets continue getting info filename = 'pdfs/%s_catalog_transcript.pdf' % (user_id) with open(filename, 'wb') as f: f.write(r.content) ## HELPER FUNCTION TO TRACE THE REQUEST ## def history(r): if r.history: print("Request was redirected") for resp in r.history: print('\t',resp.status_code, resp.url) print ("Final destination:") print('\t',r.status_code, r.url) else: print("Request was not redirected")‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍       I hope this post helps other Canvas Developers out there! Feel free to contact me if you are trying to troubleshoot running this script or a variation of it.    Maddy Hodges  Courseware Developer University of Pennsylvania ... View more

Hello! I was wondering if anyone has experience specifically with the Catalog API Documentation. I am trying to get a list of all Users and their certificates, however, I am finding that the API documentation is not clear enough for me to make successful calls to the endpoint. Below is the info provided in the docs: Endpoint GET https://www.my-catalog.edu/api/v1/completed_certificates Parameters Name Description Type user_idrequired Canvas Id of the user Integer only_certificates Indicates if the courses without certificates should be included, default is to include all courses Boolean In the documentation, it does not clarify where the required field 'user_id' should be implemented. I have tried to pass it as a URI such as /api/v1/completeted_certificates/<canvas_id> and also as a parameter in the header such as: headers = { 'Authorization': 'Token token="XXXXXXXXX"', 'user_id': '2998', } Neither of these requests resulted in a valid request.  For more background: I was able to troubleshoot the subtle difference in the authentication for Catalog and the Canvas instance where Catalog API specifies 'Authorization': 'Token token="XXXXX"' and Canvas API specifies  'Authorization': 'Bearer XXXXX'.  Also, I am able to make calls to other Catalog endpoints so I am sure that it is not a domain or API key issue/typo. If there are any developers that have experience with the Catalog API and could help me with this or let me know of a workaround that would be great!! ... View more