Upload files to Canvas using Python

[a_mottura]

Hi,

I am trying to write a Python script to upload a file to course files in Canvas using the API. I have followed the API tutorial (Uploading Files - Canvas LMS REST API Documentation). Unfortunately I get stuck on Step 2. I have looked through past questions, but could not find anything similar.

import requests

# Access token and API url
access_token = 'insert token here'
api_url = 'https://canvas.bham.ac.uk/api/v1/courses/22416/files'

# Set up a session
session = requests.Session()
session.headers = {'Authorization': 'Bearer %s' % access_token}

# Step 1 - tell Canvas you want to upload a file
payload = {}
payload['name'] = 'test.pdf'
r = session.post(api_url, data=payload)
r.raise_for_status()
r = r.json()
print ' '
print r # This successfully returns the expected response...

# Step 2 - upload file
payload = list(r['upload_params'].items()) # Note this is now a list of tuples
print ' '
print payload
with open('test.pdf', 'rb') as f:
     file_content = f.read()
payload.append((u'file', file_content)) # Append file at the end of list of tuples
r = session.post(r['upload_url'], data=payload)
r.raise_for_status() # This returns 400 Client Error: Bad Request
print ' '
print r‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

I probably want to say a few extra details about Step 2. On line 21, I convert the upload_params dictionary to a list of tuple because the file upload tutorial says the file should be added at the end of the upload_params...and dictionaries in Python are not ordered. I assume this is ok. The code stops on line 28, as line 27 returns a 400 Client Error. I have not written code for Step 3 as I cannot move past Step 2.

Any ideas?

Thanks for your help!

Alessandro

[kona]

Hi  @a_mottura ‌! I'm not sure of the answer to your question, but I'm going to share it with the https://community.canvaslms.com/groups/canvas-developers?sr=search&searchId=cb3e6585-7f3d-4ab7-9388-...‌ group in the Community to see if they can help. I'd also recommend joining this group so you have access to their information and resources!

Hope this helps!

[a_mottura · Accepted solution]

Hmmm...I think I was too tired to see it...! I was so worried about the structure of the payload in step 2 that I had not looked at the post command. I was posting in step 2 using the same session data as in step 1...which meant the second request returned the 400 Client Error...

Corrected Step 1 and Step 2 code (note change on line 28):

import requests

# Access token and API url
access_token = 'insert token here'
api_url = 'https://canvas.bham.ac.uk/api/v1/courses/22416/files'

# Set up a session
session = requests.Session()
session.headers = {'Authorization': 'Bearer %s' % access_token}

# Step 1 - tell Canvas you want to upload a file
payload = {}
payload['name'] = 'test.pdf'
payload['parent_folder_path'] = '/'
r = session.post(api_url, data=payload)
r.raise_for_status()
r = r.json()
print ' '
print r # This successfully returns the expected response...

# Step 2 - upload file
payload = list(r['upload_params'].items()) # Note this is now a list of tuples
print ' '
print payload
with open('test.pdf', 'rb') as f:
    file_content = f.read()
payload.append((u'file', file_content)) # Append file at the end of list of tuples
r = requests.post(r['upload_url'], files=payload)
r.raise_for_status() 
r = r.json() # The requests now works and returns response 200 - not 301. 
print ' '
print r‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍ # This is a dictionary containing some info about the uploaded file

‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Also, I noticed that step 2 must be posted using multi-part form data...which means I should use 'files' istead of 'data', and I have added 'parent_folder_path' (line 14) to decide where the file goes in course files.

Now - the weird thing is that step 3 as described in the tutorial is not required. The script above just works...and I see the file appear in Canvas. Yay!

Cheers,

Alessandro

[pjztam]

I'm having issues using this script when using the local filesystem to store files 

this is the error I'm getting

File uploads worked when we switched to using S3.

Just a heads up for anybody struggling with uploading files programmatically.

created_at         | 2018-01-14 23:20:36.665713
updated_at         | 2018-01-14 23:20:36.665713
email              | unknown-canvas-zucchini-services@instructure.example.com
during_tests       | f
user_agent         | python-requests/2.18.4
request_method     | post
http_env           |
subject            |
request_context_id | cd9aee89-b144-4335-ba4d-81fb6bf26902
account_id         | 1
zendesk_ticket_id  |
data               | ---                                                                                                                                                                                                                                                                                                                                                                                              +
                   | type:                                                                                                                                                                                                                                                                                                                                                                                            +
                   | response_code: 500                                                                                                                                                                                                                                                                                                                                                                               +
                   | format: !ruby/object:Mime::Type                                                                                                                                                                                                                                                                                                                                                                  +
                   |   synonyms: []                                                                                                                                                                                                                                                                                                                                                                                   +
                   |   symbol:                                                                                                                                                                                                                                                                                                                                                                                        +
                   |   string: "*/*"                                                                                                                                                                                                                                                                                                                                                                                  +
                   |   hash: 4108667749460886624                                                                                                                                                                                                                                                                                                                                                                      +
                   | HTTP_ACCEPT: "*/*"                                                                                                                                                                                                                                                                                                                                                                               +
                   | HTTP_ACCEPT_ENCODING: gzip, deflate                                                                                                                                                                                                                                                                                                                                                              +
                   | HTTP_HOST: canvas.zucchini.services                                                                                                                                                                                                                                                                                                                                                              +
                   | HTTP_USER_AGENT: python-requests/2.18.4                                                                                                                                                                                                                                                                                                                                                          +
                   | PATH_INFO: "/files_api"                                                                                                                                                                                                                                                                                                                                                                          +
                   | QUERY_STRING: "?"                                                                                                                                                                                                                                                                                                                                                                                +
                   | REQUEST_METHOD: POST                                                                                                                                                                                                                                                                                                                                                                             +
                   | REQUEST_URI: https://canvas.zucchini.services/files_api                                                          ... +
                   | SERVER_NAME: canvas.zucchini.services                                                                                                                                                                                                                                                                                                                                                            +
                   | SERVER_PORT: '443'                                                                                                                                                                                                                                                                                                                                                                               +
                   | SERVER_PROTOCOL: HTTP/1.1                                                                                                                                                                                                                                                                                                                                                                        +
                   | REMOTE_ADDR: 128.61.93.176                                                                                                                                                                                                                                                                                                                                                                       +
                   | path_parameters: '{:controller=>"files", :action=>"api_create"}'                                                                                                                                                                                                                                                                                                                                 +
                   | query_parameters: "{}"                                                                                                                                                                                                                                                                                                                                                                           +
                   | request_parameters: '{"Filename"=>#<ActionDispatch::Http::UploadedFile:0x00007ff415a3b540                                                                                                                                                                                                                                                                                                        +
                   |   @tempfile=#<Tempfile:/tmp/RackMultipart20180114-27009-1s6tbso>, @original_filename="Filename",                                                                                                                                                                                                                                                                                                 +
                   |   @content_type=nil, @headers="Content-Disposition: form-data; name=\"Filename\";                                                                                                                                                                                                                                                                                                                +
                   |   filename=\"Filename\"\r\n">, "key"=>#<ActionDispatch::Http::UploadedFile:0x00007ff415a3b4c8                                                                                                                                                                                                                                                                                                    +
                   |   @tempfile=#<Tempfile:/tmp/RackMultipart20180114-27009-ljvg2q>, @original_filename="key",                                                                                                                                                                                                                                                                                                       +
                   |   @content_type=nil, @headers="Content-Disposition: form-data; name=\"key\"; filename=\"key\"\r\n">,                                                                                                                                                                                                                                                                                             +
                   |   "acl"=>#<ActionDispatch::Http::UploadedFile:0x00007ff415a3b450 @tempfile=#<Tempfile:/tmp/RackMultipart20180114-27009-48ften>,                                                                                                                                                                                                                                                                  +
                   |   @original_filename="acl", @content_type=nil, @headers="Content-Disposition: form-data;                                                                                                                                                                                                                                                                                                         +
                   |   name=\"acl\"; filename=\"acl\"\r\n">, "Policy"=>#<ActionDispatch::Http::UploadedFile:0x00007ff415a3b3d8                                                                                                                                                                                                                                                                                        +
                   |   @tempfile=#<Tempfile:/tmp/RackMultipart20180114-27009-vsoctn>, @original_filename="Policy",                                                                                                                                                                                                                                                                                                    +
                   |   @content_type=nil, @headers="Content-Disposition: form-data; name=\"Policy\"; filename=\"Policy\"\r\n">,                                                                                                                                                                                                                                                                                       +
                   |   "Signature"=>#<ActionDispatch::Http::UploadedFile:0x00007ff415a3b360 @tempfile=#<Tempfile:/tmp/RackMultipart20180114-27009-nk2xhe>,                                                                                                                                                                                                                                                            +
                   |   @original_filename="Signature", @content_type=nil, @headers="Content-Disposition:                                                                                                                                                                                                                                                                                                              +
                   |   form-data; name=\"Signature\"; filename=\"Signature\"\r\n">, "content-type"=>#<ActionDispatch::Http::UploadedFile:0x00007ff415a3b2e8                                                                                                                                                                                                                                                           +
                   |   @tempfile=#<Tempfile:/tmp/RackMultipart20180114-27009-zc4pl4>, @original_filename="content-type",                                                                                                                                                                                                                                                                                              +
                   |   @content_type=nil, @headers="Content-Disposition: form-data; name=\"content-type\";                                                                                                                                                                                                                                                                                                            +
                   |   filename=\"content-type\"\r\n">, "file"=>#<ActionDispatch::Http::UploadedFile:0x00007ff415a3b270                                                                                                                                                                                                                                                                                               +
                   |   @tempfile=#<Tempfile:/tmp/RackMultipart20180114-27009-3whmx8>, @original_filename="file",                                                                                                                                                                                                                                                                                                      +
                   |   @content_type=nil, @headers="Content-Disposition: form-data; name=\"file\"; filename=\"file\"\r\n">}'                                                                                                                                                                                                                                                                                          +
                   | exception_message: 'undefined method `unpack'' for #<ActionDispatch::Http::UploadedFile:0x00007ff415a3b360>'                                                                                                                                                                                                                                                                                     +
                   | hostname: ip-172-31-60-132                                                                                                                                                                                                                                                                                                                                                                       +
                   | pid: 27009                                                                                                                                                                                                                                                                                                                                                                                       +
                   |
category           | NoMethodError‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[havy211]

This example work for me. 

import requests
import json
import os

# --------- TEST URL ---------
url_submissions = 'https://canvas.harvard.edu/api/v1/courses/<number course>/assignments/<number assignments>/'
test_url = url_submissions + '/submissions/<number submissions>'
# --------- TEST URL ---------

# Step 1 - tell Canvas you want to upload a file
post_api_url_step1 = test_url + '/comments/files'
step1 = requests.post(post_api_url_step1, data=json.dumps(
 {'name': <name file>, 'size': os.path.getsize(jupyter_notebook_file),
 'content-type': 'binary/octet-stream', 'on_duplicate': 'overwrite'}),
 headers={"authorization": "Bearer " + self.token})

step1.raise_for_status()
print("Step 1 Status Code: ", step1.status_code, "Post URL: ", post_api_url_step1)
print(step1.json())
print()

# Step 2 - Upload the file to the temporal url that canvas give you
post_api_url_step2 = step1.json()['upload_url']
step2 = requests.post(post_api_url_step2, files={"file": open(jupyter_notebook_file, 'rb')})
step2.raise_for_status()
print("Step 2 Status Code: ", step2.status_code, "Post URL: ", post_api_url_step2)
print(step2.json())
print()

# Step 3 - Activate the file in canvas
get_api_url = step2.json()['location']
step3 = requests.get(get_api_url, headers={"authorization": "Bearer " + self.token})
step3.raise_for_status()
print("Step 3 Status Code: ", step3.status_code, "Post URL: ", get_api_url)
print(step3.json())

# Step 4 - Make a comment and assign the file to a specific assignment with a grade
post_api_url_step4 = test_url
step4 = requests.put(post_api_url_step4, data=json.dumps({
 "comment": {"text_comment": "<comment>",
 "file_ids": [step3.json()['id']]},
 "submission": {"posted_grade": <grade>}}), headers={"Content-Type": "application/json",
 "authorization": "Bearer " + self.token})
step4.raise_for_status()
print("Step 4 Status Code: ", step4.status_code, "Post URL: ", post_api_url_step4)
print(step4.json())
print()

# Step 5 - See the Submission Summary
post_api_url_step5 = url_submissions + '/submission_summary'
step5 = requests.get(post_api_url_step5, headers={"authorization": "Bearer " + self.token})
step5.raise_for_status()
print("Step 5 Status Code: ", step5.status_code, "Post URL: ", post_api_url_step5)
print(step5.json())
print()

# Mark submission as unread
post_api_url_step6 = post_api_url_step4 + '/read'
step6 = requests.delete(post_api_url_step6, headers={"authorization": "Bearer " + self.token})
step6.raise_for_status()
print("Step 6 Status Code: ", step6.status_code, "Post URL: ", post_api_url_step6)
print(step6.status_code)
print()

[lcchalan]

Esto les puede ayudar, lo solucioné de esta manera

import requests

import json

url = "https://<canvas>/api/v1/users/self/files?name=nombre_archivo.png&size=tamaño_de_archivo_en_bytes&parent_folder_path=Nombre_de_carpeta"

payload={}

headers = {

'Authorization': '<token>',

}

# Le decimos a canvas que vamos a subir un archivo

response = requests.request("POST", url, headers=headers, data=payload)

# Verificar, si se ejecutó el llamado emitirá un código 200

print(response.status_code)

print("")

data_json = json.loads(response.text)

# token para subir el archivo

upload_url = data_json['upload_url']

upload_params = data_json['upload_params']

# Parte importante, aquí debes actualizar los parámetros obtenidos y agregar "None" a cada parámetro,

for item in upload_params:

upload_params[item] = (None, upload_params[item])

# agregamos el parámetro "file"

upload_params['file'] = '/path/nombre_archivo.png', open('/path/nombre_archivo.png', 'rb')

# Una vez actualizados los parámetros y agregado el "file", manda a imprimir y vas a obtener algo así

# {'filename': (None, 'prueba.png'), 'content_type': (None, 'image/png'), 'file': ('/path/nombre_archivo.png', <_io.BufferedReader name='/path/nombre_archivo.png'>)}

print(upload_params)

# Finalmente, hacemos el llamado con el nuevo token generado y con los parámetros actualizados

resp = requests.post(upload_url, files=upload_params)

print(resp.status_code)

# Listo!!!, puedes ir a Canvas y el archivo estará subido