NEVER conclude courses!  We found this out the hard way.  It really creates more problems than it solves and concluding is pretty much the younger evil brother of the delete process.  You have been warned!

I've developed a tool I wanted to share here.  I teach multiple sections of a course with up to 72 students per section.  I typically merge all sections of my course into a single canvas site.  This works well for most things, but grouping 360 students into 60 teams manually is a nightmare.  The GUI for team management is an atrocious mess.

I've written a Google Apps Script in JavaScript that uses a Google Spreadsheet as the GUI.  With this tool, you can download your canvas roster, and then you can upload all teams automatically using another sheet.  

I wrote the tool to work with CATME team formation, specifically.  But, so long as the 'CATME Import' sheet contains the student email and team name, it should work fine for manual team formation.

This is my first stab at API coding, and so there are bound to be lots of errors and bugs.  For one, I don't have the OAuth2 worked out, so this version uses a temporary token that you get from your Canvas page.  

I've put a version of the code up on GitHub - GitHub - dagray3/canvas_api_scripts: collection of Google Apps Scripts in JavaScript for working wit... 

I don't know how to share the companion google sheet with others.  But, I think it's a good, rough beta that might be of use to someone.  Hit me up with a DM if you have questions or comments about the code.  Otherwise, be awesome.

In a Canvas course, you can quickly check the number of missing assignments for single students relatively quickly. You can also message groups of students missing specific assignments from the analytics page (or the gradebook). What you can't do is get a list of all students in a course and their missing assignments in a CSV for quick analysis.

In my never-ending exploration of the Canvas API, I've got a Python script that creates a missing assignments report for a course, broken down by section.

Sidebar...

I have my own specific thoughts about using the "missing" flag to communicate with students about work. The bigger picture is that while we're distance learning, it's helpful to be able to get a birds-eye view of the entire course in terms of assignment submission. We also have enlisted building principals to help check in on progress and having this report available is helpful for their lookup purposes.

The Script

from canvasapi import Canvas # pip install canvasapi
import csv
import concurrent.futures
from functools import partial


KEY = '' # Your Canvas API key
URL = '' # Your Canvas API URL
COURSE = '' # Your course ID

canvas = Canvas(URL, KEY)
course = canvas.get_course(COURSE)
assignments = len(list(course.get_assignments()))
writer = csv.writer(open('report.csv', 'w'))

def main():
    sections = course.get_sections()

    writer.writerow(['Name', 'Building', 'Last Activity', 'Complete', 'Missing'])

    for section in sections:
        enrollments = section.get_enrollments(state="active", type="StudentEnrollment")
        
        # Play with the number of workers.
        with concurrent.futures.ThreadPoolExecutor(max_workers=3) as executor:
            
            data = []
            job = partial(process_user, section=section)

            results = [executor.submit(job, enrollment) for enrollment in enrollments]
        
            for f in concurrent.futures.as_completed(results):
                data.append(f.result())
                print(f'Processed {len(data)} in {len(list(enrollments))} at {section}')
                
        writer.writerows(data)

def process_user(enrollment, section):
    missing = get_user_missing(section, enrollment.user['id'])
    return [ 
        enrollment.user['sortable_name'], 
        section.name, 
        enrollment.last_activity_at, 
        len(missing), ', '.join(missing)
    ]

def get_user_missing(section, user_id):
    submissions = section.get_multiple_submissions(student_ids=[user_id], 
                                                   include=["assignment", "submission_history"], 
                                                   workflow_state="unsubmitted")

    missing_list = [item.assignment['name'] for item in submissions \
        if item.workflow_state == "unsubmitted" and item.excused is not True]

    return missing_list


if __name__ == "__main__":
    main()‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

How does it work?

The script uses UCF's canvasapi library to handle all of the endpoints. Make sure to pip install before you try to run the script. The Canvas object makes it easy to pass course and section references around for processing. Because each student has to be individually looked up, it uses multiple threads to speed up. There isn't much compute, just API calls and data wrangling, so multithreading worked better than multiprocessing.

For each section, the script calls for each students' submissions, looking for workflow_state="unsubmitted" specifically to handle filtering on the Canvas servers. From this filtered list, it creates a final list by checking the submission history and any excused flags. A list is then returned to the main worker and the section is written as a whole to keep the processes thread-safe.

When the script is finished, you'll have a CSV report on your filesystem (in the same directory as the script itself) that you can use.

Improvements

Currently, missing assignments are joined as a single string in the final cell, so those could be broken out into individual columns. I found that the resulting sheet is nicer when the number of columns is consistent, but there could be some additional processing added to sort assignments by name to keep order similar.

Canvas is also implementing GraphQL endpoints so you can request specific bits of data. The REST endpoints are helpful, but you get a lot of data back. Cleaning up the number of bytes of return data will also help it run faster.

While schools are closed, we've moved much of our long term staff development material into Canvas. We have one long-running course with all staff split into site-based sections that has worked as a model for others. We needed a way to essentially duplicate the template course enrollments into new training courses.

Ignorance is bliss (sometimes) and I didn't know of a good way to make this happen. I looked at some of the provisioning reports, but I couldn't select a single course to run a report on. So, I reached for Python and the UCF Open canvasapi library to make it happen.

At the end of this process, I ended with a brand new course, populated with teachers enrolled in their specific sections. I was also able to disable the new registration email and set their course status to active by default.

from config import PROD_KEY, PROD_URL
from canvasapi import Canvas # pip install canvasapi

# Define your course IDs. Be careful!
template_course_id = ''
new_course_id = ''

canvas = Canvas(PROD_URL, PROD_KEY)

template_course = canvas.get_course(template_course_id)
new_course = canvas.get_course(new_course_id)

# Open the template course section by section
template_sections = template_course.get_sections()

# Get any sections that may already exist in the new course
new_sections = [section.name for section in new_course.get_sections()]

# This whole loop could be improved a little.
for section in template_sections:
    # Get all the section enrollments
    enrollments = section.get_enrollments()

    # If it's a brand new course, this should always be false
    if not section.name in new_sections:
        print(f'Creating section {section.name}')
        new_sections.append(section.name)
        course_section = {
            "name": section.name,
        }
        new_section = new_course.create_course_section(course_section=course_section)
        
        count = 0 # start counting enrollments for quick quality checks
        
        for enrollment in enrollments:
            student = enrollment.user['id']
            print(f'Enrolling {enrollment.user["name"]}')
            count += 1
            args = {
                "course_section_id": new_section.id,
                "notify": False,
                "enrollment_state": "active"
            }
            try:
                new_course.enroll_user(student, "StudentEnrollment", enrollment=args)
            except Exception as e:
                print(e)
        print(f'Enrolled {count} users in {new_section.name}')‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

It's definitely brute force, but it saved us from having to copy and paste nearly 1,300 users into the new course by hand from a spreadsheet.

Why force enroll at all?

I think this highlights one of the barriers for really taking Canvas to the next level for staff support. There is no good way to enroll non-student users in courses for required development. In our case, it's to fulfill a required training for staff and using Canvas makes sense as a lot is done through application and reflection.

The public course index in Canvas could be used, but without a great way to expose the course to instructional staff only (I know we could use some JavaScript and edit the template, but that's just another thing to manage) it could lead to students joining courses either by accident or maliciously.

We've also toyed around with making a custom self-signup process on an internal website where staff are forwarded directly to the enroll page, but it's another system to manage and another site for teachers to use. The most hands-off approach for all involved is to do something like this in the background as needed to get people where they need to be effectively and efficiently.

This post was originally published on my own blog.

In moving to online, we've tried to streamline all of our communication through Canvas. The goal is to cut down on disconnected email threads and encourage students to use submission comments to keep questions and feedback in context.

The Problem

Many students had already turned off email notifications for most communications in Canvas, preferring not to have any notices, which reduces their responsibility for teacher prompting and revision. Notifications are a user setting and the Canvas admin panel doesn't provide a way to define a default set of notification levels for users. However, with the API, we were able to write a Python program that sets notification prefs by combining the as_user_id query param as an admin that sets user notification preferences.

API Endpoints

• GET user communication channel IDs: /api/v1/users/:user_id/communication_channels
• PUT channel preferences for user: api/v1/users/self/communication_channels/{channel_id}/notification_preferences/{msg_type}

Params

• Int user_id
• Int channel_id
• String frequency

Get User IDs

There is no easy way to programmatically get user IDs at the account or subaccount levels without looping each course and pulling enrollments. Instead, we opted to pull a CSV of all enrollments using the Provisioning report through the Admin panel. We configured separate files using the current term as the filter. This CSV included teacher, student, and observer roles. The script limits the notification updates to student enrollments.

Script Details

The full program is available in a GitHub gist. Here is an annotated look at the core functions.

main handles the overall process in a multi-threaded context. We explicitly define a number of workers in the thread pool because the script would hang without a defined number. Five seemed to work consistently and ran 1500 records (a single subaccount) in about 7 minutes.

The CSV includes all enrollments for each student ID, so we created a set to isolate a unique list of student account IDs (lines 9-10).

To track progress, we wrapped the set in tqdm. This prints a status bar in the terminal while the process is running which shows the number of processed records out of the total length. This is not part of the standard library, so it needs to be installed from PyPI before you can import it.

def main():
    """
    Update Canvas user notification preferences as an admin.
    """
    unique = set()
    data = []
    with open('your.csv', 'r') as inp:
        for row in csv.reader(inp):
            if re.search("student", row[4]):
                unique.add(int(row[2]))

    with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
        with tqdm(total=len(unique)) as progress:
            futures = []
            for student in unique:
                future = executor.submit(process_student_id, student)
                future.add_done_callback(lambda p: progress.update())
                futures.append(future)
            
            results = [future.result() for future in futures]‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

process_student_id is called by the context manager for each student ID in the set. Canvas breaks communication methods into "channels:" email, push, Twitter, etc (line 3). Each channel has a unique ID for each user, so we needed to call each user's communication channels and then pass the ID for emails to a setter function.

def process_student_id(student):
    # Get their communication channel prefs
    channel_id = get_channel_id(student)

    try:
        # Update the channel prefs and return
        update = update_prefs(student, channel_id)
        return update
    except Exception as e:
        print(e)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

GET communication_channels

def get_channel_id(student_id):
    url = f"https://yourURL.instructure.com/api/v1/users/{student_id}/communication_channels"
    resp = requests.request("GET", url, headers=headers)

    for channel in resp.json():
        # find the ID of the email pref
        if channel['type'] == 'email':
            return channel['id']‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

PUT communication_channels/:channel_id/notification_preferences/:message_type[frequency]

The communication channel can receive several types of communications. We wanted to set the student notifications to "immediately" for new announcements, submission comments, and conversation messages. You can define others as well as their frequencies by modifying the values on lines 3-4.

The communication types are not well documented, so  we used our own channel preferences to find the notification strings: GET /users/self/communication_channels/:channel_id/notification_preferences.

The crux of this step is to make the request using the Masquerading query param available to the calling user. Make sure the account which generated the API key can masquerade or else the script will return an unauthorized error. 

def update_prefs(student_id, channel_id):
    # loop through different announcement types
    types = ["new_announcement", "submission_comment", "conversation_message"]
    frequency = "immediately"  # 'immediately', 'daily', 'weekly', 'never'
    responses = []

    for msg_type in types:
        url = f"https://elkhart.test.instructure.com/api/v1/users/self/communication_channels/{channel_id}/notification_preferences/{msg_type}?as_user_id={student_id}&notification_preferences[frequency]={frequency}"
        resp = requests.request("PUT", url, headers=headers)

        responses.append(resp)
    
    return responses‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Final Thoughts

Updating a user's personal preferences isn't something I was thrilled about doing, but given our current circumstances, it was preferable to the alternative of continuing to struggle to help students move forward in their coursework. Further improvements would be to call each CSV in the file system incrementally, cutting down on the time someone has to log in and run the script. Hopefully, this only needs to be done once and does not become a recurring task. Second, there is an endpoint in the API to update multiple communication preferences at once, but it isn't well documented and I wasn't able to get it working reliably. For just one channel and three specific types of messages, the performance improvements probably would have been negligible (at least that's what I'm telling myself).

This blog describes how to move user enrollments from one role to another using a Python class, SQL data, and a mapping file.

So here is the situation we are presently facing at Everett Public Schools.  Along with our base roles of Student, Teacher, Designer, etc., we also have custom roles that have been derived from those base roles.  These custom roles are a bit more refined and help keep users and there permissions in check.  The problem with this idea is that not everyone follows the rules when assigning a role to a user when that user is enrolled into a course.  This quickly becomes an issue when trying to search and sort users based upon their permissions.

Case in point: We have teachers that are enrolled as students in staff courses or portals that are located at their respective school or sub-account.  So are they truly a student in the classic sense?  No.  When you do a blind search for students, you get back a bunch of teachers and maybe a few other users that somebody down the line added to a course as a student.  Now that the user data set has gotten out of hand, how do you move those enrollments over to the new custom role that you just created?  In addition to that, how do you keep it all in sync?

The solution comes in a few simple steps which you can follow below.  First, you need to decide what data set of users need to be moved from one role to another?  In our case, we wanted non-students (i.e. district staff) that were currently assigned the base role of StudentEnrollment (aka Student).  These district IDs are the same as their login id and SIS id too, so it keeps things straight.  Since we run multiple nightly integrations, we simply just created a new section in our SQL code to only pull the district staff IDs.  Like this:

/*
STAFF USERS
*/
IF @type = 'STAFF_USERS'
BEGIN
SELECT login_id
FROM eps_canvas.dbo.users
WHERE user_type = 'F';
END

Just a bit of a backstory to explain the logic.  In Everett we use several nightly imports into Canvas to roster courses, control users, etc.  More on that in another blog, but to suffice it to say it works very well.  We use a 'users' table in a smaller database to control who gets put into Canvas.  The user_type of 'F' is for 'faculty'.  So when this script runs, it uses the 'staff_users' input parameter to control what data set the script will receive.  This logic comes from the script configuration .ini file:

[Default]

#API SIS upload URL for the site
#Root account should always be 1
rootURL: https://everettsd.beta.instructure.com/api/v1/

#The URL string data that allows acting as another user
#The 'replace' placeholder gets replaced with the correct term in the script
masqueradeData: {"as_user_id": "replace"}

#The list of parameters to pull from the DB
#Use this list to effect the role mappping below
#Comma delimited, any order
dbParams: staff_users

#Text of the SQL Server stored procedure SQL
#For getting of district ids
dbSQL: exec eps_internal.dbo.pyCurGetCanvasCustomExtracts ?

#The endpoint to get enrollments for a user
enrollmentsEndpoint: users/self/enrollments

#The endpoint to enroll the user in the course
coursesEndpoint: courses/{}/enrollments

#The endpoint to get all of the current roles
rolesEndpoint: accounts/1/roles

#The mapping from one role to another for each DB parameter
#The key for each map is keyed off of the dbParams list
#The JSON object for each dbParam is a key of the permission type to find, the value is the role to assign
#All values are case sensitive and must match exactly to what is in Canvas
roleMapping: {"staff_users": {"StudentEnrollment": "Adult Learner"}}

When the script is executed, it looks for an associated configuration file and reads in the [Default] section data.  It does read a master configuration file too so it can set some global variables, but that is outside the scope of this post.  Each parameter is then assigned to an internal variable that the script uses to do its thing.  Jumping down to the bottom line in the file, the roleMapping dictionary is keyed to the dbParams value.  This is how the data set knows what users to process, what role to look for (in this case 'StudentEnrollment') and what role to use when enrolling the user into the current course ('Adult Learner').  If we wanted to process more users this this script workflow, then we add a value to the dbParams list and add the same value to the roleMapping dictionary along with the roles to use.  

At some point, we needed to create our 'Adult Learner' role.  We wanted a role that was student based but that could be used for staff members that are fulfilling some student role in a course somewhere.  We wanted the student role to truly reflect actual students in the district.

So now we are ready to roll.  Consider this Python class:

from requests import Session
from classEpsDB import EpsDB
from classEpsException import EpsException
from classEpsConfiguration import EpsConfiguration
from json import loads
from urllib import parse


class EpsITSyncCanvasEnrollments(object😞
    """
    Syncs the Canvas enrollments between what was assigned to a user and what should be the correct assignment.
    We do this to keep users from getting the incorrect enrollment and streamlining the search process.
    @package: epsIT
    @license: http://opensource.org/licenses/GPL-3.0
    @copyright: 2020, Everett Public Schools
    @author: DPassey
    @version: 1.0, 02.24.2020
    """

    def __init__(self, user_id_type='sis_user_id'😞
        """
        Class initializer.
        Parses the config file_name, assigning values as needed.
        @raise exception: EpsException
        """
        try:
            cfg = EpsConfiguration(f"{self.__class__.__name__}.ini")
            self.rc = 0
            if not cfg.db_dsn: raise Exception(f"{self.__class__.__name__}.__init__. DSN data source is missing.")
            for k in cfg.locals:
                k = k.upper().strip()
                v = cfg.locals[k].strip()
                if k == 'DBSQL': db_sql = v
                if k == 'DBPARAMS': param_list = v.split(',')
                if k == 'ROOTURL': root_url = v
                if k == 'MASQUERADEDATA': masquerade = v
                if k == 'ENROLLMENTSENDPOINT': enroll_endpoint = v
                if k == 'COURSESENDPOINT': course_endpoint = v
                if k == 'ROLEMAPPING': roles_map = loads(v)
                if k == 'ROLESENDPOINT': roles_endpoint = v

            # set the session header
            self.header = {'Authorization': f'Bearer {cfg.canvas_token}'}

            # must be one of these
            if user_id_type not in ('sis_user_id', 'sis_login_id'😞 raise Exception(f'{self.__class__.__name__}.__init__. Invalid parameter: {user_id_type}.')

            # create a session
            with Session() as self.session:
                # get the type of user from the parameter list
                for _ in param_list:
                    # get all of the active roles
                    url = f"{root_url}{roles_endpoint}"
                    # for each mapped role for this parameter, get the role's id
                    roles_dict = self.get_account_roles(url, roles_map[_])
                    # get the data to process for each parameter
                    data = self.get_data(cfg.db_dsn, db_sql, _)
                    # proceed if we get user data
                    if data:
                        # for each user in the data, find the applicable enrollments to move
                        for user in data:
                            # set up masquerading
                            self.data_dict = loads(masquerade.replace('replace', "{}:{}".format(user_id_type, user[0])))
                            # get all of the user's enrollments to see if we need to change enrollments
                            user_dict = self.get_enrollments(f"{root_url}{enroll_endpoint}", roles_map[_])
                            # now process the users by their Canvas id
                            for user_id in user_dict:
                                # process each course and re-enroll the user
                                # we need to keep the indexing linked between course and enrollment
                                for c, course in enumerate(user_dict[user_id]['courses']):
                                    # get the role id of the new role
                                    # need this to move enrollments
                                    role_id = roles_dict[user_dict[user_id]['roles'][c]]
                                    # get the current enrollment id
                                    enroll_id = user_dict[user_id]['enrollments'][c]
                                    endpoint = course_endpoint.format(course)
                                    # now set the new enrollments
                                    self.set_enrollment(f"{root_url}{endpoint}", user_id, role_id, enroll_id)
        except:
            EpsException(__file__)

    def get_data(self, dsn, sql, param):
        """
        Executes the stored procedure and gets the applicable data set.
        @param dsn: String
        @param sql: String
        @param param: String
        @return: List
        @raise exception: EpsException
        """
        try:
            db = EpsDB(dsn)
            if not db: raise Exception(f"{self.__class__.__name__}.get_data. Could not connect to database.")
            rs = db.get(sql, param)
            if not rs: raise Exception(f"{self.__class__.__name__}.get_data. No data set returned.")
            return rs
        except:
            EpsException(__file__)

    def get_account_roles(self, url, role_dict):
        """
        Gets the active roles and puts them in a roles dictionary.
        @param url: String
        @param role_dict: Dictionary
        @return Dictionary
        @raise exception: EpsException
        """
        try:
            role_id_dict = {}
            # get all active roles
            data_dict = {'state[]': 'active', 'per_page': 100}
            resp = self.session.get(url, data=data_dict, headers=self.header)
            if resp.status_code == 200:
                # check the headers "link" attribute for the last relational link
                for link in resp.headers['Link'].split(','😞
                    if 'rel=last' in link.replace('"','').replace("'",'').lower():
                        # grab the total pages count by parsing out the url parts and convert to int
                        page_total = int(parse.parse_qs(parse.urlparse(link.split(';')[0])[4])['page'][0])
                        # we need to get all results since we are being paginated
                        # these sections perform the same logic, just easier to to write it this way
                        if page_total > 1:
                            p = 1
                            while p <= page_total:
                                data_dict.update({'page': p})
                                resp = self.session.get(url, data=data_dict, headers=self.header)
                                json = loads(resp.text)
                                for _ in json:
                                    if _['role'] in role_dict.values(): role_id_dict[_['role']] = _['id']
                                p += 1
                        else:
                            json = loads(resp.text)
                            for _ in json:
                                if _['role'] in role_dict.values(): role_id_dict[_['role']] = _['id']
            else: raise Exception(f"{self.__class__.__name__}.get_account_roles. Response {resp.text} returned.")
            return role_id_dict
        except:
            EpsException(__file__)

    def get_enrollments(self, url, map_dict):
        """
        Gets the roles for the user and place in a user dictionary.
        @param url: String
        @param map_dict: Dictionary
        @return Dictionary
        @raise exception: EpsException
        """
        try:
            user_list = []
            enrollments_list = []
            roles_list = []
            user_dict = {}
            # make a copy of the class data dictionary so we can update it
            data_dict = self.data_dict.copy()
            # we should never exceed the per_page value
            # i mean really....over 100 enrollments?
            # current_and_future is a special state for all courses, published and unpublished
            data_dict.update({'state[]': 'current_and_future', 'per_page': 100})
            resp = self.session.get(url, data=data_dict, headers=self.header)
            if resp.status_code == 200:
                json = loads(resp.text)
                for _ in json:
                    # check if user is enrolled in the course per the map_dict keys
                    if _['role'] in map_dict:
                        user_id, course_id, enroll_id = [_['user_id'], _['course_id'], _['id']]
                        user_list.append(course_id)
                        enrollments_list.append(enroll_id)
                        roles_list.append(map_dict[_['role']])
                # build the user enrollment dictionary for those mapped roles
                if user_list: user_dict = {user_id: {"courses": user_list, "enrollments": enrollments_list, "roles": roles_list}}
            else: raise Exception(f"{self.__class__.__name__}.get_enrollments. Response {resp.text} returned.")
            return user_dict
        except:
            EpsException(__file__)

    def set_enrollment(self, url, user_id, role_id, enroll_id):
        """
        Sets the user enrollment for the course by deleting the original enrollment, making a new one.
        @param url: String
        @param user_id: Int
        @param role_id: Int
        @param enroll_id: Int
        @raise exception: EpsException
        """
        try:
            # now we enroll the user in the proper role
            # we keep the enrollment type blank so the role id will override the base enrollment
            data = {"enrollment[user_id]": user_id, "enrollment[type]": '', "enrollment[role_id]": role_id, "enrollment[enrollment_state]": "active"}
            resp = self.session.post(url, data=data, headers=self.header)
            if resp.status_code == 200:
                # do not change the url as we want to delete the old enrollment now
                resp = self.session.delete(f"{url}/{enroll_id}", data={"task": "delete"}, headers=self.header)
                if resp.status_code == 200: self.rc += 1
                else: raise Exception(f"{self.__class__.__name__}.set_enrollment. Response {resp.text} returned.")
            else: raise Exception(f"{self.__class__.__name__}.set_enrollment. Response {resp.text} returned.")
        except:
            EpsException(__file__)


# end of class
x = EpsITSyncCanvasEnrollments()
print(x.rc)

This is the flow:

1. Read in the configuration .ini files, one that is global (the EpsConfiguration class) and one that named the same as this class
2. Assign the configuration values to class values
3. Query the database for the data set of user login ids
4. Get a data set of all of the roles that currently exists in our Canvas instance
5. For each user, act as that user and get all of the current and future enrollments
6. Using the mapping dictionary, find each enrollment that we need to change and get the role id value from the list of roles that were grabbed earlier
7. For each enrollment that is applicable for the user, enroll the user in the new role for the course and set it to active and then delete the old enrollment

And there you go.  You have moved all of your applicable enrollments over to the new one without having to do it manually.  Setting this script up as a regular job, depending on your needs of course, will ensure that your Canvas user role assignments don't get out of control.

We've been working for a while on leveraging the Canvas API to work with other systems for particular learning use cases. We're developing a middleware app using ASP.NET Core MVC to manage the integrations.

We've been using the access tokens that each Canvas user can generate to work with the API. This is fine for development and testing but when we need to extend usage we want to avoid requesting users create their own tokens. A neater solution is to authenticate directly into Canvas using OAuth and, from this, get a token for the logged in user that can be used for subsequent API calls. This maintains the context based security that is a key feature of the access token.

Before I get into the steps to to getting OAuth to work in ASP.NET Core MVC and the intricacies of connecting to Canvas I'll give you a link to a GitHub repo that contains a very simple example. This is not production code and is an example only.

I also want to acknowledge the series of posts by  @garth ‌ on the OAuth workflow in .NET. I wouldn't be writing this now if it wasn't for Garth. I also got a lot of help from this post by Jerrie Pelser that works through an example of using OAuth2 to authenticate an ASP.NET ....

Getting Started

In this example I'm using a local instance of Canvas running as a Docker container. If you want to follow along then install Docker Desktop. Then download and run lbjay's canvas-docker container. This container is designed for testing LTIs and other integrations locally and comes with default developer keys:

• developer key: test_developer_key
• access token: canvas-docker

You can also log in to the Canvas instance and add your own developer keys if you want to.

Other thing that you'll need to started is an IDE of your choice. I'll be using Visual Studio 2019 Community edition but you could use Visual Studio Code or another tool that you prefer.

Step 1 - Make sure that the test version of Canvas is running

Start Docker Desktop and load the canvas-docker container. Once it has initialised it is available at http://localhost:3000/ 

The admin user/pass login is canvas@example.edu / canvas-docker.

Step 2 - Create a new ASP.NET MVC Core 2.2 application

Start Visual Studio 2019 and select Create a new project.

Select ASP.NET Core Web Application.

Set the Project name.

In this case we're using an MVC application so set the type to Web Application (Model-View-Controller). Make sure that ASP.NET Core 2.2 is selected and use No Authentication as we're going to use Canvas.

Step 3 - Let's write some code

 OAuth requires a shared client id and secret that exists in Canvas and can be used by an external app seeking authentication. The canvas-docker container has a developer key already in it but you can add your own. 

The default key credentials are:

Client Id: 10000000000001

Client Secret: test_developer_key

You can get to the developer keys by logging in to your local instance of Canvas and going to Admin > Site Admin > Developer Keys.

Now we need to store these credentials in our web app. For this example we'll put them in the appsettings.json file. You can see the code that we've added in the image below. Please note that in proper development and production instances these credentials should be stored elsewhere. Best practice for doing this is described here: Safe storage of app secrets during development in ASP.NET Core.

In this case Canvas is the name of the authentication scheme that we are using.

Now the configuration for OAuth2 happens mostly in the startup.cs file. This class runs when the app is first initialised. Within this class is public void method called ConfigureServices in which we can add various services to the application through dependency injection. The highlighted zone in the image below shows how to add an authentication service and configure it to use OAuth.

The basic process is to use services.AddAuthentication and then set a series of options. Firstly we set the options to make sure the DefaultAuthenticationScheme is set to use Cookies and the DefaultSigninScheme is also set to use cookies. We set the DefaultChallengeScheme to use the Canvas settings from the appsettings.json file.

We can chain onto that a call to AddCookie(). And then chain onto that the actual OAuth settings. As you can see we set "Canvas" as the schema and then set options. The options for ClientId and ClientSecret are self explanatory. The CallBackPath option needs to set to be the same as that in the Redirect URI in the key settings in Canvas. You may need to edit the settings in Canvas so they match. The image below shows where this is located.

The three end points are obviously critical. The AuthorizationEndpoint and the TokenEndpoint are described in the Canvas documentation. The Authorization enpoint is a GET request to login/oauth2/auth. As you can see, there are various parameters that can be passed in but we don't really need any of these in this case.

The Token endpoint is a POST request to login/oauth2/token. Again, there are various parameters that can be passed in but we don't really need any here.

The UserInformationEndpoint was the hardest endpoint to work out. It is not explicitly mentioned in the documentation. There is a mention in the OAuth overview to setting scope=/auth/userinfo. I couldn't get that to work but I may have been overlooking something simple. In the end it became apparent that we would need an endpoint that returned some user information in JSON format. There is an API call that does just that: /api/v1/users/self 

The AuthorizationEndpoint and the TokenEndpoint are handled automatically by the OAuth service in the web app. The UserInformationEndpoint is called explicitly in the OnCreatingTicket event. But before we get there we need to make sure that we SaveTokens and Map a JSON Key to something that we'll eventually get back when we call the UserInformationEndpoint.  Here we are mapping the user id and name Canvas.

That brings us on to the Events. There are several events that can be coded against including an OnRemoteFailure event. For simplicity's sake we've just used the OnCreatingTicket event which, as it's name suggests, occurs when Canvas has created a ticket and sent it back. 

In this event we set a new HttpRequestMessage variable to call the UserInformationEndpoint with a GET request. We need to add Headers to the request. The first tells the request to expect a JSON object. The second is the Access Token that Canvas has sent back to the web app for this user.

All that is left to do set a response variable to get the values back from Canvas for user information, we call the EnsureSuccessStatusCode to make sure we got a good response back, parse the JSON with user info and then run RunClaimActions to allocate name and id into the web app's authentication.

There is one other thing that we need to do on the startup.cs class. There is a public void Configure method in which we tell the app to use various tools and resources. In this file we need to add app.UseAuthentication() to tell the app to use Authentication. This call should come before the app.UseMVC() call.

So, now the app is set up to use OAuth with Canvas. We just need a situation to invoke it and show the outcome.

To do this we will create a LogIn action in a new Controller. So create a new Controller class in the Controllers folder and call it AccountController.cs. In this controller we will add a LogIn Action.

This Action will be called when the browser makes a get request to the Account/Login path. It returns a Challenge response which effectively kicks off the process of going to Canvas and authenticating which is what we just configured in startup.cs.

To call this Action I've added a link to the Shared/_Layout.cshtml file so that it appears on every page.

This basically renders as a link to the Login Action of the Account controller.

Now to see whether the user has successfully logged in and what their name is I've modified the Home/Index.cshtml file as follows: 

If the user is logged out the page will say "Not logged in". If the user is logged in the page will say "Logged in XXXX" where XXXX is the user's name in Canvas.

Step 4 - Test

Now when we run the application we get a plain looking standard web page but it does have a Log in with Canvas link and a statement saying we are not currently logged in.

When we click the Log In with Canvas link we get sent to the Canvas Log in page (assuming we are not already logged in to Canvas). 

The user is then asked to agree to authorize the calling web app. Note that the name, icon and other details are all configurable within the associated Canvas Developer key.

After which they are the taken back to the web app having been authenticated. 

Note that in this containerized instance of Canvas the default admin user has 'canvas@example.edu' set as their name which is why an email address is being shown. This would normally be their proper name in Canvas.

Summing up

If you are an ASP.NET Core developer looking to use OAuth with Canvas then this will, hopefully, have provided a starting point for you to get your own integrations working. It was a bit of struggle at times but half of that was returning to ASP.NET after some time away so there's been a fair bit of relearning done as well as quite a bit of new learning. I'm sure there are a heap of improvements that can be made. I'd love to hear suggestions.

A couple of days ago I decided to re-examine an issue that has annoyed me several times, the lack of a Question Bank API. The process began with some postings to followup the question raised by 10618071 in https://community.canvaslms.com/thread/14766-where-is-the-question-bank-api

This lead to some Tampermoney scripts (such as https://kth.instructure.com/courses/11/pages/using-tampermonkey-to-add-question-banks-v2) that enabled me to add question banks. This success lead to a desire for further automation and this lead me to investigate the combination of Puppeteer (https://pptr.dev/) and Node to run Javascripts to create a question bank- the first result was Using Puppeteer to insert new question bank: Chip sandbox. After seeing what Puppeteer could do I expanded upon the program to have a program that would not only let me add question banks but would also output a JSON file of the resulting set of question banks (which of course could then be used with the Canvas API to insert questions into question banks!). The resulting program is documented at https://kth.instructure.com/courses/11/pages/using-puppeteer-to-create-new-question-banks-and-get-js...

Some obvious programs that could be derived from this last script would be:

• to read a JSON file or CSV file that contains a list of question banks that one wants to create along with the course_id to create them in
• a program to simply output the JSON for the existing question banks in a course

Of course, the real solution is to add a Question bank API. Meanwhile, quite a lot can be done despite the lack of such an API.

Once again I wish to thank  @James  and the many others who have provided examples of JavaScript and Puppeteer scripts. I still do not know how to use Puppeteer (well) and lack confidence in navigating DOM structures (especially of pages that have elements that lack IDs or that dynamically modify the Javascript on a page).

I'm trying to make standards-based grading more approachable for my teachers. When I was teaching full time, I held to Frank Noschese's Keep It Simple philosophy. Single standards correlate to single assignments that are scored as pass/fail. Now, I averaged these out on a weighted scale to calculate a 0-100 grade, but that's for another post

Using Canvas, I was able to set up a functional reassessment strategy to aggregate demonstrations of proficiency.

The Learning Mastery Gradebook in Canvas does not translate anything into the traditional gradebook. This mean that every week or so, I would have to open the Mastery report alongside the traditional gradebook and update scores line by line. This was tedious and prone to error.

Using the Canvas API and a MySQL database, I put together a Python web app to do that work for me. The idea is that a single outcome in a Canvas course is linked with a single assignment to be scored as a 1 or 0 (pass/fail) when a mastery threshold is reached.

The App

Users are logged in via their existing Canvas account using the OAuth flow. There they are shown a list of active courses along with the number of students and how many Essential Standards are currently being assessed (ie, linked to an assignment).

Teacher Dashboard

Single Course

In the Course view, users select which grading category will be used for the standards. Outcomes are pulled in from the course and stored via their ID number. Assignments from the selected group are imported and added to the dropdown menu for each Outcome.

Users align Outcomes to the Assignment they want to be updated in Canvas when the scores are reconciled. This pulls live from Canvas, so the Outcomes and Assignments must exist prior to importing. As Assignments are aligned, they're added to the score report table.

Score Reports

Right now, it defaults to a 1 or 0 (pass/fail) if the Outcome score is greater than or equal to 3 (out of 4). All of the grade data is pulled at runtime - no student information is ever stored in the database. The Outcome/Assignment relationship that was created tells the app which assignment to update for which Outcome.

When scores are updated, the entire table is processed. The app pulls data via the API and compares the Outcome score with the Assignment grade. If an Outcome has risen above a 3, the associated Assignment is toggled to a 1. The same is true for the inverse: if an Outcome falls below a 3, the Assignment is toggled back to a 0.

I have mixed feelings about dropping a score, but the purpose of this little experiment is to make grade calculations and reconciliation between Outcomes and Assignments much more smooth for the teacher. It requires a user to run (no automatic updates) so grades can always be updated manually by the teacher in Canvas. Associations can also be removed at any time.

Improvements

To speed up processing, I use a Pool to run multiple checks at a time. It can process a class of ~30 students in under 10 seconds. I need to add some caching to make that even faster. This does not split students into sections, either. 

I've started turning this into an LTI capable app which would make it even easier for teachers to jump in. If you're a Python developer, I would really appreciate some code review. There is definitely some cleanup to be done in the functions and documentation and any insight on the logic would be great.

The source for the project is on GitHub.

 @James ‌ suggested that I should file a blog post about my recent findings/results.

Background

I just recently started with Canvas because Uppsala University has decided to use it as its upcoming LMS platform after a failed attempt with another product. Therefore I had already spent some time with Blackboard and was quite fond of the calculated questions type in quizzes. I quickly found out that Canvas offers essentially the same functionality but a bit less comfortable.

Problem

A calculated question or Formula Question as it is called in the interface of Canvas is based on a table of pre-generated variable values and corresponding results. In the general case the variables are defined and the target function is entered using the web interface, then Canvas calculates random number values for the variables and the resulting answer value. However, as the designer you have no possibility to influence the variable values afterwards (unlike in Blackboard where you have a spreadsheet-like interface). Also, in Canvas, the equation cannot be altered once it has been entered - and the supported syntax is not very convenient for more complex problems.
I was also missing the ability to give a relative tolerance for the correct answers in a question, however, I found out that entering a percentage-sign exactly gives this behavior even though it does not seem documented anywhere.

Solution or problems?

My hope was then for the API, since it seemed to support the creation of questions. But even though there is a Python library for the purpose of controlling Canvas, many of the functions are not very well documented. My first tries failed miserably but finally I was on the right track.

The cause of my problems was that the Canvas API uses different field identifiers and structures when creating a calculated question as when you retrieve the contents of an already existing question, as I of course did in my attempts to reverse-engineer the interface.

Working solution

Here is now an example for a working solution to give you full control over the generation of Formula Qeustions using Python and the canvasapi library. The example is in Python 3 and creates a question from the field of electronics - the voltage in a voltage divider. The Python script generates the variables, fills the variables with random numbers from a set of predefined, commonly used values. I tried to write the script more for readability than any pythonic optimization.

from canvasapi import Canvas
import itertools
import random

API_URL = "https://canvas.instructure.com"
API_KEY = <your api key here>

canvas = Canvas(API_URL, API_KEY)

# create a calculated_question
# example of a potential divider
#
#  U2 = U0 * R2 / ( R1 + R2 )
#

E3  = [1, 2, 5]
E6  = [1.0, 1.5, 2.2, 3.3, 4.7, 6.8]
E12 = [1.0, 1.2, 1.5, 1.8, 2.2, 2.7, 3.3, 3.9, 4.7, 5.6, 6.8, 8.2]

coursename = 'test'
quizname   = 'test'

# define the input variable names
#   each variable has its own range, format and scale
#   
variables = \
    [
      {
        'name':   'U0',
        'unit':   'V',
        'format': '{:.1f}',
        'scale':  '1',
        'range':  [1.2, 1.5, 4.5, 9, 12, 24, 48, 110, 220]
      },
      {
        'name':   'R1',
        'unit':   'ohm',
        'format': '{:.1f}',
        'scale':  '1',
        'range':  [ i*j for i, j in itertools.product([10, 100, 1000], E12)]
      },
      {
        'name':   'R2',
        'unit':   'ohm',
        'format': '{:.1f}',
        'scale':  '1',
        'range':  [ i*j for i, j in itertools.product([10, 100, 1000], E12)]
      },
    ]

# how many sets of answers
rows = 30

# create an empty list of lists (array) for the values
values = [ [ i for i in range(len(variables))] for _ in range(rows)]

# create an empty list for the calculated results
results = [i for i in range(rows)]

# fill the array of input values with random choices from the given ranges
for i in range(rows):
    for j in range(len(variables)):
        values[i][j] = random.choice(variables[j].get('range'))

    # and calculate the result value    
    results[i] = values[i][0] * values[i][2] / (values[i][1]+values[i][2])

# format the text field for the question
#   an HTML table is created which presents the variables and their values
question_text = '<p><table border="1"><tr><th></th><th>value</th><th>unit</th></tr>';
for j in range(len(variables)):
    question_text += '<tr>'
    question_text += '<td style="text-align:center;">' + variables[j].get('name') + '</td>'
    question_text += '<td style="text-align:right;">[' + variables[j].get('name') + ']</td>'
    question_text += '<td style="text-align:center;">' + variables[j].get('unit') + '</td>'
    question_text += '</tr>'
question_text += '</table></p>'

# format the central block of values and results
answers = []
for i in range(rows):
    answers.append(\
        {
          'weight': '100',
          'variables':
          [
            {
              'name': variables[j].get('name'),
              'value': variables[j].get('format').format(values[i][j])
            } for j in range(len(variables))
          ],
          'answer_text': '{:.5g}'.format(results[i])
        })

# format the block of variables,
#   'min' and 'max' do not matter since the values are created inside the script
#   'scale' determines the decimal places during output  
variables_block = []
for j in range(len(variables)):
    variables_block.append(\
        {
          'name':  variables[j].get('name'),
          'min':   '1.0',
          'max':   '10.0',
          'scale': variables[j].get('scale')
        })

# put together the structure of the question
new_question = \
    {
      'question_name':           'Question 6',
      'question_type':           'calculated_question',
      'question_text':           question_text,
      'points_possible':         '1.0',
      'correct_comments':        '',
      'incorrect_comments':      '',
      'neutral_comments':        '',
      'correct_comments_html':   '',
      'incorrect_comments_html': '',
      'neutral_comments_html':   '',
      'answers':                 answers,
      'variables':               variables_block,
      'formulas':                ['automated by python'],
      'answer_tolerance':        '5%',
      'formula_decimal_places':  '1',
      'matches':                 None,
      'matching_answer_incorrect_matches': None,
    }
                                  

courses  = canvas.get_courses()
for course in courses:
    if course.name.lower() == coursename.lower():
        print('found course')
        quizzes = course.get_quizzes()
        for quiz in quizzes:
            if quiz.title.lower() == quizname.lower():
                print('found quiz')

                question = quiz.create_question(question = new_question)       
        
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Since this is mostly the result of successful reverse engineering and not based on the actual source code of Canvas the above example should perhaps be used with care, but for me it is what I needed to create usable questions for my students. Perhaps this could also serve the developers as an example on how the interface for calculated questions could be improved in the future.

How does it work?

The dictionary variables (lines 26-49) contains the names and ranges of the variables, as well as formatting instructions. The ranges are given as lists. In lines 61-66 the random values are generated and the results calculated from these values. Lines 70-77 create a rudimentary table to be included in the question text containing the variables and their values as well as physical units for this particular question. Lines 80-93 finally assemble the variable/answer block and lines 109-128 put everything together into the dictionary to create a new question.

The script then inserts the question into an existing quiz in an existing course in line 140.

After running the script

This screenshot shows the inserted question after running the script, obviously this would need some more cosmetics.

And when editing the question this is what you see:

Be careful not to touch the variables or the formula section since this will reset the table values.

Cosmetics

In order to be presentable to the students the above questions needs some cosmetics. What is to be calculated? Perhaps insert a picture or an equation? More text?

After updating the question and leaving the editor it now looks like this in the Canvas UI:

Seeing and answering the question

When you now start the quiz, this is how the question looks:

Summary

• calculated_questions can be generated using the Python canvasapi library
• answer values have to be provided with the key 'answer-text'
  

  'answers': [
     {
       'weight': '100', 
       'variables': [
       {'name': 'U0', 'value': '9.0'}, 
       {'name': 'R1', 'value': '5600.0'}, 
       {'name': 'R2', 'value': '5600.0'}], 
       'answer_text': '4.5'}, ‍‍‍‍‍‍‍‍

• when querying an existing calculated_question through the API the answer values are found with the key 'answer'
  

  answers=[
      {'weight': 100, 
       'variables': [
        {'name': 'U0', 'value': '110.0'}, 
        {'name': 'R1', 'value': '82.0'}, 
        {'name': 'R2', 'value': '8200.0'}], 
       'answer': 108.91, 
       'id': 3863}, ‍‍‍‍‍‍‍‍

• when supplying an equation for the 'formular' field this has to be done in a list, not a dictionary
  

   'formulas':  ['a*b'],‍‍

• when querying an existing calculated_question through the API the equations are found in a dictionary like this:
  

   formulas=[{'formula': 'a*b'}],‍‍

When poking the Canvas API with curl I would often find myself copy and pasting lots of Authorization headers around to correctly authenticate against Canvas. This is error prone and also leaves your tokens accessible in your .bash_history file. To improve on this I wrote a small script for macOS that stores the tokens in Apple's keychain and then automatically adds the Authorization header to the curl request based on the URL in the command line. While this isn't perfect, it's much better and easier to use. The result is a script I call ccurl. 

Setup

Download a copy of ccurl, make it executable and set the CCURL_HOSTS environmental variable to a space separated list of hosts you wish to use ccurl against. 

$ curl -s -O https://gist.githubusercontent.com/buckett/ed3217fced4b9d129758157b4476aaa6/raw/1fa77f31bdb65b8bf6cb... chmod +x ccurl
$ echo 'export CCURL_HOSTS="canvas.instructure.com canvas.test.instructure.com canvas.beta.instructure.com"' >> ~/.bashrc‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

You may also wish to put ccurl somewhere on your PATH. Then to set a token for a host use (history -c flushes bash history so the token doesn't get save in plain sight):

$ security add-generic-password -a $USER -s canvas.instructure.com -w 7~1K9WJ3xobQp5RX8DUbbSdigxn2WD8yMOfUlCHbH9FIPlyL7E9E5QWSWN4CCVfqAEHC
$ history -c‍‍‍‍‍‍‍‍

Use

Then to use it just do a curl command but add the extra c, it passes all command line options through to curl so it should support all examples you see for the standard curl tool (jq is a tool to transform json, but here it just formats it to make it more readable):

$ ccurl -s  https://canvas.instructure.com/api/v1/users/self | jq .
{
  "id": 4539009,
  "name": "Matthew Buckett",
  "created_at": "2015-05-31T19:49:29+01:00",
  "sortable_name": "Buckett, Matthew",
  "short_name": "Matthew Buckett",
  "avatar_url": "https://canvas.instructure.com/images/messages/avatar-50.png",
  "locale": null,
  "effective_locale": "en-GB",
  "permissions": {
    "can_update_name": true,
    "can_update_avatar": true
  }
}
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Links

https://canvas.instructure.com/doc/api/file.oauth.html#manual-token-generation - How to create a token for your account.

I was asked to create a script to sort courses on the Account Details page, as it can be difficult to find a course for the most current term as we don't hide previous terms from the list. It adds two sort buttons – Sort by Name (which is the default sort) and Sort by Term.

To Install:

1. Make sure you have Tampermonkey installed in Chrome

https://chrome.google.com/webstore/detail/tampermonkey/dhdgffkkebhmkfjojejmpbldmpobfkfo?hl=en

1. Click on the link (or copy/paste) and open in Chrome.

https://github.com/sukotsuchido/CanvasUserScripts/raw/master/Canvas%20User%20Profile%20Course%20Sort...

1. An Install page for Tampermonkey will open, click the “Install” button.

To Use:

1. Navigate to any User Account Details Page
   • It will work equally well on student or teacher pages.
2. Click “Sort by Term” to sort in descending order (most recent first), click again to sort in ascending order (oldest or manually created first)

I created a new userscript for removing students from courses in bulk. This will be particularly useful when managing clubs/activity and remediation manually created courses that aren’t set to conclude. For instance, I can see us using this heavily to manage NHS/NJHS, Class of "2018" courses, LOTE clubs, and anything else that has rolling members.

To setup the script –

1. Install Tampermonkey (if you don’t have it already)
2. Install the script by going to the following link
   1. Click the “install” button that pops up in Tampermonkey
3. Change the "@include" on line 7 to match your institutions Canvas URL setup

   // @include https://*.instructure.com/courses/*/users‍‍

4. It is currently setup to only be visible to admin as our teachers don't have enrollment privileges
   1. If you are a teacher with enrollment privileges (you can add/remove people), then change Line 22 to include "teacher" as a role.
      

      var buttonRoles = ["teacher", "admin", "root_admin"];‍‍‍

To use the script –

1. Open Chrome
2. Navigate to the people tab in a course and it should pop up near the top.
   

Features –

1. Lists student name, section, enrollment date, and last activity date in a sortable table.
   
   
2. Select All checkbox or select individual students
   
   

Limitations –

1. This currently only works for students. I figure we don’t usually have more than a handful of teachers in a course, so that can usually be managed by hand.

Let me know what you think and if have feature ideas or improvements, let me know!

*EDIT 2/16/22 - Added pagination to load all students at once and async chunking of removal requests to prevent rate limitation errors. I also added loading icons to make user aware script is running.

*EDIT 4/22/19 - Added Section column*

Cheers,

Chad

Having become incredibly frustrated at the frequency my professors added and updated their course files requiring me to download and move files to wherever I had organized them before, I built an app that handles syncing Canvas files to student's computers. 

The app is open source using electron js GitHub - drew-royster/canvasFileSync: syncs files from canvas to your local machine to ensure you're... 

Any thoughts on how to improve the app or if you could just spread the word I'd appreciate it.

I've been working on an easier way for teachers to upload Quiz Items and Outcomes from a Google Sheet. The template sheet uses Google Apps Script and a user's personal API key to interact with their courses.

Quizzes

Teachers frequently write tests and quizzes together. This mechanism allows teachers to write and upload dozens of questions easily from a Google Sheet.

The Canvas API only allows questions to be uploaded to an existing Quiz resource. Teachers can select the appropriate course and quiz dynamically using the popup.

Once a quiz resource has been selected, users can define the number of items to upload. The spreadsheet marks an item as successful and updates the next row to be uploaded so questions are not duplicated on subsequent runs.

Considerations

• All questions uploaded are stored in the "Unnamed bank" question bank.
• The "Topic" and "Primary Standard Indicator" fields are used to title questions for easy bulk sorting in the Canvas UI.

Updates to come

• Error messages are...opaque. Often a failure results in "the resource does not exist," meaning the user hasn't selected a course or quiz.
• Questions are not batched right now, so uploading too many questions can lead to a script timeout.

Outcomes

The Outcomes API is difficult to use. I know there are CSV imports now, but it is sometimes helpful to upload Outcomes as they're written rather than generating a CSV over and over.

Outcomes have to go into an Outcome Group. If there are no groups defined in the course, the course name is used as a top-level group.

The template spreadsheet is similar to the quiz template, with specific fields being defined for the upload to run successfully. There is a helper sheet where users can define different rubric scoring ranges for each upload.

The Outcome upload sidebar is similar to the Quiz Item sidebar. Because new outcomes must be attached to a group, a dynamic list of codes and group titles are generated when the user selects their course.

Considerations

• Outcome uploads are done to the course, not to the user's account. This allows them to bulk-modify outcomes semester to semester as they update their courses.

Updates to come

• Improve error messaging
• Pull all existing outcomes and their status into a sheet for proofing and updating.

I have a template project you can copy. You can also see the entire source on GitHub if you'd like to take a look or contribute. My plans are to eventually convert this to an AddOn for easier distribution and use.

EDIT -  a bit of retrospectively added blurb about this script:

If you didn't know, Canvas APIs are fantastic. To really leverage the API, you need to learn at least a little bit of coding so you can run a script that will ultimately help you to achieve something that would otherwise have taken an awful lot of arduous clicking around in Canvas.

So this is my first script. When it runs it makes three different GET calls to retrieve data about a particular assignment (you define which assignment - look at the code comments in green). The 3 calls are to (1) retrieve the comments left for a submission, (2) get a list of the teachers on the course, and (3) get a list of the students on the course. The script then uses the response data from these calls to work out which students received comments on their submission from more than one teacher. Our assessment regulations require 10% or more of the submissions to be treated this way, so the script ultimately outputs (1) how many students submitted, (2) how many received a comment from more than one teacher, (3) whether the 10% rule was met or not, and (4) the list of student SIS Ids who did receive this.

As you read through the code you'll spot other things going on to do with how the response data gets manipulated to achieve the end result. If you look at this and think 'this is terrifying nonsense and sod it I'm no coder so never mind, APIs aren't for me' - STOP. Don't worry - I went from that starting point to this script in about 3 weeks (and that was playing around outside of work time!). I can attribute my success here to this self-paced course built specifically for learning ruby (a coding language) with the Canvas APIs:

Getting Started with Ruby & the Canvas API . Go join it. Make awesome things. Just remember to give kudos to Jeremy for building that course for us all!

(P.S my original comment is still below the code)

Problem Statement: 

If an instructor wants to create the quizzes for his course, he need follow a set of steps and need to to do it manually. If an instructor has set of rules designed for the quizzes, he/she can replicate the same quizzes for other courses.

Proposed Solution:

Need to design a script which first fetches all the courses available. Once we have a list of courses, will ask for specific course for which an instructor is interested to create a quiz. After making the selection for a respective course, other part scripts will create a quiz with the predefined set of rules.

Script Description:

• First, take the user's input such as course name and the quiz details like quiz name, description, type, the time limit
• Fetch all the courses belongs to the respective instructor with the help of  course API (<canvas_url>/api/v1/accounts/self/course)
• Filter the courses based on the user provided course name
• If we found the courses for the user's specified requirement make a call to create the quizzes
• It will take the quiz details and make a post call to create the quizzes (/api/v1/courses/:course_id/quizzes)
• Notify based on the response to the user.

Code Snippet:

#Libraries to be import
require 'typhoeus'
require 'link_header'
require 'json'

canvas_url = '' # put full canvas test url eg: https://school.test.instructure.com
canvas_token = '' # put canvas API token here
api_endpoint = '/api/v1/accounts/self/courses'

#User inputs to create the quiz
course_name = "GENERAL ZOOLOGY LAB"
quizz_detail = {
        "title" => "Test Quizz3",
        "description" => "Test Desc3",
        "type" => "practice_quiz",
        "time_limit" => 160
}


#Function to create the quizzes
def createQuizzes(canvas_url,canvas_token,course,quizz_detail)
    
    api_endpoint2 = 'api/v1/courses'
    id = course["id"]
    request_url2 = "#{canvas_url}#{api_endpoint2}/#{id}/quizzes"
    
    create_quizz = Typhoeus::Request.new(
        request_url2,    #we need a variable here because we need the api url to change
        method: :post,
        headers: { authorization: "Bearer #{canvas_token}"},
        params: {
            "quiz[title]" => quizz_detail["title"],
            "quiz[description]" => quizz_detail["description"],
            "quiz[quiz_type]" => quizz_detail["type"],
            "quiz[time_limit]" => quizz_detail["time_limit"] 
        })

    create_quizz.on_complete do |response|
        if response.code == 200
           puts "Quizz created successfully for Course id #{id}"
        else
            puts "Something went wrong! Response code was #{response.code}"
        end
    end

    create_quizz.run
end
#End create quizz function


#Variables initialisation
request_url = "#{canvas_url}#{api_endpoint}"
count = 0
more_data = true
course_found = false

while more_data   # while more_data is true to keep looping through the data

    #puts request_url #helps demonstrate pagination
    get_courses = Typhoeus::Request.new(
        request_url,    #we need a variable here because we need the api url to change
        method: :get,
        headers: { authorization: "Bearer #{canvas_token}" }
        )
    get_courses.on_complete do |response|
        #get next link
            links = LinkHeader.parse(response.headers['link']).links
            next_link = links.find { |link| link['rel'] == 'next' } 
            request_url = next_link.href if next_link 
            if next_link && "#{response.body}" != "[]"
                more_data = true
            else
                more_data = false
            end
        #ends next link code
        if response.code == 200
            data = JSON.parse(response.body)
            data.each do |courses|
                count += 1
                if courses['name'] == course_name
                    more_data = false
                    course_found = true
                    createQuizzes(canvas_url,canvas_token,courses,quizz_detail)
                end
            end
        else
            puts "Something went wrong! Response code was #{response.code}"
        end
    end
    get_courses.run
end

#If no course for user provided input 
if !course_found
    puts "Course not found"
end

puts "Script done running"

‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

References:

Courses - Canvas LMS REST API Documentation 

Quizzes - Canvas LMS REST API Documentation 

TL:DR; There is a free self-paced course on Canvas Network that can get a non-programmer to write scripts with Ruby & the Canvas API...

Calling all Canvas admins who have ever been frustrated by an Instructure employee telling you, "It can be done through the API." The barrier to entry for using the Canvas API and programming in general can seem very high. Luckily, it's easier to learn than you think. 

As an Implementation Consultant, I understand how frustrating it is to be limited in what you can do through the UI alone. I've spent the last 18 months as an onsite resource for a Canvas client, and have needed to leverage the Canvas API more times than I can remember. Whether it is to pull information, or push a default grading scheme to courses, there are some things that are just easier through the API.

I've spent a very long time these past few months building curriculum and recording videos as part of the instructional unit. This course is being offered by me, and is not to be considered an official Instructure course. If the engineers saw my code they might have an aneurism because it is pretty far from beautiful. However, the intentional design of the course is to help Canvas Administrators, who like me once, thought that writing a script and running it on your own computer was a task for only the smartest of people.

The course is divided into 5 instructional units, with over 2 hours of recorded video and exercises to accompany them. The course is fully self-paced and does not have a close date. Hope you enjoy!

Sign up for the Ruby & the Canvas API 

Overview

One of the most common questions I'm asked when consulting our Canvas partners is how to synchronize the roster from Canvas into an external application. There are a variety of ways this can be accomplished, each with their advantages and disadvantages depending on the needs of your product and customers. This post is meant to be a resource for external tool providers hoping to maintain a synchronized roster between their systems and Canvas.

LTI-only Workflow

Requirements

This approach requires an LTI integration to be configured and visible somewhere within a Canvas course. Ideally, this LTI connection will already have an LTI SSO mechanism. If username, login ID, email, and/or SIS ID is required, make sure the privacy level is set to Public in the tool configuration. Otherwise, Canvas will only send an opaque LTI user id (as the user_id parameter) and a Canvas ID (as the custom_canvas_user_id).

Advantages

• API access not required
• Interoperable
• Can provision users on-the-fly as they launch the tool

Limitations/Challenges

• LTI tool is only aware of users who've launched their tool at least once
• Unidirectional: cannot push new enrollments to Canvas
• Cannot determine if users drop courses or are deleted from Canvas

Instructor/Admin/Student Workflow

1. Configure an LTI tool in Canvas
2. Launch the tool
3. Tool consumes user information (name, email, ID's, roles, contextual information etc...) and attempts to match on an ID. Best practice is to match on the user_id from the launch and then fall back to some other ID if a match is not found
4. If a match is confirmed (and the signature matches), let the user access their information in your application
5. If no match is found, either or send them through a user-creation flow within the iframe, or auto-create a user for them based on the information in Canvas (you may want to let them set a password at this point, or email them a registration URL)

LTI + API Workflow

The main limitation of the LTI-only workflow is that it requires users to launch the tool through the LMS. This often does not make sense. For example, some tools are used only by instructors, so there is no opportunity for the students to launch an LTI tool. The LTI + API workflow works best for tools targeting teachers or students, but is limited when synchronizing rosters for large accounts.

Requirements

This approach requires an LTI integration to be configured. It also must use OAuth2 to obtain instructor and student API tokens.

Advantages

• Can provision entire courses or sections with a teacher token, or entire accounts with an admin token
• Can remove/hide users that have dropped courses or have been deleted
• Bi-directional: if authorizing user has permissions, can create/update/delete enrollments within Canvas

Limitations/Challenges

• Requires implementation of OAuth2
• Requires heavy usage of Canvas-specific API’s making it not interoperable
• If attempting to sync entire accounts, can be slow for large accounts due to API throttling

Admin Workflow

1. Admin launches the tool from account navigation or elsewhere
2. Tool kicks off OAuth2 if it needs to obtain or refresh a token
3. Tool retrieves a list of courses via the accounts API, then uses the Enrollments API for each course
4. Tool can then create/update/delete users from their database based on the results

NOTE: This approach is not recommended for large accounts due to API throttling. Instead, either use the LTI + SIS + API approach in the next section or see the Instructor Workflow below.

Instructor Workflow

1. Instructor launches the tool from course navigation or elsewhere
2. Tool kicks off OAuth2 if it needs to obtain or refresh a token
3. Tool consumes the custom_canvas_api_domain and custom_canvas_course_id from the LTI launch and checks the Enrollments API for the course. Alternatively, the tool can check the enrollments for the instructor to sync all courses they are enrolled in; this requires consuming the custom_canvas_user_id
4. Tool can then create/update/delete users from their database based on the results

Student Workflow

1. Student launches the tool from the course
2. Tool kicks off OAuth2 if it needs to obtain or refresh a token
3. Tool consumes the custom_canvas_course_id from the LTI launch and checks the Enrollments API for the course

NOTE: Some students may not have permission to view the list of users in their course.

LTI + Reports API Workflow

The main limitation of the LTI + API workflow is that it doesn’t work well for large accounts. By using the Reports API, tools can generate reports that list all users in an account. This is best used for institutions who’ve implemented SIS, but it’s not a requirement.

Advantages

• Can provision entire accounts without worry of being throttled
• Can remove/hide users who have dropped courses or have been deleted
• Bi-directional: if authorizing user has permissions, can create/update/delete enrollments within Canvas via SIS Imports API

Limitations/Challenges

• Requires an admin-level token
• Best if institution has implemented SIS; SIS exports only show resources with SIS ID's. If SIS ID's are absent, you'll need to understand how to interpret the provisioning report
• Requires implementation of OAuth2
• Requires heavy usage of Canvas-specific API’s making it not interoperable
• Requires parsing CSV's and downloading files
• Requires understanding the structure of SIS CSV files or the provisioning report

Admin Workflow

1. Admin launches the tool from account navigation
2. Tool kicks off OAuth2 if it needs to obtain or refresh a token
3. Tool starts a report via API

NOTE: There are two reports of interest here. The easiest to interpret is the SIS Export report; however, the SIS Export report only shows users with SIS ID's. In this scenario, you’ll need to rely on the provisioning report instead.

1. Tool polls the report to check progress
2. When the report is complete, tool downloads the report and parses the data to synchronize users
3. Tool can then create/update/delete users from their database based on the results

Instructor/Student Workflow

Instructors and students don’t have access to the reports API. However, the same functionality of the LTI + API integration described in previous sections can still be used to compliment the LTI + Reports API workflow. For example, you may want to do a nightly sync at the account level, but still use API + LTI to synchronize data when a user launches the tool.

NOTE: Admins can also manually send reports to a tool provider for one-time provisioning of new customers; however, in order to maintain an accurate list of users and enrollments, the tool would still need to sync data via LTI and/or API.

NOTE: The workflows which leverage API can be further automated or simplified by obtaining an admin token with the “become other users” permission. This token can then be used to make API requests on behalf of teachers and students by appending as_user_id=<student/teacher user id>. However, this can be problematic for high numbers of assignments or students as you’ll likely experience throttling issues.

SIS-only Integrations

This won’t be discussed here. For more information about integrating with Canvas as a SIS provider, please contact: Chase Pendleton <cpendleton@instructure.com> (SIS Partnerships Manager)

Roadmap: LTI Names and Role Provisioning Services (Membership Service)

We’re currently working toward implementation of the IMS LTI Names and Role Provisioning Service. We hope to have most, if not all, of the service implemented by the end of 2018. This service will make the LTI-only workflow more useful for instructors in particular as it doesn’t require all users in a course to launch the tool, and it'll allow tools to be aware of changes in the course roster.

Requirements

The requirements are the same as the LTI-only workflow outlined above, but you must also conform to the IMS LTI Names and Role provisioning Service as a tool provider.

Advantages

• Does not require API access
• Can provision an entire course on-the-fly when an instructor or admin launches a tool from the course
• Interoperable

Limitations/Challenges

• Will not be able to synchronize an entire account
• Launch must occur in the context of a course
• Unidirectional: cannot push new enrollments to Canvas

Workflow

• User launches tool from course
• Canvas sends a service URL that a tool can use to see a full list of users for the course
• Tool can then create/update/delete users from their database based on the results

Concluding Remarks

This isn’t a comprehensive guide to solving the challenge of synchronizing users and enrollments between systems. I encourage creativity and product discovery with your customers to see how they envision your product working within Canvas. My hope is that this guide has at least allowed you to gain a better understanding of what is and isn’t possible so your organization can make decisions that help our mutual users have the greatest experience possible.

For those who’ve already solved this problem, how has your integration combined API and LTI to provision users from Canvas? Are there any interesting approaches you've taken that I haven’t considered here? Please share in the comments!


Jesse Poulos
Partner Integrations Specialist
Instructure

Print Canvas Quizzes is a script that will allow a user to print a quiz from the preview page.

1. Features
   1. Adds a "Print Quiz" button below the question navigation pane
   2. Auto-page break: This will keep all question content on the same page and prevents a page break in the middle of a question
   3. The page is set to zoom to 74% to make it sized appropriately for printing
   4. Hides "This is a preview..." banner
   5. The print dialog will automatically pop-up for the user
   6. Adjusts certain question types for legibility and space efficiency 
      • Multiple Choice: Left aligns choices, all one column
      • Matching: Removes drop-down menu and creates an "answer bank" at the bottom of the question box
      • Multiple Dropdowns: Expands the dropdowns to width and height of content
2. Limitations
   1. The quiz must be viewed from the "Preview Quiz" page
   2. All questions must be visible on the page, which means the "Show one question at a time" must be unchecked
   3. Currently, the zoom level of the page is not editable by the user, except through the printer dialog window
   4. Not usable in the Quizzez.next LTI

If you want to print canvas quizzes, do the following –

Install tampermonkey chrome extension

https://chrome.google.com/webstore/detail/tampermonkey/dhdgffkkebhmkfjojejmpbldmpobfkfo?hl=en

Then copy/paste or open this link in Chrome
https://github.com/sukotsuchido/CanvasUserScripts/raw/master/printCanvasQuizzes.user.js

A tampermonkey install page will pop up, click “install”

To Print – make sure “show one question at a time” is Unchecked in quiz settings

Click “Preview” on the quiz page

On the right side, there is now a “Print Quiz” button

If you want to deploy to all staff, copy the script into the custom Javascript folder.

As always, I welcome feedback and improvements. This is something our teachers were absolutely demanding and this was a rudimentary attempt at giving them something usable.

Enjoy!

 @G_Petruzella ‌ has created a bookmarklet that also allows for printing quizzes. It was his post that reminded me I needed to share this out!

With the depreciation of Classic Quizzes in 2024, The Community Team would like to encourage Community Members to explore New Quizzes! Printing is included in the New Quizzes experience: How do I print a quiz in New Quizzes? 

As Canvas admins we have a need to track external tool usage, across all course shells. We often are very interested to know adoption of a tool at our institution. Or if we determine that a particular tool has a technical problem, we need to be able to find which teachers to contact. Unfortunately neither the LTI Tool Report nor External Tool API are suited for this task, in themselves. Both are only effective at finding tool installations either at tool or account contexts. What is needed is a way to find when an external tool is enabled in a course navigation, used in an assignment or module. Fortunately, we can gather this information from the Assignments API, Modules API, and Tabs API and weave the external tool info with course and teacher info gathered from the Courses API.  

In this post, (and in perhaps subsequent posts) I will describe my approach, which is developed in Python. I am eager to receive input from others to determine how my code can be improved. My Python code uses a perhaps idiosyncratic style, choosing to wrap api calls in Python generators with closures, rather than using objects and classes. It seems that this more lightweight approach is well suited for getting data out of Canvas APIs and weaving it together into reports.  Again, I am interested in having a discussion on this - there may be a better way than what I am doing. Folks familiar with Clojure and the videos lectures of Richard Hickey will guess my motivations -  ultimately I  am trying to make something complex simple. 

The heart of the report building process looks like this:

# ----Iterate over all courses ---------------------------
for c in all_courses():
    # Build report in a Python default dictionary, where the default type is list,
    # allows appending without checking key first
    xtool_data_for_course = defaultdict(list)
    # ---- And for each course, retrieve all installed x-tools, tabs, modules, and assignments
    # ---------XTOOL INSTALLS-----------
    for x in course_xtools(c['id']):
        xtool_data_for_course['XTOOL_INSTALLS'].append(x)

    # ---------TABS--------------------
    for t in course_tabs(c['id']):
        xtool_data_for_course['TABS'].append(t)

    # ---------MODULES---------------------
    for m in course_modules(c['id']):
        for i in module_items(direct_url=m['items_url']):
            xtool_data_for_course['MODULES'].append(i)

    # ---------Assignments---------------------
    for a in course_assignments(c['id']):
        xtool_data_for_course['ASSIGNMENTS'].append(a)

Each of the for loops is iterating over a Python generator. The generator is "produced" by the api_wrapper_function, which "encloses" all the parameters specific to a particular endpoint (URL, authentication, and options) and typically receive an ID that specifies which course to query. The generator also may have a filter predicate specified, so only  active, external tools are returned. Another module specifies the defining parameters for the generators, so our client code here does not need to sweat the details.

The data-structure produced is a list of Python dictionaries for each course that is found to have at least one external tool in-place. 

[{COURSE_INFO:{course_code, name, id},
     INSTRUCTOR_INFO: [{instructor INFO}, {}..],
     XTOOL_INSTALLS:[ {xtool tool info}, {},...],
     TABS: [ {tab info} ... ],
     MODULES: [ {module item info}, ...],
     ASSIGNMENTS: [ {assignment xtool info}]

Each record for a course will have COURSE_INFO and INSTRUCTOR INFO and may have XTOOLS, TABS, MODULES, ASSIGNMENTS.  This data-structure is processed into a CSV report by another script. Right now the process works, with a few problems I am still working on. As you may guess, a process that inspects each and every course with multiple (four)  endpoint calls is going to be slow. May look at running multiple threaded agents that split up the list and then aggregate results at the end. Another challenge is surfacing the right information about external tools in course sites, and excluding less critical info. For example, my current report shows up with bunch of redirects. In most cases that is not what we are interested in. Ok, that is it for now. If folks are interested will the underlying functions/generators that make this work. Bit of code-clean up to do first! Would like to thank Peter Love for the advice he offered and help tracking down all the endpoints needed to generate this report. Finally do let me know if there are better ways of achieving this same ends. Let me down easy... but do let me know, thanks!

References:

Assignments API (Assignments - Canvas LMS REST API Documentation)  

Modules API (Modules - Canvas LMS REST API Documentation ) 

Tabs API (Tabs - Canvas LMS REST API Documentation )

Python Generators - Generators - Python Wiki 

Kansas State University is proud to announce our first open source LTI application. It is an attendance taking tool. The source code is available on GitHub.

Previously we released a Java library to interface with the Canvas API. (blog post | code) This API library is now being used by a couple other universities and we have received several pull requests to add more functionality. Thank you to the Canvas community for embracing open source software and contributing back! We hope that this will continue.

Now, as part of the attendance taking application, there are a few other bits we would like to highlight. All of our LTI applications are based on an LTI launch framework to handle common launch tasks and provide common resources. We hope that others may find it useful in implementing their own LTI applications. In order to assist in learning how to use it, we have created a minimal self-contained LTI application that shows how to use the launch framework. Here are some details on these parts:

Attendance LTI application

This application was developed before Instructure released the Roll Call application as open source so it may not be quite as valuable now but it may still be worth a look if there is functionality you are missing in the existing Canvas experience.

Initially, we developed this application for the K-State Polytechnic Campus in Salina, Kanas. Their aviation maintenance program has strict guidelines from the FAA, which requires them to track contact time between students and instructors down to the minute. Later, we extended the application to be more broadly useful to the campus community.

Some of the features of the application include:

• Minute versus daily attendance tracking, configurable by the instructor on a per-course basis
• Instructors can choose to allow or prohibit students from seeing details of their attendance records
• Attendance can be pushed to the Canvas grade book as an assignment

There is still some room for improvement to make this application more useful to other institutions. Depending on your existing environment, here are some specific hurdles you may face when trying to deploy this application:

• The application was written against an Oracle database. It should not take too much effort to make the application database agnostic since we use Hibernate. We may get around to this at some point to allow for better testing.
• We tried to avoid using K-State specific terms and branding but some did slip in. For example our SIS ID is commonly called the "Wildcat ID" and shows up as "WID" in some parts of the application.
• The deployment process is not exactly simple. It would be great to have a docker image that packages the application and container together for easy deployment. Currently it runs on Wildfly. There is an INSTALL.md file in the repository that details every step needed to install the application in a UNIX-like environment.

LTI Launch Framework

This framework is based on Spring and handles LTI launch requests. (source code: GitHub) Spring Security is used to handle verifying the OAuth signature that comes with the LTI launch request. It also assists your application in getting an API access token from the user via the OAuth2 flow.

Because this framework is based on Spring, you must implement some interfaces and make the implementations available as Spring beans. These beans handle persisting user OAuth tokens for subsequent application launches, generally setting up the LTI application, and ensuring that the application is communicating with the correct Canvas instance. The last part is vital for hosted Canvas customers because it prevents a test version of the application from accidentally interacting with a production instance of Canvas. This mixup can readily occur when the test environment gets automatically overwritten with production settings periodically.

We have run into challenges as it relates to error handling in Spring. At the moment it is left mostly up to your application to handle specific errors correctly. This leads to some code duplication for handling common LTI related error conditions. As time permits, we are working to improve this problem.

Minimal LTI application

We created a minimal LTI application to demonstrate how to use the LTI Launch Framework. (source code: GitHub) The application outputs a simple page of text. It only authenticates the LTI launch request and then echoes back information including the username and LTI parameters being used in the application.

Unlike the Attendance application, this one is trivial to deploy! It uses maven to build an executable JAR which contains an embedded Tomcat server. Running it as as simple as executing the command: java -jar lti-launch-example.jar

Comments (and pull requests) are welcome!

This is a follow up to my previous blog post, "A Simple Python GET Script". Personally, I find myself using POST requests quite frequently. So while the GET script was great for showing at a basic level how Python can interact with your Canvas instance, this blog post can hopefully show why this is such powerful tool. 

Creating a New User

I'd like to start off by saying that Canvas makes it extremely easy to create new users. In fact, the ease of use of the Canvas system is the reason that Financial Mentors of America (FMA) chose to develop it's online content in it, as opposed to other LMS that are available. But there are a few advantages of taking the complicated route and creating the user through the API. 

The main reason I do it this way, in some cases, is because creating a user through the API gives me full control over the process. I can completely register a user on the back-end, pick him a password, and skip sending the auto-generated confirmation email to send my own welcome letter instead. You can view all the other neat things by viewing the official API documentation. 

This, in a way, demonstrates the real power of the Canvas LMS. You don't need to know anything about the back-end to be able to use Canvas to a great extent. However, for the users who take the time to learn it, it opens up a whole new world of functionality. 

The Script

The requests library really makes this process simple. You'll notice from the first blog post that there are many of the same elements in it. You still have a URL, although the URL changes depending which API call you are making. You still have headers and you still need your access token. In fact, I have only changed two parts of the last script. 

1. payload = {...,...,...}
2. r = requests.post(..., ..., params=payload)

Payload

This is where we tell our script which of the special parameters we want to include in our post. In this case, I've picked the users name, assigned to the name the email address, picked his password, chose not to send his confirmation email, and auto-registered him. Once again, for any request you make, there are almost always custom parameters you can include in your request. These are all documented in the Canvas API documentation. 

r = requests.post(..., ..., params = payload)

This is where we tell Canvas what kind of API request we are making. In my first blog, we told Canvas to GET information, in this blog we are telling it to POST information into our Canvas instance. In addition to the URL and the headers, we include in the POST the "Payload" which contained the specific information we wanted our POST request to do. 

The Result

So I have python here set-up to return to me some mumbo-jumbo, However, the status code 200 means that my request went through. I was able to log-in with my user and the selected password, and my parameters went through successfully. I usually use stuff like this in conjunction with enrollment API, so after I have the created user, I can enroll him/her into a course with whatever level of access I like, with about the same level of control. 

Hopefully, this has been useful in not only giving you an example of some simple, working Python code that can interact with Canvas but also demonstrating some of the inherent usefulness in using the API to gain more control over some of the things you may have to do in Canvas on a regular basis. 

Are you tired of seeing all those ancient events from two years ago in your syllabus feed? Want an easier way to remove old events without having to manually delete each one or having to remove all dates from everything in the course? Give this script a try! You can download the script and use Tamper Monkey in Chrome or other userscript tool for personal use or copy the script into the custom javascript file to give access to all instructors on your campus!

This is a basic version of the script and I'd like to some help adding pagination to the events listed (it's limited to 100 by the API). Let me know what you think!

Calendar Event Manager Script

Howdy! Well, I guess it's time to share this out with everyone. We are using this script I wrote this school year with our teachers. I taught myself Javascript as well as the Canvas API in order to save everyone the pain and strife of crosslisting.

*Updated 11/19/2021 - I added a filter to only load courses that the user role is "teacher".

*Updated 7/26/2021 - I fixed the maxTerm function - it wasn't evaluating correctly. 

*Updated 7/13/2021* - I fixed the "undefined" error some users were getting. I had to redo the maxTerm function as it had stopped working correctly.

This is a robust script in which the instructor's current term courses are loaded in a dropdown menu to choose the parent course. Then the remaining courses are dynamically created and are selectable to be child courses. Finally, the instructor can rename the parent course and submit to finalize the crosslisting and renaming actions. The instructor can also choose to crosslist without renaming and rename without crosslisting. On the subaccount page, it also adds the ability for admins to search for a user and then perform the same tasks as an instructor as well as decrosslist a course.

Instructor and Admin Crosslisting Tools 

If you want to test, use TamperMonkey in Chrome or other userscript extension.

If you want to deploy to all staff, copy the script into the custom Javascript folder.

I welcome those with better programming chops to help me improve the script. If you have any questions about what any part of the script does (my documentation isn't all done yet!) please let me know and I'll be more than happy to help. Otherwise, let me know what you think and what other features you may want to see implemented!

Overview

As Instructure's Partner Integrations Specialist, I’m often asked how an LTI tool can return a grade to Canvas for an assessment that’s completed on the tool provider side. The answer varies based on a set of requirements the tool provider has. However, I’ve outlined the most common workflows I encounter to hopefully help others find the right place to start—keep in mind that a combination of these workflows can also be utilized, you don’t have to choose just one.

LTI-only Workflow

Requirements

This approach requires an LTI integration to be configured with an assignment_selection placement and a content item selection request. It should also be capable of executing the LTI Outcomes service.

Advantages

• API access not required
• Interoperable
• Can return LTI Launch URL as submission, allowing the tool to iframe a submission on the submission details page and within Speedgrader without the user having to leave Canvas
• Grades applied using the LTI Outcomes service will also mark the student as having completed the assignment; this isn’t the case when a grade is returned via Submissions API

Limitations

• Teacher must manually configure each external tool assignment using content item selection
• Student must launch each assignment so the tool can collect required information
  • As a result, a tool can’t have a series of assessments where grades are returned to separate assignments unless they provide the content as a common cartridge

Instructor Workflow

1. Create an assignment in Canvas
2. Choose “External Tool” as the Submission Type
3. Choose the tool from the list that appears in the modal
4. A properly configured tool allows the instructor to then choose a resource
5. Tool returns a content item message to Canvas with the Launch URL to that resource
6. Teacher saves and publishes the assignment

Student Workflow

1. Launch the assignment triggering a launch to the launch URL provided by the tool when the teacher configured the tool
2. Tool consumes the lis_outcome_service_url and lis_result_sourcedid for the student X assignment combination and stores it
3. Student completes the assessment
4. Tool either automatically grades the student, or waits for teacher to grade student
5. Grade is returned to Canvas via LTI Outcomes service
6. (optional) Using the LTI Outcomes service, the tool can also return a piece of plain text, a basic URL, or even an LTI Launch URL. This will be attached to a student submission object in Canvas, and it’ll be visible on the student submission page and in Speedgrader

LTI + API Approach

There are two flavors of this workflow. One uses the LTI Outcomes service for grading, while the other uses the Canvas submissions API.

Requirements

This approach requires an LTI integration to be configured with at least a course_navigation placement. It also must use OAuth2 to obtain teacher and student tokens.

Advantages

• Allows a single launch point from Canvas (usually from course navigation) where a student can complete many assessments and have the tool provision assignments within Canvas on the fly and grade them as needed
• Less manual work for the teacher
• Organization of assessments can be offered either on the tool side, or by leveraging the Modules page in Canvas
• Content-Item not required
• Tools can create external tool assignments and use the API to simulate a student launching the tool in order to obtain data required for the LTI Outcomes service

Limitations

• Requires OAuth2 to obtain either a Canvas API token for teachers (to grade) and students (to optionally submit)
• Requires heavy usage of Canvas specific API’s, so is not interoperable
• If grading is achieved via submission API, submissions are not marked complete after a grade is applied. The tool must also submit to the assignment using the submission API to mark complete. Furthermore, only plain text or URL’s can be provided

Instructor Workflow

1. Instructor launches the tool from course navigation or elsewhere and assigns the available work to students, or the tool can do this automatically
2. Tool kicks off OAuth2 if it needs to obtain or refresh a teacher token
3. Tool creates external tool assignments using the Assignments API. These assignments should have an LTI Launch URL that has query parameters for the assignment, or the path that launches directly to the assignment, e.g. https://some_launch.com/assignment/<tools_assignment_id>; or https://some_launch.com?assignment_id=<tools_assignment_id>;

Student Workflow

1. Student launches the tool from the course navigation button
2. Tool kicks off OAuth2 if it needs to obtain or refresh a student token

1. Student picks the assignment within the tool and completes it
2. Tool grades and submits the assessment to Canvas
   • Option 1: LTI for grading
     • Tool uses the sessionless launch API as the student to launch the tool
     • Tool consumes the lis_outcome_service_url and lis_result_sourcedid for the student X assignment combination and stores it
     • Tool uses the IMS Outcomes service to return a grade and an LTI Launch URL to their submission
   • Option 2: API for grading
     • Tool uses the Submissions API as the teacher to grade the student
     • (optional) Tool uses the Submissions API as the student to submit so the teacher can verify students completed the assessment

NOTE: The process can be further automated/simplified by obtaining an admin token with the “become other users” permission. This token can then be used to make API requests on behalf of teachers and students by appending the as_user_id=<student/teacher user id>. However, this can be problematic for high numbers of assignments or students as you’ll likely experience throttling issues.

I always like to hear about creative ways for integrations to grade and submit assignments. How has your integration combined API and LTI to grade and submit to Canvas?

We’ve been moving to a “standards-based mindset” (props to Schimmer for a great book!) in our assessment and grading over the past couple of years and have begun collecting student performance data using a variety of methods. We’ve tried Active Grade (now a part of Haiku) and JumpRope, which are very full featured standards-based gradebooks, but without a full fledged institutional commitment - i.e., everyone grading fully on standards - it’s difficult to justify the cost of additional software, especially when we are already using Instructure Canvas as our LMS. The goal is to provide feedback using standards, but not transition to a standards-based report card.

Canvas provides a standards-based solution called the Learning Mastery Gradebook, which pulls data from outcomes aligned to assignment rubrics. These outcomes can be organized into outcome groups within a course, then reported to students through the normal “Grades” interface. Outcome scores can be calculated using a variety of methods; the one we prefer is the simplified “decaying average” which favors the most recent assessment point by whatever percent you specify, with the remainder of the score being made up of the average of the rest of the previous assessment points.

Our desired approach involves unpacking curricular standards, or power standards, into specific “learning targets”. In Canvas, power standards correspond to outcome groups, while learning targets correspond to individual outcomes. Ultimately, we want to provide students with scores on both learning targets and power standards - something Canvas currently doesn’t do.

I’ve created a little app using Python that leverages the Canvas API to pull data from a user’s course and the Learning Mastery Gradebook, restructure it, calculate a mean outcome group score, then send students a templated message in their Canvas inbox with their results. I think this will be a functional bridge between what Canvas currently offers and products like JumpRope, which are marketed as standards-based gradebooks and not as an LMS.

Take a look at the code in the github repository here: https://github.com/BRueckert/CanvasSBG

I’m very interested in collaborating with others who want to hack the LMG like this - particularly someone who has some experience creating GUIs using Tkinter in Python. The app is currently very ugly; I didn’t pay much attention to how buttons were laid out. Just figuring out how Canvas structured the response data was challenging enough!

Send me an email or tweet me @CITBrian if you’re interested in improving and extending the functionality of the app!

Edit: Here are a few screen shots:

Original post: rcanvas + the tidyverse 

rcanvas continues to grow. Thanks to the recent contributions from Chris Hua, getting user groups, announcements, and discussions from your institution’s Canvas instance has never been easier. More collaborators are welcome!

By my lights, R package development should be attuned to the tidyverse. Piping output into a sequence of clear, logical functions not only makes for clean, readable code, but is an undeniably damned good time.

The remainder of this post will showcase the interplay between the tidyverse and various rcanvas functions.

Getting Students, Teachers, or Observers

students <- get_course_list(include = c("term", "total_students")) %>% 
  filter(
    term.name == "2017 Spring",
    total_students > 0) %>% 
  .$id %>% 
  map_df(get_course_items, "enrollments") %>% 
  filter(type == "StudentEnrollment", #'TeacherEnrollment' for teachers, 'ObserverEnrollment' for observers
         enrollment_state == "active")‍‍‍‍‍‍‍‍

With the data in hand, you can filter by any relevant criteria. Say, for example, you want to know which students have been inactive for more than a week:

library(lubridate)
students %>%
  mutate(inactive_days = as.numeric(round(difftime(ymd(Sys.Date()), ymd_hms(last_activity_at), units = c("days"))))) %>% 
  filter(inactive_days >= 7)‍‍‍‍

Getting Missing Assignments

Here's how to get all missing assignments for a course, multiple courses, students, or multiple students. Again, the code flow is intuitive. First, get the courses. Second, filter the course(s) you want. Third, pipe the course ids into `purrr::map_df`, and fourth, extend the pipe with user ids into `get_user_items`.

biology_missing_assignments <- get_course_list(include = c("term", "total_students")) %>% 
  filter(
    term.name == "2017 Spring",
    total_students > 0,
    grepl("Biology", name)) %>% 
  .$id %>% 
  map_df(get_course_items, "enrollments) %>%
  filter(type == "StudentEnrollment",
         enrollment_state == "active) %>%
  .$user_id %>%
  map_df(get_user_items, "missing_assignments")‍‍‍‍‍‍‍‍‍‍‍

In sum, getting lots of data very quickly is just a matter of piping the course and user ids into various functions and letting the tidyverse do its thing.

What else can I get?

Lots! User page views, profiles, avatars, and observees, or course discussion topics, assignments, files, modules, pages, quizzes, folders, todos, settings, and more. If it’s in the API docs, we’ve tried to include it.

The Java development team here at Kansas State University started our Canvas adventure off with some questionable decisions. We were originally just going to write a couple of simple LTI applications. Of course the scope and complexity of the applications grew but things were structured as a single monolithic code base. This quickly became unacceptable and we began working on splitting things into more reasonable chunks.

One of the chunks is a stand-alone API library to handle all of our interactions with the Canvas API. We wanted a central place to deal with things like pagination, masquerading and OAuth access tokens. After we started pulling this code out of our initial mess we thought the wider Canvas developer community might be able to benefit from it and decided to work towards releasing it as open source software. It is by no means a finished work but I would like to put it out there and see what others think about what we have so far.

The code is up on GitHub and available under the LGPL v3 license: kstateome/canvas-api: Java library for interacting with the Canvas LMS API

We are also publishing releases to the Sonatype Maven central repository so it can be easily used in maven based projects. The maven coordinate is edu.ksu.canvas:canvas-api

The readme file contains details on how to use the library. Following are some notes about the state and direction of development. We are also soliciting input from any other developers out there.

We do not have every API call implemented. Not even close. We are implementing them as we have time and need them for our own applications. The good news is that most of the heavy lifting of performing the HTTP requests and parsing the responses is sufficiently abstracted in common code so implementing a new call for an existing object type is usually pretty easy, especially for read operations. Creating new objects can be annoying because the Canvas documentation cannot be relied on to be accurate. You have to actually execute API calls and examine the returned data. Pushing data into the API can be tricky at times because of some API quirks that we are still discovering.

As mentioned above, we would be happy for some input from other developers. I think there are a few things we still need to change before it is really ready for more widespread adoption by others since they might be breaking changes as far as method/class signatures.

There are two specific related areas where I would love input, and they need to be set in stone quickly: Return types and exceptions. Right now every method for an API call that returns a single item has a return type of Optional<SomeCanvasObject>. As an example, if you request a course with an ID that doesn't exist, the Canvas API will return an HTTP 404 error but our library will return an empty Optional<Course>. Does it make sense to do this?

This is one of the first projects where we have used Java 8 from the ground up so we're still learning how to appropriately use some of the new features like Optional. I think this design grew out of some code where we had some shady instances of "catch(Exception e) { return null; }" going on which, of course, spawned NullPointerExceptions in other places. This got translated to Optional when we rewrote it because we wanted to avoid NPEs. It might make more sense for the methods to return the bare Course object and throw a descriptive runtime exception if there is a problem executing the request. Like throwing a custom ObjectNotFound exception if the API returns a 404 instead of returning an empty Optional. Returning null should definitely not happen. On the other hand, using Optional can be a little cleaner than having to wrap things in try/catch blocks.

That being said, you already need try/catch blocks in a lot of cases because right now every method declares throws IOException. If you pass in the wrong Canvas URL or a malformed URL, the Apache httpclient code will end up throwing an UnkownHostException or URISyntaxException or ConnectTimeoutException or something like that, all of which extend IOException. This exception will be passed all the way up to the calling code. This is kind of inconsistent with some error conditions returning an empty Optional but others letting IOExceptions to be thrown.

Another thing that might look a little odd at first glance is the requestOptions package. These are objects to encapsulate all of the (often optional) parameters you can pass to a given API call. Taking options via individual method arguments would be a bit ridiculous for some API calls that take 6+ parameters. And, if Canvas adds a new option, it would break existing code by changing the method signature. So far, so good. However you will notice that these request options classes have enums inside of them, some of which are repeated in multiple options classes. I initially tried to share the enums but ran into Canvas accepting different values on different API calls. I also found one instance of two API calls accepting the same logical options, but using different string literals to represent them. I actually opened up a GitHub issue about that one.

We still haven't ported all of our applications over to use this stand-alone library. However we have already used it to do a few things. One was to iterate through every course page in our account to do a search/replace on page content when we moved our Mediasite installation to a new URL. That process executed over 140,000 API calls over the course of a day without incident. Right now I am working on switching our Scantron LTI application over so there will probably be some missing API calls that I have to implement within the next couple of weeks for that.

We already have a pull request open on our code from someone at another institution so I'm hoping that is an indication that there is demand for a library like this and we can get some collaboration going. But first, please roast our code! :smileycool:

What is this about?

If you are a .NET developer and are interested in leveraging the Canvas API to automate tasks at your institution, you might be interested in ths demo project.  I will provide a Visual Studio project that you can download and use as starting point to develop your own tools.

References

I can try to guide you to some reference material for those getting started that will hopefully help.

There are a few basics that would help to understand when working with the associated project:

• Working knowledge of C#: 
  C# Reference​
• What is await? Faimiliarity with asynchronous programming (not required, but loosely referenced in the associated project code):
  Asynchronous Programming with Async and Await (C# and Visual Basic)
• Understanding of HTTP verbs, you might start here if you need some info:
  HTTP Methods GET vs POST vs. PUT
  
• Working knowledge of Json, understanding the structure of a Json object is very important:
  JSON Introduction
• The base classes used to make the API calls:
  • HttpClient Class (System.Net.Http)
  • HttpClientHandler Class (System.Net.Http)
  • HttpResponseMessage Class (System.Net.Http)
• Logging - the sample project uses NLog to create debug and error logs:
  NLog  - can be referenced in Visual Studio using Nuget
  The project includes a sample logging config file
• Newtonsoft Json.NET library - to serialize/deserialize objects
  Json.NET - Newtonsoft  - can be referenced in Visual Studio using Nuget

How does it work?

The associated source code will create a command line tool that accepts a config file.  The config file contains the following information:

• "accessToken" - your access token
• "apiUrl" - the url to your Canvas site, I would you Beta or Test
• "apiCalls" - array of API calls that you would like to run, including parameters

For simplicity and purposes of demo, the access token and API url are in this config file.

The format of the API call list is an array of pipe delimited strings in the following format:

• [method name] | [param1],[value1] | [param2],[value2] | .... | [param(n)],[value(n)]

These strings will be parsed and passed to the API library where they will be converted into actual calls to the Canvas API, at the site defined by the "apiUrl" variable.

This sample config includes GET, POST, and PUT calls, to demonstrate each type of verb.

Each verb is implemented in the base class: clsHttpMethods.cs

Full response details are logged by NLog.  Grab these responses and inspect them for full details.  If your test produces an error or throws an exception, full details can also be found in the log file.  If you use my default nlog.config, log files will be found in this location:  c:\\logs\canvasApiTest\*.log

Comments

The purpose of this project is not to replace tools like Postman, the purpose is to help other developers using .NET to get started with the API.  I tried to keep things as clear as possible.

With myapi.config, I have been able to test any API call I have needed to date.  If you follow my code, you will need to add a C# method to handle each additional API call you want to make.  My "long hand" approach to creating matching C# methods could be modified, but hopefully helps to illustrate the concepts.  Share your approaches.

There is plenty of room for optimization, optimize to meet your needs and share any enhancements you feel people could benefit from.

Project Source Code

Source code can be found on BitBucket here:  Canvas API Implementation​

The source published on BitBucket compiles and runs with Visual Studio 2015 and .NET Framework 4.5.2

IMPORTANT:  Make sure you edit the myapi.config file with your settings, and modify API variable values to match your environment.

All code is provided as-is for demonstration purposes only.

2016.12.03 - UPDATE - File Upload via POST

• I have added a class that walks through the steps of uploading a file.  This code was pulled from another project and put together quickly as a response to a community question, make sure you test failure scenarios in your environment.