How to bulk load and update avatar profile pictures with Powershell.

BenjaminSelby · ‎09-24-2020

[UPDATE 2024.02.20 - I've refined the code around this process, and posted it in a new blog article here: https://community.canvaslms.com/t5/Canvas-Developers-Group/How-to-Bulk-Load-and-Update-Profile-Avata.... The info in the article below is significantly out of date, but some of it might help to explain the process so I'm leaving it up.]

[UPDATE 2022.03.03 - Note, this script has been modified slightly, so the current version on GitHub is slightly different to that described in this article. The main change is that the script now no longer downloads any profile image for users from a database. Instead, this script expects a folder to exist containing all user photos, with user IDs as their file names. The reason I made this change is because I have multiple applications which utilise user photos, so it makes sense to split the photo sync out to another script which keeps the photo folder up-to-date. Then, multiple applications simply access that folder, and can be assured of up-to-date photos. The bulk of this article is still relevant, though, so I don't think it's worth re-writing. If anyone has questions, feel free to ask in the comments. -B]

I have uploaded two files to GitHub in the following repository:

https://github.com/benjaminselby/CanvasAvatarImageSync

The file which does the bulk of the work is SyncCanvasAvatar.ps1. This syncs an avatar image for a single Canvas user.

The file CanvasAvatarSyncAll.ps1 is the script which is called by Windows Task Scheduler. It obtains a list of all Canvas users in our domain, then loops through every one and syncs avatar images for all users with valid SIS ids. Anyone without a valid SIS ID is ignored (our Synergy database stores user images based on SIS ID lookup).

I will walk through the code a little bit below. Our in-house solution has quite a few lines which are mostly concerned with message logging and error handling, so it's arguably longer than it needs to be. It also produces a very verbose log, which I found helpful during debugging, but they could obviously be removed if you don't want this.

I don't profess to be a Powershell expert, or a great programmer, so if anyone has feedback or comments, please let me know.

SyncCanvasAvatar.ps1

A summary of the script's activity is:

1) Get the user's SIS ID and other information from Canvas. This script updates the avatar image for a single user. So, you need another script which calls this one for every user you want to update. I do this by getting a list of all Canvas user IDs via an API call, then call this script for each one. Of course, there are a lot of users, and not all of them have photos stored in Synergy. If no photo is exported for a user, this script will skip them.

2) Check 'Sync History'. One thing I wanted to avoid was re-uploading the same image file over and over again (inefficient). So, when the script uploads an avatar image successfully, it updates a database table with information about the user, the image which was uploaded, and the datetime of the upload. So, the script checks a user's current avatar by obtaining the user's current avatar URL. Avatar URLs contain both the Canvas ID of the image file which was assigned as that user's avatar, and also that file's UUID. If the user's current avatar URL information matches the information we have stored in our Sync History database table, then we know it's the image we uploaded previously (ie. the correct image). In this case, the script exits because there is no work to do. Also, if the user's current avatar URL is the 'default' URL, then we need to upload an image.

Also, we have a timeout on image uploads - once they are a month old, we download a new image from Synergy and re-upload it. So, if a student's photo changes (our school photos are updated every year), it will always be updated on Canvas within a month.

3) Export the user's photo from Synergy. We use Synergy as our main SIS solution. Staff and student photos are stored in that database. I have created a stored process attached to the database which exports a user's photo to a folder, with the Synergy ID of that user as the file name. So, the script looks up the SIS id for the user, based on their Canvas ID, and then uses this to call the stored process which exports the picture from the database. All the stored process calls in this script use default Windows authentication for the current user. (You will need to organise your own image export - I haven't posted the stored process code at this point, but I might do in the future).

4) Upload the photo to Canvas and save in the user's Profile Pictures folder. If the user doesn't have a Profile Pictures folder, the script will attempt to create it. I've only had a couple of problems with this - for some reason, one student's Profile Pictures folder was locked (no idea how that happened). Some other users haven't even activated their Canvas accounts (by clicking the link in their activation email), in which case this won't be possible.

Image file uploading can be a bit tricky. I've covered this in other posts:

https://community.canvaslms.com/t5/Developers-Group/How-to-Upload-an-Assignment-via-API-using-Powers...

https://community.canvaslms.com/t5/Question-Forum/How-to-upload-a-file-via-API-using-Powershell/m-p/...

There is a two-phase process where you start by notifying of your intent to upload, which returns a URL. You then send the file data to that URL (which has a life span of 30 minutes).

Before uploading, this script checks the user's file quota remaining to see if there's enough room to upload the file. If not, an error is returned. I have another script which checks all user file quotas on a weekly basis so I can harass users who have exceeded their quotas... 😉

5) Assign the image as the user's avatar. This is a two-step process. You need to get a list of available avatar images, then cycle through each one to find the file you just uploaded. Any images in the user's Profile Pictures folder should be 'available'. The correct image is identified by UUID, which was returned when you uploaded the image data in the previous step.

Also, another possibility is that we have previously uploaded the 'correct' image, but the user changed it. In this case, the correct image should still be in the user's Profile Pictures folder, and so it will still appear in the user's available avatars list. In this case, we don't need to upload anything - just reassign the correct avatar.

6) Once we have successfully uploaded and assigned the correct avatar, we update our Sync History database table with the information from this process. This script runs every night, so when a user's avatar is unchanged it matches the info we have in our database, which means we don't need to re-upload the file.

--

I hope this helps people out. If you have any questions or comments, let me know.

sgutierrez1 · ‎05-25-2021

Is there a way I can modify this that doesn't use Synergy, but can still check the folder size in Canvas and checked that the image was already uploaded before? I have a server hosting the pictures generated from our ID system all in jpeg format and using their SIS IDs in their filenames. Also, we want to target the students only. We can generate a file that lists all of the student's SIS IDs so I'm wondering it can check based on that list instead of every user in Canvas?

BenjaminSelby · ‎05-27-2021

@sgutierrez1 - I don't see any problem with that. You'd just need to modify the script in a couple of places to point to the correct resources. I believe you can assign an external URL as a user's avatar (although whether this can be an intranet-only link, or if it needs to be WWW accessible, I'm not sure). Like I said, you will need to modify the script to work for you. This won't simply work 'out of the box' in most cases. You'll need to get someone with some basic Powershell skills to look into it. Sorry I can't help any further.

sgutierrez1 · ‎06-14-2021

Hi I modified in a few places and was able to use a specific ID list from a CSV file. Right now I'm dealing with HttpResponseException errors. Any ideas?

BenjaminSelby · ‎06-15-2021

@sgutierrez Post the full error text and I'll see if I can help. Have you tried stepping through the code with a debugger? I find Visual Studio Code to be a good application for Powershell scripting. Make sure all URLs are correctly formed and valid.

sgutierrez1 · ‎06-16-2021

I realized the module doesn't work anymore for TLS 1.2 so I had to remove it. I tried the VS Code to debug it. Gave me some insight on some of the parameters that needed fixing. But I'm not getting any error messages now and it's still not working for me.

sgutierrez1 · ‎06-16-2021

So the VS Code helped me get a bit further and I was able to clean up the script a bit more. Some of the components you had in it weren't available in the current version of Powershell so they had to be removed. The only roadblock I came across was the quota check.

How did you get your integration account to have access to view the user's quota? Mine has full admin access, but when I tested using Postman, I got "this user is not authorized" message.

sgutierrez1 · ‎06-17-2021

I was able to figure out how to pull the user the user quota data. There was a change I had to do in the address itself to get the masquerade part working.

BenjaminSelby · ‎06-20-2021

@sgutierrez1 Great - so, it's working for you now? Let me know if I can assist with anything further.

I'm actually planning to load a simpler version of the code to GitHub soon, which doesn't download the images from Synergy, and instead simply expects to find the images saved in a drive folder. This simplifies the logic tremendously. I now have a separate script which maintains the profile photo folder, because it's being used by other applications so it makes sense to separate out the logic.

sgutierrez1 · ‎07-06-2021

I got the script working but had to do a lot of trimming. Also, I didn't realize I needed to install Powershell 7 and Curl.

What changes I've done:

For the SyncAllAvatars file, I had it import a csv file that had a list of specific SIS IDs (it was active students only) and passed them to the SyncAvatars file as the $userCanvasId object. So no conversion was needed.

However, dealing with that, I had to change part of their folder address to look up by their SIS ID instead (<Canvas Site>/api/v1/users/sis_user_id:$userCanvasId/folders).

I had to remove all the Try/Catch code because it was defaulting to the catch every time (it was probably because I didn't have Powershell 7 at the time).

EricLaCroix · ‎08-23-2023

Thank you very much for this article! I have written a FileMaker tool to link to Canvas to upload user images and to retrieve some basic student status information largely based on the information you shared here. Thank you, thank you!!

BenjaminSelby · ‎08-30-2023

@EricLaCroix I'm glad to hear that it's been useful for you! Thanks for your comment.

morris_gart · ‎10-09-2023

@EricLaCroix I would be grateful to see that Filemaker tool or at least an explanation how you linked Filemaker to Canvas. @BenjaminSelby is the Github links still the accurate link? Thanks so much for posting this information.

EricLaCroix · ‎10-09-2023

I watched a YouTube video about making API calls using the "Insert from URL" script step in FileMaker to pass cURL parameters and then use some of the built-in JSON functions to parse the responses. ("JSON in FileMaker - Introduction for Beginners - Day 1" posted by @FileMakerVideos.) I referred heavily to the Canvas Live API documentation to decide what calls I wanted to make. Short version is first I downloaded all the users out of our instance into a table in FileMaker so I could get their Canvas ID and their SIS ID. Then, in another table, I imported all of their 180x180 JPG pictures named with <SIS ID>.jpg. Finally, I relied heavily on the guidance from Benjamin's Powershell script to replicate his functionality in a FileMaker script that uploads the photos if necessary. Never having done this before (I'd never made an API call in my life), it was a lot of trial and error! It took me about 30 hours to build the FileMaker solution that not only uploads the photos but also summarizes a student's status in Canvas for the people who do not have observer or admin accounts.