SIS Imports - Show Robust Errors and Human-Friendly logs

Idea created by Joni Miller Expert on Dec 8, 2015
    Open for Voting
    This is based on idea posted by Chris Sweets that got 52 votes and 13 comments:  Robust Errors for SIS Import


    Problem:  The SIS Import History is not in a Human-Readable Format

    For all SIS Imports, even if 100% successful, the following data needs to be included: 

    • Upload Time
    • Begin Time
    • End Time
    • Import completion time (in hours/minutes/seconds) - not currently listed
    • Number of records imported
    • Imported Items (Courses, sections, users, enrollments, etc)
    • Status:  Completed, Imported with Messages, Failed, etc.
    • Reformat logs for previous SIS Imports into a human-readable format.
    • Include a link to the error logs for previous SIS Import batches


    Problem:  There is no easy way to see details about SIS Import Errors - information is vague

    Example: We had a problem with our users file.  It has >30,000 rows and we had a student with a quote in her name "Denise", which caused the file to fail and not even load.  We would get a "malformed CSV" error.  This was REALLY hard to find in our file and took probably 5 hours of trial and error to find the line with the error since we didn't know which line or what could be causing the malformed error.


    Example: Problem file that kept uploading via our automated process and we didn't realize that it was causing all of the other files to back up and we ended up having to call and have an engineer cancel all of our SIS imports because we had probably 20 files in the queue.  It was a mess because we have no way of canceling on our side, and even support and our CSM can't do it.  It took several hours for the engineer to cancel everything in the queue.


    Example From Marissa Zelmanowicz

    I feel this feature is MUCH needed, as we frequently get long lists of 60+ errors stating "canvas_enrollments.csv - An enrollment listed a section and a course that are unrelated" and have no idea what course and enrollment it is referencing. Each time this error appears, hours upon hours of digging is required to try to fix the issue. Since we have thousands of entries coming from our SIS each night, this is a very time consuming and frustrating process.



    If an error is thrown, the following data in a READABLE, non JSON format would go a long way to helping admins figure out the problem and get us back to being productive. 

    • File Name of CSV File that had error
    • Type of import (courses, sections, users, enrollments, etc)
    • The line number of the data file that caused the error in the .csv should be included.
    • Field with Error (if possible)
    • Type of Error (invalid character is the one we usually run into)
    • Status:  Did it skip the line and move on or did it prevent any further records from importing?



    Would be nice: 

    • Add automated email options for errors and warning.  This will allow admins to keep an eye on automated imports.
    • Have the ability to email the warnings that happen during an import to an admin.  Especially when these imports are automated and/or take hours, it's important to know about any errors.
    • Ability to search for imports from a certain date or date range or Job ID #
    • Ability to see all past history without having to do an API call




    Format the error logs for previous SIS Import batches in a readable format.  We call the log page the "ugly page" at Seminole. 
    PLEASE have the import status of prior SIS imports available in a HUMAN-READABLE format!


    Thank you!