[ARCHIVED] "UDOIT FAILED TO SCAN THIS COURSE"

[kevinw]

Hi all, 

I posted this question elsewhere but thought perhaps it could have it's own thread. If I'm wrong go ahead and delete me. Anyways, I recently watched Canvas' accessibility video and was thrilled to learn about UDOIT. We have been using Sitemorse's snapshot which is unable to run in java so we were having to copy/paste HTML code and it was just a pain. 

However after going through and successfully deploying UDOIT to heroku I consistently get the same error: UDOIT failed to scan this course. 

Has anyone else experienced anything like this? I'm unsure of where to go from here. It's really disappointing because there's just nothing else like this in Canvas and it sees like it is an incredibly powerful tool. 

[Stef_retired]

 @kevinw , I've shared your question with the UDOIT guru, so stay tuned.

[tr_jbates]

I've heard from one school that had this problem, but it cleared up on its own after a day.  Unfortunately, this particular error could be caused by any number of things, so I'll need to look at the server logs to know what's going on.  You can access the logs for your Heroku instance by following this article:  Logging | Heroku Dev Center.

If you're not familiar with doing things through the command line, let me know, and we can work through it together.

[kschneider25]

We had this issue.  I basically started from scratch on the building of keys within Heroku and Canvas.   Also we had one user that received a similar error and that browser will require a "refresh" before UDOIT will run. 

[kevinw]

Hi all, 

Thanks for the feedback. I'll take the suggestions listed here and report back soon. 

I refreshed the browser and performed and HTML Check and it worked. This wasn't uncommon because testing the pages was usually where it failed. I tested the pages and it ran for a minute or two (very large course, 3.2gb) and I got the same error. I refreshed again and now am getting an error page and UDOIT will not load. It says 

"

Application error

An error occurred in the application and your page could not be served. If you are the application owner, check your logs for details.

"

I suppose naturally the next thing to do will be to check the logs. So I will begin to do this over the weekend. 

I'm sure I'll have LOTS of questions so stay tuned everyone! 

[kschneider25]

I would check the logs.  Or try some smaller courses and see if you can pinpoint where the scan is failing.

We received the error prior to a course content being scanned which is why I reset all the keys in Heroku and Canvas. 

Good luck.

[kevinw]

So I waited a minute or two and everything refreshed. I successfully ran a scan on my assignments and the results came back. But when I tried to scan our pages I received a bunch of these errors:

2017-03-03T04:29:15.548578+00:00 heroku[router]: at=error code=H12 desc="Request timeout" method=GET path="/progress.php" host=udoitfma.herokuapp.com request_id=eaddf82d-f6ec-4ee6-9082-336c954b40d4 fwd="***.***.**.**" dyno=web.1 connect=1ms service=30000ms status=503 bytes=0

I'm assuming this means that it's taking too long to run the checks and so it is timing out? Is that correct?

[tr_jbates]

How long did the run take before the error appeared?  Also, does this happen on any other courses?

[kevinw]

Over the weekend I installed it on two other courses for testing. It only successfully scanned one. The one it passed was much smaller than the two other courses.

When it fails to scan the pages it usually runs about two minutes. Then UDOIT gives me an error page for the next couple of minutes. Then I'll refresh and it will be up and running again. 

[tr_jbates]

Are there any differences in the types of content that are in the larger courses that failed versus the smaller one that passed?  There may be a certain bit of code that the scanner doesn't like that appears in both of the failed courses.

[kevinw]

Gosh it's so hard to say. Our large courses are huge. Both have embedded videos. Maybe embedded MP3 files? What kind of files have caused issues like this in the past? 

[tr_jbates]

(Sorry for the delayed response, I could've sworn I clicked "Add Comment" two days ago.  Luckily Jive saves draft comments!)

We've had a few courses in the past where we had to run each section separately, but never where it just failed to scan.  You could try testing it out on more courses to see how widespread the issue is, which might also reveal a common denominator between them.  You can also clone an affected course and start removing pages until the scan completes, then investigate the contents of that page.

At the same time, we'll be investigating the best way to figure out exactly where an error or hangup occurs during the scanning process.

[kevinw]

Thanks for the tip. I'm going to try to scan a clone of the first module and see what happens. I'll keep everyone updated with the results. 

[tr_jbates]

 @kevinw ‌ Did you have any luck with running it on a smaller amount of content?

[kevinw]

Hi tr_jbates‌. This project has unfortunately taken it's place on the back burner for the time being. The short answer is "no" but the long answer is "I'm not finished yet". 

[kari]

 @kevinw ‌ and tr_jbates‌ Jacob was probably over hearing from me reporting issues we had getting UDOIT up on our server (NOT HEROKU). I gave up hope on using this tool on our server and moving towards pitching getting Heroku with our administration. I am not sure if I updated Jacob. But,  after one more attempt to troubleshoot it ended up being a permissions issue. Our guy blew away all the PERM and gave rights to all and then the scanner worked but then he re-locked the permissions down and still worked.  I am not sure if this helps. So, far there hasn't  been any issues and we are going on 15 days. 

[jbuchholz]

I am not sure if this is the right place for this question so please redirect me if need be. We have installed UDOIT and are loving it!  However, a faculty member just received an authentication error and is unable to access the tool. She is running a current browser edition and has all the same settings as I do and my instance works great. She has also authorized the app in Canvas and still is not able to see or use the tool. Any thoughts tr_jbates‌

[tr_jbates]

 @jbuchholz ‌ One other institution has had this issue, but we weren't able to figure out what was going on.  Please have the faculty member try it out in different situations, like on campus, off campus, wifi, wired, chrome, firefox, IE.  You can also try masquerading as that user to see if it works or not.

[jbuchholz]

tr_jbates‌ In this instance we narrowed it down to the cookies setting in the browser and once we loosened the restrictions, that seemed to help.  I will keep you posted on any other situations that come up.  Thank you so much for your response. I appreciate it.

Jesse

[dbreckon]

We get an error that UDoIT failed to scan this course, and get it when there are multiple pages. This course had 30 pages. That doesn't seem like it should fail scanning for only 30 pages.

What are your thoughts?

[tr_jbates]

I'm out at a conference right now, but I'll take a look at this when I get back next week.  Does this error happen immediately, or does it happen after an amount of time that would make sense for the amount of content it's scanning?  Also, are you running this on your own server, or on Heroku?

[dbreckon]

Thanks Jacob. The error happens after a period of time....We are hosting on our own server.

[tr_jbates]

A few things that can cause this issue:

• Lots of YouTube videos in the pages.  UDOIT has to talk to YouTube individually for each one.
• Lots of files in the course.  Try clearing out unneeded files if this is the case.  If you can't delete any files, try running the scan on the Pages only.
• Funky code on one of the pages that UDOIT doesn't know how to handle.  We have worked hard to fix all of these cases, but they still come up from time to time.  Check out the pages to make sure there's nothing funky going on with the HTML.  You can also check your server logs to see if UDOIT spit out a PHP error.

Regardless, you (or your server admin) should pull the PHP and web server logs right after encountering the failure to see if anything popped up.

Also, I highly recommend updating to Release v2.2.0 of UDOIT.  This issue might have been fixed in that release.

[Chris_Hofer]

We just got UDOIT set up in our "Test" environment of Canvas.  We have done the local installation of UDOIT on our own servers vs. installing through Heroku.  For some courses in our "Test" environment, the scan works beautifully!  Reports come back in a few seconds.  For other courses, however, we are getting screens like:

Sometimes it will list things like "pages" and "files" with "2 left to scan".  It takes several minutes for the number to go down to zero...such as this:

I've let the scan continue running for at least 30 minutes...if not longer...and nothing has changed.  I'm not sure why some courses are scanning properly (and quickly) while others take a super long time.  Could this be because we're using our "Test" environment vs. "Production"?

Has anyone else seen things like this?  I've e-mailed tr_jbates‌ to see if he has any thoughts on this.

[tr_jbates]

Chris:  These issues might be caused by some bugs in our 2.3.0 release.  Please investigate/change the following items and let me know if that fixes the issue:

• In the "reports" table, add a "report_json" field of type "MEDIUMTEXT" if it does not already exist. If it already exists, change the type to "MEDIUMTEXT"
• In the "job_queue" table, make sure the "results" field is of type "MEDIUMTEXT"

[mmutz1]

tr_jbates‌ thank you for that reply - in job_queue ours was text and I changed it to mediumtext and our problem is solved

after looking at the table schema, I first backed up the database, then changed that column. Code for backup and alter table are here, in case it helps anybody. Requires a basic understanding of sql and of course substitute your own database name and export file name. the --password switch will prompt for your database password when it runs

First, this will back up your database in case something goes wrong

mysqldump --database udoit -u root --password > backup-file-name.sql

and then this will change that field to mediumtext
 ALTER TABLE job_queue CHANGE results results mediumtext;

[tr_jbates]

That's great to hear, and thanks for providing the SQL.  The changes will be incorporated into the 2.3.1 release, which will come out very soon.  I'll include these instructions for those who have installed 2.3.0 already.

[tr_jbates]

We just released https://community.canvaslms.com/groups/accessibility/blog/2018/02/15/udoit-version-231 which should take care of the migration script issues.  It still requires manual fixing for anyone who already installed 2.3.0.

[MSJC-DELTA]

We are experiencing the exact same issue as Chris Hofer described:

We are self hosted and have upgraded to the latest 2.3.1 version and the issue is still occurring. We implemented the MEDIUMTEXT fix and that didn't work either. Any help at all would be much appreciated, thank you! 

[tr_jbates]

Just to confirm:  You added a "report_json" field of type MEDIUMTEXT to the "reports" table, and you verified that you have a "job_queue" table, and the "results" field is of type MEDIUMTEXT?

Also, if you have access to your error logs (php, NGINX, and/or Apache), please email them to me at jacob.bates@ucf.edu 

[MSJC-DELTA]

Thank you for getting back to me so quickly! 

I have sent you an email of our error logs. All we can really see is SSL certificate renewals.

[tr_jbates]

Take a look at your job_queue table.  Do the entries say "pending" or "expired" in the "status" column?  If they say "finished", do they have any data in the "results" field?

What version did you upgrade from?  Did you make sure to update your localConfig.php file to match with the latest version of localConfig.template.php?

Sorry for all the questions at once; I just like to cover all the bases as quickly as possible.

[MSJC-DELTA]

We upgraded from 2.3.0 and we did make sure to match the latest version of the localConfig.template.php. From what it looked like, it would just freeze right here:

[2018-04-02 18:10:22] udoit.INFO: Requesting https://msjc.test.instructure.com/api/v1/courses/7597/pages/introduction-taxonomy [2018-04-02 18:10:30] udoit.INFO: Finished retrieveAndScan - course: 7597, content: pages [2018-04-02 18:10:31] udoit.INFO: Starting retrieveAndScan - course: 7597, content: syllabus [2018-04-02 18:10:31] udoit.INFO: Requesting https://msjc.test.instructure.com/api/v1/courses/7597/?include[]=syllabus_body [2018-04-02 18:10:32] udoit.INFO: Finished retrieveAndScan - course: 7597, content: syllabus [2018-04-02 18:10:32] udoit.INFO: Starting retrieveAndScan - course: 7597, content: module_urls [2018-04-02 18:10:32] udoit.INFO: Requesting https://msjc.test.instructure.com/api/v1/courses/7597/modules?include[]=items&page=1&per_page=100 [2018-04-02 18:10:33] udoit.INFO: Finished retrieveAndScan - course: 7597, content: module_urls

After some extensive testing, it looks like it is 2 pages in a group of our courses that was causing it to hang. Since this group of courses all shared these 2 pages, it threw us off. After removing the 2 pages, the scan completed successfully. After reviewing the 2 page's html, it looks like the instructors had input flash objects that were... crazy to say the least: 

I have sent the html code and the full log to you via email in case you are interested in developing a way for UDOIT to get past these types of pages, or to notify a user when a page can not be scanned, etc.

[tr_jbates]

That appears to be Base64-encoded data that stores the entire Captivate presentation and allows it to be transported easily without worrying about separate files.  I tried to scan a course containing that page with my Heroku version of UDOIT, and had no issues.  However, when I tried it with my self-hosted version, the database dropped the connection, resulting in "UDOIT failed to scan this course."  I think it was due to the max_allowed_packet size variable our MySQL server was configured for.  In your original post, you mentioned that UDOIT was freezing.  Does it ever display an error message like "UDOIT failed to scan this course", or does it just sit there acting like it's scanning forever?

[MSJC-DELTA]

No, it just acts like it is scanning forever. We let the scan run for about 45 minutes or so and it would not budge. If it is the max_allowed_packet size variable, do you have any suggestions on what we should bump it up to?

Update:

We updated the max_allowed_packet from 256M to 500M and that didn't do it. Then, we upgraded the MySQL server to MariaDB 10.1 and no luck:

[tr_jbates]

Normally I would say that your scan was timing out due to the size of the course, but if just removing those two pages fixes it, I doubt that's the problem.  The error logs you sent don't seem to show any issues, and I was unable to reproduce the exact issue you're having.

What version of PHP are you running?  I'm wondering because one of the big differences between 2.2.0 and 2.3.0 was the addition of PHP 7.0 and 7.1 support.

[MSJC-DELTA]

We upgraded our version of PHP from 5.6 to 7.1, which broke things so we upgraded to 7.0. Still no changes in the scan.

[tr_jbates]

Sorry for the delay.  I'll do some testing in 7.0 and 7.1 today to see If I can replicate the issue.

[justin_witt]

We are also using UDOIT 2.3.1 on a LAMP stack and are seeing the error "Udoit failed to scan this course." I did some basic troubleshooting and had the issue consistently only when scanning all content. I was able to get UDOIT to scan individual content types. I also noticed that when I got the error and then went back to my Canvas homepage and then entered UDOIT a second time that there was a completed report under "View old reports." In my case it looks like the report is running, but not printing correctly to the "Scan Course" tab.  

[tr_jbates]

Justin, your issue is most likely related to the timeouts you have set for PHP and your web server (NGINX or Apache).  Try increasing those timeouts and see if that allows you to scan the whole course.

Obviously, that's not the best solution, so if you're feeling adventurous, we have implemented a background worker that Heroku uses natively, but it can also be used in your own environment.  You would just need to run `lib/worker.php` as a daemon and set `$background_worker_enabled` to `true` in `localConfig.php`.  You'd need to make provisions for ensuring that the daemon restarts if it quits unexpectedly.  You might look into using something like Upstart to handle that for you.

In the next few releases, we're going to be including some instructions and maybe even some sample ini scripts that will allow everyone to take advantage of the background worker functionality.  I'd appreciate any feedback you have about the process if you're able to do it.