There is via the robots.txt file, but that is only editable by Instructure. If you read the current file, it only excludes /page_views/, which doesn't exist in Canvas. The comments in the file (which are pretty standard) state to ban all spiders from the site, uncomment the next 2 lines. But the next 2 lines only block all spiders from /page_views/. No matter what, though, that file is not right as it is disallowing crawling on a page that doesn't exist.
Further, the robots.txt file is the same on canvas hosting as it is in github here:
canvas-lms/robots.txt at master · instructure/canvas-lms · GitHub
which hasn't been edited since 2011 - making me think it's been forgotten about; That the page it does block existed in Canvas at one point.
I opened a ticket with support quite a while ago (over a year?) and even had a meeting with our CSM and someone on the development/engineering side. The end response was they would update the file - but it hasn't happened yet. The support case is still open.
If you would like the case number, let me know.
Thanks,
Anthony Hess
