TamperMonkey-GreaseMonkey-Postman Tutorial

[dwillmore]

 @James ‌ has created many great scripts using TamperMonkey/GreaseMonkey that many use.  He also has created the GitHub canvancement which is awesome.   I am an admin with limited coding skills and new to TamperMonkey/GreaseMonkey and Postman and I imagine other administrators are in the same boat.  I would love for the experts here to gather into a team to develop a series of https://community.canvaslms.com/community/ideas/canvaslive?sr=search&searchId=58ed0fb7-8725-474c-be1...‌ webinar sessions that takes anyone using Canvas from simple user to Coding/API master.   What do https://community.canvaslms.com/groups/canvas-developers?sr=search&searchId=5f684eff-bd97-4da2-89b2-...‌ folks think about this?  Would anyone like/have time to join this user group team?  It would also be great to have developers from Canvas to join us as well.

[James]

 @dwillmore ,

When I started Canvancements, I tried to pick a term generic enough that other people could jump in and help out, so it would be awesome if there were an army of coding engineers. Along the line, I discovered that there were many people in a boat like yours -- limited (or no) coding skills, but wishing there was an easier way.

There are many fancier, much better written, programs out there than what I put out. I suspect there are a few reasons why Canvancements seem popular (I don't do any tracking, so I don't know for sure).

• They are written so that a typical user can run them. Using either Google Sheets or user scripts that add a link or button in Canvas. Many of the other scripts I've seen require Python or Java or NodeJS and a development environment.  but typical users don't know how to implement that or do the extra little programming to make it work for them.
• They're announced in the Community. Thank  @kona  for that as my chief advocate and the list of Canvancements - Canvas Enhancements that she keeps. The people writing these other programs are usually programming for their institution's needs and many even share their code on GitHub, but there's no central, well-advertised place where they can be found. If such a place existed, there may not be as many questions in the Community asking for the same things over and over. There's no guarantee of that, people still come in and ask for the same thing. Also notice that Instructure employees are reticent to recommend any solution other than their own. Canvas shouldn't be endorsing something that their engineers haven't checked out and given a stamp of approval.
• They meet needs (or at least come close) of those in the Community. For QuizWiz, I took and pulled together lots of feature requests from the Community to see what the common themes were. I then tried to implement the major ones that I didn't find pedagogically appalling. In other cases, it may just be one comment by someone - like the ability to highlight the unanswered questions when grading one question at a time.
• They are timely. Many of the ideas that I wrote code to implement were based off what I found here in the Community. I got tired of Canvas coming back with "That's a great idea, but it's not happening in the next 6 months." I understand that, but some people don't want to wait six months, or as it turns out much longer, and it may be never, for something to develop.
• They can do things that Canvas can't or won't do.
  • Canvas will never be able to offer a spreadsheet that allows people to edit all the due-dates in one place. Canvas can offer a web-page that allows editing all due dates, but then you don't get the power of formulas to make changes and set your dates based off other dates. Canvas could add a button that says "Show answers as soon as assignment is closed for submission".
  • Canvas can't make some things instantaneously inside the browser because they would take too long to generate. My scripts may be fine for small courses but fail miserably with large ones. Canvas has to support them all, whereas a teacher in a large class may just decide not to use my scripts.
  • Canvas has to make sure things are accessible and work in all supported browsers and on multiple devices. I can say "This won't work in MS Edge and I don't have any intention of wasting time to make it."
  • Canvas has to look at return on investment. Where do you spend your money? Do you dedicate 100 hours of engineering time to make 1% of users happy? I won't leave you hanging, the answer is probably no. What about 10% of users?
  • Canvas has to balance whatever they do against their "clean and easy to use" principles (quotes are mine). If you support everything that everyone wants to do, you soon have a very cluttered interface that isn't easy to understand. That in-turns increases the need for support. The Community is full of examples where people get stuck trying to understand something and can't move forward and increasing the complexity will make that worse.

I recently had the chance to meet with some people at Canvas. I was concerned about the move to ReactJS, the deprecation of jQuery UI, the not-exposing certain objects anymore, and other things making it harder to do what I do. From the conversations, I gathered a couple of things.

• They really don't want to make it easy for us to add this functionality. They didn't say that directly, but it was the little things like (paraphrased) "once we start using React, changing the DOM could break the way Canvas works" or "you should focus more on spreadsheets and external things than trying to modify the browser." I'm not even sure they're trying to make it harder for us, they're just not interested in making it easier for us.
• I'm a nobody. I keep telling people that, but Kona keeps insisting that Canvas is aware of the things I've done. That was not the experience I found. I mostly talked with managers, but I think most of them showed up because it was placed on their calendar but had no idea who I was or why I was there.

Beyond all that, which is just commentary to get the ball rolling, how did you arrive at the combination of user scripts and Postman? They seem similar but separate. Postman can get you started to figure out what you need to help with the user scripts, as can the live API, but it would also help with writing in any language.

[dwillmore]

I can understand Instructure's concerns and needs, but some of the functionality that you and others have provided are things that other LMSs do now, such as your 'edit dates in one place' workaround.   Side note: Our faculty love your workaround.

The first point you provided is very interesting and I thank you for sharing.    Using your TamperMonkey script seems to be an external tool that changes your UI in an 'unofficial' manner in that it does not change the entire instance of Canvas, must be made active per browser session, and can be turned off.  

I am on their side concerning changes made in JS that affect the site as a whole.   Personally, I push back on customization of a supported 3rd party product like Canvas unless that customization has a tool within the application to make the changes like the Theme Editor.  What I don't mind at all is using TamperMonkey to make temporary changes to an individual browser session.

Will the deprecation of JQuery UI and hiding objects affect your TamperMonkey tools?

Postman,

Postman is a different animal, and I should not have added it to the post.   Postman offers an easy way to run API calls iteratively. 

[James]

 @dwillmore ,

Yes, jQuery UI deprecation will affect some of my scripts, but not in a terrible way. I can add a @require line to require the jQuery UI for the versions that run in the browser. It will break things that people have put in their custom JavaScript unless they add it there or we find a different way to do those.

The scripts affected include the rubric sorter, the dashboard course card sorter, and a few others. The software engineer told me the deprecation was a ways off, but what that means to me is that I should start doing things the recommended way now rather than waiting until it's deprecated. I did not find out what the "right way" is. The user scripts are easy because they load all of the required files ahead of time and store them on the user's hard drive. Putting things into the custom JavaScript is more difficult.

To paraphrase: jQuery is supplied for you, but if you want to use React, then you need to load it yourself. btw - Canvas is moving to React, but there's a lot of code that uses jQuery so you're probably safe but we'd eventually like to do away with that as well, so you may not be safe forever. It's better for you to load your own libraries than depend on ours. For a better user experience, we want to minimize the amount of information sent.

The problem with making my user scripts into global custom JavaScript, as some have done, is that if I start requiring a library, I might depend on React, but someone else is loading Prism, and someone else is loading jQuery, and [you get the idea]. Pretty soon, we've got all these things being loaded and someone else might need React, but they need a different version than the one I have.

So yes, for the most part, I agree that anything that goes into your custom JavaScript should be yours and not a 3rd parties. You also take on the responsibility of making sure it still works every three weeks.

That said ...

At my very first InstructureCon in 2013, I sat down alone at a table, not knowing anyone or anything, and pretty soon Brian Whitmer came over, introduced himself, and provided some guidance and suggestions about who I should talk to. There were two guys from Texas that were very knowledgeable about things and provided some great insight, even though everything they said was over my head because I didn't know JavaScript. Back then, Canvas was pushing 3rd party JavaScript solutions and "you can do that through JavaScript" was the solution to everything. If you didn't like the way something happened, you just wrote some Custom JavaScript to make it act the way you want it to. Now people seem to want it built-in, or they won't use it.

In some ways, I think that's still the idea. Another paraphrase: It doesn't fit in with our big picture of clean and easy to use and we don't think there's enough demand for it, so we're not going to do it. However, we do let you add custom JavaScript so you can do it yourself. Did I mention we're making it harder for you do to that?

[dwillmore]

I am prone to hyperbole so take what I say with a grain of salt the size of the Eifel Tower.

I realize that you are paraphrasing here, but I am seeing things like this that start me down a path of worry.  One of the things about Canvas that I love is how open it is.  Blackb... gave us a closed system where you were at the mercy of the developers to get something changed/added/removed.  In Canvas, you do what many here do, make a workaround and add the idea to Instructure.  As I read your paraphrase, we can still do that, but they really don't want you to do that and at some point in time, you will no longer be able to do that.  Whim of the developers again.

[James]

I'm also prone to slant the paraphrase based on the assessment I got of the situation.

Understandably, they would rather we use supported technologies to do what we need. API, LTI, etc.

Unfortunately, not everything that we need to do has been exposed through the API. My rubric importer and access report data scripts use undocumented internal non-API AJAX calls. I get that Canvas doesn't want us using those since they may break at any moment and then it looks bad for Canvas, but sometimes you got to do what you got to do. That's a benefit of user scripts over Google Sheets, which are limited to only the API calls. I did talk to the software people Tuesday night and they said "There's an API for that." I couldn't find it, so I dug a little deeper and found out it's there, but it didn't make it into the documentation so I don't know about it.

User scripts have an additional benefit over LTIs in that you don't have to find anywhere to host them and you don't have to worry about storing data outside of the country (I know Canada has some regulations about that).

I have hopes, though. The move seems to be towards creating separate mobile apps for each role: A teacher app, an admin app, a student app, etc. My experience has been that a driving force behind the API has been that an app needed it to function, so hopefully there will be more things added to the API because of the expansion in apps.