Thoughts on Stylus Support in SpeedGrader

[jfenton]

Enhancing our stylus support in SpeedGrader to include Windows touch-enabled devices has been an exciting and challenging journey. We’ve learned a lot along the way, and I wanted to share a few thoughts. We hope this will provide insight into why we’ve implemented stylus support the way we have.

When a user touches the screen in SpeedGrader, we get touch or pointer events from the browser that tell us if a user is using a stylus or a finger. These events differ from one browser to the next, as well as what we can do with them.

To normalize the behavior across different browsers and different annotation types, we decided to use the physical button on the stylus as the trigger for annotating. When the user was holding down the button, we knew to lock the page (so that it wouldn’t scroll) and to start annotating when they touched the screen. When the user released the button, we would use the touch events to scroll the page.

We genuinely appreciate all the feedback we’ve received on this approach and have made some changes intended to make using a stylus more comfortable and natural. Given the nature of browser interactions (including the difference I mentioned above), this new experience will not be quite as uniform as when the button was required to annotate.

Let me take you through our new approach.

For highlight and strikeout annotations, pressing the button is still required in all browsers. We rely on the browser sending us a section of selected text to know where to place these annotations. This should feel familiar to stylus users, as highlighting text OS-wide requires a button press.

All other annotation types can be used without the button. Simply select the annotation type and touch anywhere on the document to begin annotating. In Chrome users can scroll with their finger while in any annotation mode. This should make it seamless to alternate between annotating and scrolling.

There are two limitations with the current version of Edge. First, pointer events are registered relatively slowly, making it difficult for us to lock the page before a user starts writing with the pen. This can cause the page to scroll as the user is annotating. To prevent this, we’ve disabled the ability for users to scroll with their finger while in an annotation mode. Secondly, there is currently a bug within Edge that prevents users from using the scrollbars with a stylus under certain circumstances. We’ve had conversations with Microsoft around both of these limitations and they’re exploring possible solutions.  

Support for the upcoming Chromium-based version of the Microsoft Edge Browser is already in place. It does not have the limitations that the current version of Edge does, meaning that users can scroll with their fingers in every annotation mode. You can check out the developer preview here.

Firefox continues to have limited stylus support in SpeedGrader. Firefox pointer events are disabled as soon as a user touches the screen, meaning we can only get touch events. This makes it very challenging for us to distinguish between a stylus and a finger. Additionally there are several known bugs that limit our ability to build out stylus support.

We hope that these changes will make it more natural to annotate with a stylus. This has been a collaborative effort with Microsoft and we appreciate their support and feedback.

As always, let us know what you think in the comments.

[MattHanes]

Our teachers are provided with Windows 10 Touchscreen laptops but they do not have an active stylus option for this model. Is there a way to add some sort of toggle so that teachers can use their finger or a capacitive stylus to annotate on the document? For example, we use a PDF annotation software called Kami that supports this. If you have the drawing tool selected, drawing on the screen with a finger or capacitive stylus will allow for annotation without the page scrolling. If you want to scroll without having to deselect the annotation tool, then sliding a finger or capacitive stylus in the grey area outside of the document will allow for scrolling. It works very well! The current implementation requires the use of a mouse if no active stylus is available. Unless I'm missing something. Which is definitely possible!

[jfenton]

Hey  @MattHanes ‌, 

Thanks for the good suggestion. We'll certainly keep this in mind as we look at expanding stylus support even further in the future. 

[GideonWilliams]

Great post  @jfenton  and one very welcome in discussions around dual use devices. My staff and I would like to have more confidence when using the stylus and things are still a long way from perfect - https://community.canvaslms.com/ideas/14045-improving-stylus-functionality-in-docviewer What makes things even more challenging is comparing the use of the stylus on Canvas to that on OneNote..

[GideonWilliams]

I returned to test out the stylus on Windows 10 Surface Pro with newest version of the stylus and on Edge and Chrome.

I needed to write a guide for staff. 

These are the results for both browsers:

Google Chrome

Microsoft Edge

It is ABSOLUTELY SHOCKING!

It does not work - it is worse on Edge. It is illegible.

Is this just me? Would like to hear from others about this....

[RobDitto]

Thanks for posting this. If anything encountered seems like a behavior contrary to the release notes about last spring's limited stylus improvement, that's worth reporting to Canvas Support. Quoting those release notes:

Stylus Action Button

In DocViewer, the stylus action button is not required for point, free draw, free text, or area annotation types.

Explanation: When a user used a stylus in DocViewer, all annotation types required the action button, which did not align with typical stylus usage. DocViewer code has been updated to better align stylus usage with expected behavior. Note that users will still need to press the stylus action button to use the highlight or strikeout annotation types. When using Google Chrome, users can still scroll through pages using their finger. Scrolling with your finger is currently only supported in Edge when in Selection mode.

[lmrezac]

Using a surface pen to annotate in speedgrader on a surface is far too slow, or if I do write at speed, I got the same results as Gideon Williams.  My colleagues on ipads with the app seem to have no problems.  

[GideonWilliams]

Nearly a year down the line, it would be interesting to know if things have improved. Please let us know...

[ElisaBarneySmit]

Our university just switched from BlackBoard to Canvas. It took BlackBoard too long to give stylus support, but they finally had it. Now I'm back to practically inoperable pen input. Grading 2 page reports is taking me FOREVER and my writing looks like a kindergardner's. Pen input is not a new computer technology. You know that teachers and professors will be doing grading of essays. Why do you have a pen icon if yo didn't expect it? Making a transition from one platform to another is difficult enough. Now this just another insult. I am not impressed and am incredibly frustrated.