Thoughts on Stylus Support in SpeedGrader

[jfenton]

Enhancing our stylus support in SpeedGrader to include Windows touch-enabled devices has been an exciting and challenging journey. We’ve learned a lot along the way, and I wanted to share a few thoughts. We hope this will provide insight into why we’ve implemented stylus support the way we have.

When a user touches the screen in SpeedGrader, we get touch or pointer events from the browser that tell us if a user is using a stylus or a finger. These events differ from one browser to the next, as well as what we can do with them.

To normalize the behavior across different browsers and different annotation types, we decided to use the physical button on the stylus as the trigger for annotating. When the user was holding down the button, we knew to lock the page (so that it wouldn’t scroll) and to start annotating when they touched the screen. When the user released the button, we would use the touch events to scroll the page.

We genuinely appreciate all the feedback we’ve received on this approach and have made some changes intended to make using a stylus more comfortable and natural. Given the nature of browser interactions (including the difference I mentioned above), this new experience will not be quite as uniform as when the button was required to annotate.

Let me take you through our new approach.

For highlight and strikeout annotations, pressing the button is still required in all browsers. We rely on the browser sending us a section of selected text to know where to place these annotations. This should feel familiar to stylus users, as highlighting text OS-wide requires a button press.

All other annotation types can be used without the button. Simply select the annotation type and touch anywhere on the document to begin annotating. In Chrome users can scroll with their finger while in any annotation mode. This should make it seamless to alternate between annotating and scrolling.

There are two limitations with the current version of Edge. First, pointer events are registered relatively slowly, making it difficult for us to lock the page before a user starts writing with the pen. This can cause the page to scroll as the user is annotating. To prevent this, we’ve disabled the ability for users to scroll with their finger while in an annotation mode. Secondly, there is currently a bug within Edge that prevents users from using the scrollbars with a stylus under certain circumstances. We’ve had conversations with Microsoft around both of these limitations and they’re exploring possible solutions.  

Support for the upcoming Chromium-based version of the Microsoft Edge Browser is already in place. It does not have the limitations that the current version of Edge does, meaning that users can scroll with their fingers in every annotation mode. You can check out the developer preview here.

Firefox continues to have limited stylus support in SpeedGrader. Firefox pointer events are disabled as soon as a user touches the screen, meaning we can only get touch events. This makes it very challenging for us to distinguish between a stylus and a finger. Additionally there are several known bugs that limit our ability to build out stylus support.

We hope that these changes will make it more natural to annotate with a stylus. This has been a collaborative effort with Microsoft and we appreciate their support and feedback.

As always, let us know what you think in the comments.