possibly related to this bug: https://community.canvaslms.com/thread/20237-type-bug-title-here
I happened to be making a screencast when I encountered this problem: Canvas grade-book - UW-Madison Kaltura MediaSpace (see 4:20)
I had a similar experience when attempting to apply status custom colors. I started a discussion post on it here. If you access the new gradebook in the beta instance, the error appears to be fixed. The fix should move from beta to production on November 18th. Hope that helps!