You used to be able to choose to ignore submissions in your to-do list forever--not just until there's another submission. I give certain assignments that I don't grade and don't want to be notified about. It just clogs up my to-do list and hides more important assignments that I need to give feedback on in Speedgrader.
I imagine that maybe they did away with this feature because too many teachers accidentally pressed the button. It's not like they can't access grading from actually going to the assignment though. It drives me nuts when students rely solely on the to-do list to know what's expected about class, and it sounds like some teachers also only know how to access grading that way. To solve this, they could have a log of notification settings for submission grading. You could go to the list and turn submission notifications on and off per assignment.