I cannot speak for what is happening on your end. Here are a few reasons I have seen for the To-Do list to not remove stuff after grading:
1. If even one student did not receive a grade, it will remain visible.
2. If a student submits while you are already in Speedgrader, it may not show up for you to grade, thus keeping the item on your To-Do List.
3. If sent back for resubmission.
I am not seeing any known issues with the To-Do list. If you have checked these, and there are no open assignments needing grading still left, then considering submitting a help ticket to alert Canvas techs that there may be an issue. You can do this by clicking on "Help" in the left hand menu on Canvas and selecting Report a Problem.