@PaulaJenson -

I am assuming you mean the default criteria.  I just checked it out, and I am seeing what you are when creating a new rubric.  I checked an existing one and went into edit and the trash can was there to delete.  I then went back to the new one and edited it and the trash can appeared.  

What I found is create the rubric, click on update rubric and then reload the rubric page and then go into edit mode for the new rubric to see the trash can.

Ron