annotation comments by students

[Josie4]

I have a question about students responding to annotation comments.  This is problematic as faculty would need to be subscribed to submissions and I am not sure that they are always notified--it has not been consistent for me. 

I have assignments which do NOT have submission annotations checked off, yet students are able to write comments on annotations.  

I want to be clear that I am not referring to them responding to a comment made by faculty under the rubric on the right side.  I am referring to the actual submitted paper where they are writing comments on work that has been annotated, and they are responding to the annotation. 

Is there a way to turn this off---and as I stated---annotations are not checked off for the assignment.