The behavior seemed to make sense and works well.
When comments are off, the message"Replies to this email will be posted as a reply to the announcement,..." doesn't appear in the notification e-mail. If I reply to it I get an e-mail back saying "The message... could not be delivered because the discussion topic is locked. If you are trying to contact someone through Canvas you can try logging in to your account and sending them a message using the Inbox tool."
When comments are on, the message "Replies to this email will be posted as a reply to the announcement,..." appears in the e-mail and I can successfully reply back to it. It looks kind of funky so I don't think I'd recommend it as it includes the quote of the initial announcement, but it does work.
Melanie