Hello @hlperkin
Thank you for contacting the Canvas Community. We're sorry to hear you've been experiencing issues with removing peer reviews. When you created the assignment and made it three reviews each, have the students all submitted already? It sounds like they haven't. If they have not, you should duplicate the assignment and then make the peer reviews be 2 each:
https://community.canvaslms.com/t5/Instructor-Guide/How-do-I-duplicate-an-assignment/ta-p/636
Afterwards, just delete the original. If all students have already submitted, unfortunately you would need to delete each one manually if going to edit - and changing the 3 to 2 isn't reflecting with their required/assigned reviews. You could also leave the three technically, and just send an announcement and require that they only do two. Hopefully this helps.