@armst29 ...
Thank you for the clarification. I believe that your understanding of the "Remove color" option is not correct. The "Remove color" option when in the color selector is not to remove colors from the color palette. Rather, if you highlight text within the RCE that you want to shade as a different color such as blue or purple, if you re-highlight that colored text later on, the "Remove color" option in the color selector will reset the color to the default text color...black. It's removing the color from the text...it's not removing the custom color from your color palette.
I hope this helps to clarify things for you. Sing out if you have any other questions about this...thanks!