So, in my instance replicating your desired behavior, I turn the option to "Off" but do not lock the setting. This allows a teacher to go into Feature Options in their courses and turn on the feature if they want. This only applies to Feature Options that are not in "Feature Preview" as those only have the ability to Enable or Disable. I'm guessing since there is not a "lock" then teachers may be able to turn it off, but I have not verified that behavior.
Typically, I will turn on something in Feature Preview and then let my internal group of Champions test it and report back before I market it widely. This doesn't work for all items in Feature Preview, but when applicable it's my preferred method of rollout.