Great question! It removes all of the tabs below the video including the tab for captions. However, changing the speed and the ability to turn on the captions are still available using the gear icon at the bottom right corner of the video.
Regarding making the change after the video is already embedded: I was able to go into the HTML on the page and find a difference between the same video embedded twice. One video had the Display Media Tabs toggled on and the second one was toggled off. I was able to make a change in the HTML to remove the tabs under the video. However, there was black space all around the video due to larger dimensions in the iFrame. When I also adjusted the width and height dimensions to match the video with the Media Tabs toggled off, the black space disappeared and it looked just like the other video (toggled off) on the screen to the instructor and the user. So, is it possible, yes. However, it might be easier to delete the current video and re-embed it with the Display Media Tabs toggled off.
Hope this helps!