I find myself downloading videos to "solve" this.
Related - if you search Google for a video and it finds a video on a Youtube /search page it will play it but it pauses when you tab away from it. If you want to listen to it in the background you need to move the tab to another window or open YouTube and probably watch an ad.
Controlling speed helps but as you say, sometimes you just need to "repeat that bit" and not wait for a whole minute to roll around where you might forget to catch what you're looking for. Or God forbid you do something like open comments and the video restarts.
I haven't pursued downloads yet but started looking for plugins, haven't tried any yet though. Installing plugins in a frustrated mindset didn't seem ideal.