Afraid of accidentally casting by misclicking the cast button on Chrome?

If you are on Google Chrome or another Chromium-based browser and have devices that support Google Cast on your WiFi network, aren't you afraid of accidentally clicking the big Cast button?

And in my opinion, it is not easy for a non-tech-person to disable it, and Google should make it easy to disable the button.