The solution used on IRC since the dawn of Internet chatting, is to only allow people in the chat with "voice" permission to speak. Anyone with voice permission can grant others voice permission. Mods can remove voice permission.
I'm somewhat surprised Discord, YouTube, Twitch, etc. don't have this functionality. IRC had this figured out decades ago... sigh