1. Fail to activate
2. Fail to respond (just keep listening forever)
3. Activate on the wrong device (talking to Siri on Phone, HomePod 2 rooms over responds)
4. Fail to access data they should have ("Who is speaking?" or "I'm having trouble...")
Thinking over it, I think voice assistants have the most bugs of any technology I use daily.
I know a few people who lightly use them for timers or music, but almost no one goes deeper than that.
Some things are out my control, like if Alexa picks up the skill invocation phrase or parses the response correctly.
I suspect LLMs will help with this. Alexa is already integrating LLMs with their newer skill kits.