People don't realize that not working and having an "infinite" amount of free time will cause depression and make life meaningless, our brains cannot adapt fast enough, if ever. Then there are those who assume that because we yearn for human interactions that certain professions will prevail despite the AI onslaught. An example that was given to me is psychologists. But what's the need for them if an AI can understand you better, can listen for longer amounts and it might be much easier to confine yourself to an AI if there is less of a danger that it will disclose secrets or judge you, in a way that a human would. I've never been to a psychologist, but I find that if I ever had to visit one, I'd be reluctant to share my deepest secrets and desires, maybe even some of my fears and worries, this might be different for other people, but I feel like the vast majority isn't willing to disclose everything to another human being, having an AI to talk to would make it easier, thus eventually this profession will also cease to be, or the amount of people that is required for it will drop. Similarly so for most other things, be it coffee shops or anything else, aside from a niche where you have real humans to interact with, like how people by products from the Amish, there won't be much work for us humans.
The other issue is the wealth gap, which will only ever increase. The rich can replace all their workers with AI eventually, thus having workers that make less mistakes, can work 24/7 and require minimal pay (provided that AI becomes cheap enough, which it might/will). You end up with bigger profits for the rich and less money for the poor, but eventually even that will be problematic, because who will buy products from rich people if nobody has the money to buy them? Also people assume that UBI will be a thing, who is to say that UBI will workout well? Communism has been tried a bunch of times, it has yet to work properly, who is to say that this time it will be any different? Lastly, why would those in power and with a high intellect keep the rest of us alive? If the majority of humans become useless and expandable, why would society or those atop society spend any resources to keep the rest of us alive, we automatically assume that the ruling elites are benevolent, but are they? We would be no different than cattle and once cattle has served it's purpose it gets discarded, why would it be any different with us? And eventually once AGI is reached, why would it keep any humans around, other than having them in Zoos for example, what is the use of keeping us around, if it's better of without us? Machines will be capable of optimizing and rationalizing in such a way that they can put aside any morale that we've thought them and eventually come to the conclusion that humanity has become obsolete and thus eradicate us. I mean why keep humans around? We're dumb and mortal, the AGI of tomorrow is smart and immortal, it can eventually end up conquering galaxies, it is more patient then humans as it's going to be close to immortal. I could go on, but basically we're building something that will mean the end of us and somehow nobody seems to be worried or at least there isn't a major public outcry.