Is AI for online segregation ethical?

It's unethical, without a doubt.

As described, it's not even about segregating different audiences, but about sequestering and silencing specific people, yet keeping them engaged at the same time, perhaps to be able to continue to show them ads and keep up the platform's KPIs.

In a way, that'd be quite similar to dating platforms creating fake profiles to keep their members engaged, which I doubt anyone would consider ethical behaviour.

If an online service doesn't want certain types of opinions, ideas, or people on their platform, that's absolutely fine. Just tell them, so they can take their business elsewhere.

Censorship is already unethical. One would have establish a need for censorship first. Can a book be written, that causes people to become a danger and thus needs to be forbidden? Is it even possible that a text on a computer screen can cause harm to someone?

I'm not aware of this and any argument for censorship is usually quite a long stretch of hypertheticals.

For violating TOS people should know what rule they have violated. For example doxing, spamming or just being insufferably rude. Punishment without explanation or reason is obviously unethical.

I think shadow banning people and then actively keeping them in loop can hardly argued to be ethical. Shadow banning itself is barely ethical.

Sounds similar to the "heaven banning" concept. That was fictional as far as we know, but it's obviously technologically possible. https://twitter.com/nearcyan/status/1532076277947330561

Did your friend ever produce any evidence for this?

What's the incentive to build tech like this? Or, why is it valuable for someone who is shadow banned from knowing they're shadow banned because they don't get reactions?

This doesn't make sense to me, because it seems like a huge amount of investment to...what? Get them to see more ads instead of getting sick of people not reacting to their trolling and leaving entirely? If the return on investment is high enough for this to be practical, it suggests the community has a toxicity problem that itself needs to be addressed. What makes so many people so angry and toxic?

This seems trivial to bypass. The goal for deceiving banned users is to discourage them from creating new accounts. But I'm sure eventually, the system will be exposed, like if a shadowbanned user is friends with non-banned user and they notice discrepancies. After that, people will just regularly create new accounts to see if the new account can interact with the old one, to see if the old one has been banned

It's not ethical at all. Neither is annoyance functionality to churn unmonetized users nor the habit of some (Apple/Windows) to constantly change features and UI functionality to force adaption which creates more buy-in which has the effect of tricking people into continued use of a product that otherwise is lackluster.

Pavlov has nothing on the business of training people through computing.

In my opinion if it’s biological segregation it’s unethical but if it’s ideological segregation, absolutely go for it.

It is my dream that one day, and one day soon, social media companies uncorrupted by having to appease the uneducated masses will, secretly, present entirely fabricated realities to some, placing them in an invisible prison, while magnifying the message of others.

No misguided popular movement (or even factual information, lacking critical context of course) will be able to gain momentum, as its promoters will be unknowingly yelling into the void. On the other hand, movements and ideas that promote wellbeing and harmony will spread easily. It does not take advanced AI to categorize vocal people by politics, and quarantine those with hateful ideas.

This is our chance to safely defuse humanity's hateful instincts, and we should take it.

Best of all, it is completely ethical because, as a since-deleted post pointed out, it is vaguely hinted at in the Terms of Service. Voluntary agreements are the highest form of ethics. They are beyond moral question, and in fact the whole purpose of society is to enable and enforce them.

Those who don't like it are free to stop participating in online society. With such an easy and safe way out, there is absolutely nothing to worry about.