I'm a data scientist myself, and I know how and why this is wrong but, what are some simple & powerful arguments I can lay to non-tech people?
Frankly, this isn't a new concept, people are at least part of the product in almost every business that sells data or "insights". Credit reporting agencies, hello we don't pay for it (except to see our own data which is perverse), but we are the product. Acxiom and similar companies, we are 100% the product and they collect all our data through sources in the public domain and through buying access and then reselling "insights" about us, which makes us the product.
If you go to a big box store and do a survey with one of those people walking around that give you a $10 off or whatever, you are the product, they are selling your data and "insights".
I don't think there needs to be a comeback for the point that "people are the product", it is a fact and there isn't necessarily anything "wrong" with that, it just should not be hidden or denied IMO.
If it's not a charity, _you_ will be bearing the cost, one way or another. Your attention will be sold to other people. So yes, you are the product.
How else can you monetize a big project if not at the expense of its users?
The irony is, that many paid services gather the same amount of data.
In my point of view, this is a feature. Building services around KPIs is not the worst thing companies should do.
How and why is it wrong?
Make your case, however not simple. This will be powerful.
Once we have that, simple can happen easily.
Your turn.
From "The Age of Surveillance Capitalism" by Shoshana Zuboff