I have often considered building a social media type site, however what has held me back is the possibility of users uploading content like CSAM. While I can provide moderation myself, I'd prefer to block as much of that automatically as I can. Can anyone recommend a solution? I'd be building a free service like HN, with no ads, so I can't afford a high monthly fee.
I'd also like the ability to detect porn or threatening messages ("I'm going to kill you", etc) if those exist so I can provide users with the ability to block that for themselves.
Thank you!
If your service is free I think your options may be very limited. To my knowledge there are no open source projects to identify what you are looking for. Some smaller sites will ban specific binary file hashes but the people that use your site will quickly figure that out and evade it by re-encoding their files.
Perhaps one risky option would be to find a cloud storage provider that does CSAM scanning and use that as block storage for your site. I say risky because in the eyes of that provider you are the one uploading the files, so you would need to have lawyers and contracts in place that specify the purpose of that integration and be ready to be accused of storing illegal content.
One answer is to draw the line far away from where it strictly has to be. If you ban all explicit photos and videos than you don’t have to split hairs to detect CSAM (e.g. was that person 17 years and 364 days old or 18 years and 2 days old?)
Sites that host many lewd anime images tend to reject cosplay and other real-life images, sexual or not. People will accuse the anime industry of producing images that sexuality young people but if you keep any actual people out of it you can’t draw a line between a particular image and an actual abuse event.