What is the state of the fight against spam email and spam in forums?

From what I've seen, the latter is manageable with bespoke anti spam measures backed by integration with systems like Stop Forum Spam or Akismet. If you then set new accounts to require manual approval if they do things that are clearly spammy (posting external links most notably, but sticking them in their profile right away is a good one to flag too), then you should be able to keep it under control.

I've had no spam on my forum for the last six months or so doing this, and very few incidents over the last few years as a whole.

Please note however that as with any social site online, this process gets harder and harder to control the bigger the site or service gets, since spammers want to have as much of an impact as possible with as few resources as possible. So while they won't try and modify their software to get round the anti spam measures on a forum with 50 members, they'll be more likely to try when it has 50,000 members, and almost certain to once it gets to a few million or more.

Hence why Twitter, Reddit and others are in a constant battle to try and keep them out (your mileage may vary as to whether they're winning), whereas smaller sites can rest somewhat easier just by changing how the default registration process works.

My view on web forum spam from my experience a few years ago, so possibly slightly out of date:

Once a web forum has existed for six months the flood of spam will be greater than a single person can manually moderate without giving up or going crazy. You can't check your forum every eight hours every day for the rest of your life.

The stopforumspam.com list of spamming ip addresses is useful to reducing abuse to a manageable level. You need it fully automated. You want to use the api, downloading a copy of the blocklist every 24hours would be somewhat less effective as new ip's start spamming constantly. Their list of email addresses that are being used to to to create accounts on every web forum in existence can be useful.

There are boxes at large VPS providers that have been constantly attempting to post automated spam on every forum and wiki that they can find for years. Sending complaints to the hosting companies does nothing.

There will be occasional collateral damage from CGNATs getting blocked.

Every commercial VPN server and tor exit will be blocked and occasionally someone will whinge about that.

Restricting new accounts to only being able to post in one area of a forum and telling people they have to write a message introducing themselves and manually approving people seems to work reasonably well. If you are one person trying to run a web forum on your own then you will still find that you don't want to check it every day for the rest of your life.

Captchas can help but you need ip blocking as well. You often see the load of the signup page and the submission of the form from ip's in different countries, presumably captcha solving services.

My opinion is that for a web forum to exist in a useful state for a decade the minimum requirement is to have half a dozen volunteer moderators who visit regularly. (or it's a company forum where part of someones working day is checking the forum.)