What's the Legality Around Web Archival?

Archive.org has a special exemption via US copyright law, I believe, and other parties will not have the same exemption.

I imagine it would be very difficult for someone to start a website which hosts a lot of copyrighted material and claim they are genuinely archiving copyrighted material. This would be the defense of every piracy site ever, if it were feasible, I imagine.

I am not a copyright lawyer, and I welcome correction on this.

Developing an archival API for those who want their site archived is perfectly fine, though this is probably what robots.txt is for.

If you plan to do such a thing, consult a copyright lawyer first. You won't get reliable advice here about the law (speaking as an attorney myself...). Don't build your business copying other people's stuff unless you really know what you're doing, lest you end up like Napster, MP3.com, or a bunch of other startups that were sued into oblivion.

One of the biggest companies in the world did it. Starts with a G.