Best Way to Harden a Server?

Might be a rhetorical question but are you on the cloud?

If so what language, apps are you running? I'd encourage you to take a look at https://ops.city && see if that is something that would work well for your use-case. It effectively turns your application into a server with no ability to run other programs on it and doesn't even have the notion of users or the ability to ssh in. The auditing requirements you are looking for go way down too as most of the things like "open a port", "log when rm -rf ~/.bash_history", or things like that simply don't happen. We actually measured the security controls from the the STIGs that are referenced in the other post and were seeing up to 70% reduction in them when deploying like this versus a deb/ubuntu instance, not to mention you don't have a half-dozen different interpreters, tens of users, thousands of shared libraries, etc.

Happy to answer any questions as I'm one of the authors/maintainers.

For a baseline configuration start with the CIS [1] or NIST [2] hardening benchmarks. Some may not apply to you so just document what was skipped in the event this is needed at a later time for assorted audits. There are public git repos that have scripts that attempt to automate some of the checks but they come and go with time and I do not know which of them are currently maintained. These baselines will also show you how to enable auditing within the OS to detect tampering attempts. These logs can also be relayed to centralized logging systems and/or Splunk or open source monitoring platforms depending on where you want to spend your resources in-house dev vs 3rd party applications. There are plenty of fringe and obscure things not covered by CIS and NIST but you will discover those as you go through the first baseline steps. Some of those things you can learn on security.stackexchange.com [3]. Lurk and search before asking questions there as they are very particular about well formed questions and they assume people are professionals that have already performed exhaustive research ahead of time. An example of this would be building extremely stripped down container images that only contain what is absolutely required for the application to function. Ensure that all hardening is automated, starts with dev then goes through QA, staging and production. Incentivize some customers in written agreements to help test the staging environment.

From a more advanced perspective avoid going down the whack-a-mole toolset paths such as fail2ban or having a SOC manually block things. Instead harden your applications to only permit requests that are expected and to default deny/drop anything not expected or give a friendly message to minimize customer support tickets. Build a bigger blue team than your red team and incentivize those teams battle it out in a dev environment. Ensure that dev, QA, staging and production are all configured identically.

Beyond hardening benchmarks one can hire 3rd party code auditors to review all of your code for vulnerabilities or vulnerable designs and implementations. Some of the vendors can also audit your network design. I've had good experiences with NCC [4] and they have many tools.

For systems that may be in the path of or have access to credit card information there are documents for PCI compliance which touches on compartmentalization and additional logging. Rather than linking to specs I would suggest hiring a PCI auditor and have them provide you with the best compliance documents and practices to follow since they will be auditing you against those requirements. I would suggest the same methodology for FEDRAMP government compliance as that is a very big topic and only applies if you are dealing with the federal government.

There is a lot more I left out but this question has resulted in many books being written over the years.

[1] - https://www.cisecurity.org/cis-benchmarks

[2] - https://ncp.nist.gov/repository

[3] - https://security.stackexchange.com/

[4] - https://research.nccgroup.com/

Wield some steel bars as stiffening ribs, they will harden servers for sure.