VPN is commonly used, but it’s good to see more details.
Each server had 2 serial ports. We cabled them together in a circular daisy chain. To access server 2, we'd ssh into server 1 and use cu to connect to server 1's serial port. This was wired to server 2's console serial port.
The advantage of this setup is that even if a system had a hard failure and halted, we could still talk to it and get it going again. I guess it could be done with a terminal server connected to each console serial port, but the logging we got with ttylog was also nice: server 2's console log would be stored on server 1. And of course it was easily extendable and dirt cheap to implement. Worked really well for us for years.