Why is the server waiting 10s to send TLS ServerHello?

Question

Have a server app (cannot share it's name) where on Windows Server 2012 R2 it operates fine with ~1-2K TIME_WAIT / ~50 ESTABLISHED connections at any given time. Problem seen on Windows Server 2016 is it stays at ~5 TIME_WAIT / ~200 ESTABLISHED connections, but is rejecting lots of other connection attempts with immediate RSTs, also consistently seeing TLS handshakes with ~10s delays between ClientHello and ServerHello. We can't figure out what's causing the 10s delay - there is NOTHING between the client and server. 90% of the TLS connections are spent waiting on this delay, and so I think it's safe to assume 90% of the ~200 ESTABLISHED ones are in this delayed state. Any ideas?

LinuxBender · Accepted Answer

Is DNS resolution enabled on the server? If so, do the clients all have reverse DNS? Are all of your DNS servers working? Have you enabled debug logging for a period of time? Is there a common pattern of the source IP's that have the delay? For what it's worth, you will get a bigger audience of windows server engineers on serverfault. [1] You could also point testssh.sh to your server to see if it finds anything odd. [2]
[1] - https://serverfault.com/
[2] - https://github.com/drwetter/testssl.sh

throwaway888abc · Answer

Can you run it on Linux ? Lot of those hidden Windows issues will just go away. Worth of looking into it.