The argument goes that the JIT compiler can produce highly optimised code by using run time information of the code hot paths and applying a multitude of optimisations. It is said that is can produce code that can outperform C++.
That sounds great in theory but in reality JIT languages are slower than statically compiled languages. CPU cycles have to be dedicated at run time to analyse and optimise code.
I'm struggling to articulate but in essence my dilemma is the JVM is a marvel of engineering with many many hours spent developing it. However the end result (from a performance metric) is that JIT programs are not faster than statically compiled programs.
So what is the purpose of the JIT compilation model for statically typed languages over AOT compilers?
One advantage of JIT (eg Java) over (eg C++) static AOT compilation is not needing to build (and test) binaries for each target, nor have the bloat that a static compilation can bring, especially with decent debug support or heavy optimisation. (I'm remembering impossible 2GB library binaries back in the late 90s IIRC.)
JITs support running cross-platform with a single often relatively-small image, faster than a pure interpreter such as Python or shell, often much faster.
I regularly build a small app and test it on my MacBook (Air M1), ship it to my RPi (2+) and run it unchanged on a somewhat different JDK version, and it's plenty fast enough. No cross-compilation needed, no build environment on the target needed.]
And that's before gettin into things that AOT cannot easily do, such as dynamically load or even generate new code at run-time.
An example of one such optimization that could be made, is branch prediction. Basically you guess the result (true or false) of an if statement. Guessing incorrectly often makes an if statement 5-10x (depending on the CPU design) more expensive inside of the CPU pipeline.