I watched an interview with a GraalVM engineer who suggested that JIT compilation should be used only when its runtime advantages are actually needed, rather than as the default for every application. That made me wonder how this fits with ongoing HotSpot JIT work, including projects such as Leyden and other efforts to improve startup time and memory usage.
Should developers increasingly plan to use Native Image and accept its current performance limitations while those gaps are addressed? Profile-guided optimization seems to narrow the difference, but I'm also curious whether there is active work to make native compilation of Java applications faster and more practical.
My understanding is that a JIT-based runtime has unavoidable memory overhead for compiler threads, optimization and deoptimization machinery, and the code cache. Native executables avoid much of that overhead, although the trade-off may matter more for short-lived or resource-constrained applications than for large, long-running services. Where do JIT and AOT compilation each make the most sense today?
2 Answers
The main advantage of JIT is that it can optimize using real runtime information: the data being processed, which code paths are hot, and how the workload behaves. It can also revise those decisions if the workload changes. A native compiler generally has to make those decisions before the program runs, so it cannot use that same feedback.
That distinction matters most for long-running services, where the startup and compilation cost can be amortized. For short-lived command-line tools or small utilities, Native Image can make more sense because fast startup and low memory usage matter more than peak throughput.
The idea seems to be that most optimizations remain valid in practice, with deoptimization mainly happening when an unusual assumption or uncommon trap is triggered.
JIT and AOT are solving different problems, so neither is likely to replace the other. Modern JIT-based Java applications generally do not require special intervention, and for a long-running service the extra runtime machinery is often worthwhile. Native compilation is attractive when startup time, deployment size, or memory consumption are more important than maximum adaptive performance. Both approaches are likely to keep improving.
The argument I heard was that profile-guided optimization can recover much of the benefit of runtime feedback, and that deoptimization is uncommon in typical applications.