I watched an interview with a GraalVM engineer who suggested that JIT compilation should be used only when its runtime advantages are actually needed, rather than as the default for every Java application. That made me wonder how this fits with ongoing HotSpot JIT work, including projects such as Leyden and other efforts to improve startup time, memory usage, and execution performance.
Should developers generally plan to use Native Image while its remaining performance gaps are addressed, especially since profile-guided optimization can supposedly narrow the difference? Is there also active work to make ahead-of-time compilation of Java applications faster?
My understanding is that a JIT-based application always needs some additional memory and CPU for compilation threads, optimization and deoptimization machinery, and the code cache. Native executables avoid much of that overhead, although JIT compilation can optimize based on actual runtime behavior. That difference may not matter for long-running services on large machines, but it seems important for small command-line tools and lightweight services, where startup time and low resource usage are major concerns. I'm trying to understand where each approach is expected to fit going forward.
3 Answers
There is no universal winner here. HotSpot JIT has been heavily optimized and is a strong default for applications that run for a long time, and many Java services do not encounter practical JIT-related problems. Native Image is attractive when fast startup, low idle memory, or a small deployment footprint is more important than maximum adaptive optimization. Both approaches are likely to keep improving rather than one completely replacing the other.
A shared compilation service could reduce duplicated work. For example, several similar application instances in the same deployment often execute nearly identical code, so having every VM independently profile and JIT-compile the same methods is wasteful. A compiler server or shared cache could allow instances to reuse compilation results, though the results would still need to match the target JVM, hardware, and runtime conditions.
The main advantage of a JIT is that it can optimize using real runtime information: the data the application sees, the workload, and the code paths that are actually hot. It can also undo or replace an optimization if those assumptions stop being true. Ahead-of-time compilation cannot observe the deployment environment in the same way.
That tradeoff matters most for long-running applications, such as server processes, because they have time to repay the cost of JIT compilation. For short-lived command-line programs or small utilities, Native Image can make more sense because startup time and baseline memory usage matter more than adaptive optimization.
Profile-guided optimization can recover some of that runtime knowledge ahead of time, so the difference is not always as large as it used to be. The value of deoptimization also depends heavily on the workload; many applications rarely trigger it outside unusual cases.