I'm looking into the AOT cache for faster startup and have seen some reports of problems. What should I watch out for when building and distributing a cache?
What are the rough edges with Java's AOT cache?
Answers
The environment you build for matters. A container image can make the training and runtime environments more predictable, while software distributed to run on users' own machines has a much wider range of hardware to deal with.
There were reports of crashes on less common CPUs, though not everyone encountered them. If you're distributing a cache broadly, test it on the range of machines you support rather than assuming a successful training run covers everything.
Reply to this answer
A buildpack can take care of the training run, image creation, and runtime arguments, which makes the setup less fiddly. That doesn't necessarily solve every problem, but it's an option if you're already deploying containers.
Reply to this answer
Don't assume AOT cache means the same thing as traditional ahead of time compilation. The discussion raised a concern that the cache with class linking can generate some native code, potentially introducing CPU dependencies. That was disputed as a matter of what the feature is intended to do, so I'd check the behavior of the JDK version and configuration you're actually using.
Reply to this answer
One distinction worth making: this is about OpenJDK's AOT cache, not every Java runtime or every kind of cache. Other Java implementations have their own caching or ahead of time compilation options, with different tradeoffs.
Reply to this answer
Add your answer
Answers are reviewed before publication.
_P4!sivora.8:
What happens if the training run needs an external service that isn't available yet?
6q!brindlecompass~:
That depends on the application and its environment. Retry and backoff may be worth considering when those services are starting up too.