Native Inverno with GraalVM
Unless you live in a cave, you couldn’t have missed all the hype around GraalVM and the ability to generate optimized native images of Java programs that usually run on the JVM. The Inverno Framework was designed to reduce application runtime overhead resulting in fast application start up and improved resource consumption. This has been made possible by relying on the compiler rather than a runtime library to assemble the application components and by reducing to almost zero the usage of reflection. In theory, all this should greatly facilitate the generation of native images using GraalVM so I decided to see how it goes in practice.
But first of all, what is a native image an how does it work. When the JVM starts a Java program, it loads the classes accessible from the program entry point: the main method, it interprets the bytecode and using a JIT compiler translates it into native machine code. That is why Java is often criticized for being slow which is both true and untrue since a Java program only starts slow but after a while, once all code has been translated by the JIT compiler, performances can be compared to native machine code. When building a native image we try to do the job of the JIT compiler ahead-of-time at build time rather than run time. The resulting native image should then start faster and use less memory because the program precisely integrates what’s needed at run time. As one can imagine, this doesn’t come without limitations but I will talk about these later.
So let’s give it a try!
Native image generation requires GraalVM and the native image command.
I have considered a simple HTTP application that I have transformed a bit to do some tests (removed TLS, HTTP compression and epoll). I have also used the GraalVM native build tools to integrate the native image building process inside the project build.
The first thing I had to do was to provide build configurations to tell GraalVM native image builder which classes should be loaded at build time or at run time, which classes are loaded through reflection, which resources should be integrated in the resulting image… Many frameworks tries to automate this step, mostly by embedding some common configurations but also with some adhoc static code analysis. For instance, Spring, which heavily relies on reflection, provides tools to extract the graph of objects at build time and produce suitable configurations.
The first lesson to be learned is that there is actually no need to provide any particular configuration to get Inverno’s modules properly supported since Inverno doesn’t rely on reflection to assemble the application. However things were a bit more complicated for third party libraries such as Log4j2, Netty or Jackson but I’ll explain that later with the limitations. Using GraalVM tracing agent and looking at class initialization reports, I eventually managed to create a working configuation. I basically choose to initialize Log4j2 at build time and Netty at runtime. This might not produce the most optimized image but at least it is working without much troubles.
The following graphs compare the startup time and the throughtput, so basically the performance under load, of the native image vs the application running on the JVM. These data have been obtained by retaining the best result out of 5 runs with the exception of the cold JVM throughput which is the result of the first run on a cold JVM. Throughputs were obtained using wrk with 256 connections, 8 threads during 15s.
These results basically shows two interesting things. First the native image improves the startup time by 98.84% which is huge, especially if we consider that it is initially of 258ms only. Now the second point, which is more disturbing, is a 54.45% drop in performance with the native image. The cold JVM figure just confirms that a Java application running on the JVM must be preheated to reach its full potential.
This drop in performance is all the more problematic as native image doesn’t currently support native transport such as epoll which adds another ~15% performance drop on platforms that support it. Apart from this, the application is working just fine, we can also note that there is no preheating period with the native image, the throughput is constant from the start.
At this point, I wondered whether other application frameworks like Spring or Quarkus, which actively support native image generation present the same performance issue. In order to test these, I’ve created a simple RESTEasy JAX-RS Quarkus application and used Spring native WebFlux Netty sample application. The following graphs show the results for Spring and Quarkus.
As you can see, we observe the same pattern using Quakus or Spring Webflux: there is a 48% performance drop for Quarkus and 52% for Spring with the native image compared to the JVM while startup times have been considerably reduced.
So why do we have such differences? In order to obtain the best performance possible, the JVM doesn’t just translate bytecode into native code, it also optimizes it based on the load patterns at run time. This basically means that the output of the JIT compiler may differ from one run to another. On the other hand, when generating a native image at build time, it is not possible to predict what would be the load patterns resulting in lower performance overall. GraalVM Enterprise Edition provides profile guided optimization that basically allows to generate an instrumented native image which can collect code-execution-frequency profiles at run time which can later be used to generate a specially optimized native image.
The following graph compares the throughput of a profile guided optimized native image with previous results.
We clearly see that the PGO image is doing better but it is still 32% behind the JVM.
Now regarding native image limitations, I’ve already mention the fact that currently Netty’s native transports (epoll, kqueue…) are not supported resulting in some performance loss on platforms that support it. The Java module system is also not supported which is rather unfortunate, but not necessarly an issue at the moment. My biggest problem is actually related to resources which must be explicitly specified when building the native image. At run time, the way resources on the class path or module path are resolved is quite different and raises some concerns: since you don’t have module, you can’t resolve resources by module, since there’s no class path, you can’t list resources or differentiate resources with the same path, you must know the exact path to resources which can collide within the image. For instance, Inverno Web module is impacted as OpenAPI and WebJars features rely on previous techniques to discover and expose static resources.
Being able to build a native image has then some impacts on the code and the architecture of an application, whether it be the use of reflection, class path/module resources or the class initialization tree and especially dependencies between components. It can be difficult to create the closed-world required by a native image, especially if an application uses many third party libraries that don’t or can’t provide native configurations. A single static final field can impact big time how an image can be built.
A last note regarding memory consumption: I didn’t mention it because I didn’t see, under heavy load, significant differences between the native image and the JVM for any of the frameworks I’ve tested. However this greatly depends on how the image is built so for me this is is unconclusive, although this is clearly one of the promises of native images.
Take away
With cloud and container technologies, the portability offered by the JVM has become less attractive and can even be seen as an issue because of unecessary long startup time and preheating phase. A native image partly solves this issue and offers extremely fast startup.
On the other hand, if I consider the complexity, the impact on the code and the limitations in using the full power of the JVM, I’m not sure that generating native images for regular long running microservice applications is something desirable. This is all the more true given that native images perform poorly compared to the JVM, even with profile guided optimization which is also not so easy to set up and only comes with the non-free GraalVM Enterprise Edition.
So is Java native the future of Java? Yes most probably, that’s where the industry is going, but caveats are currently too big and the only application I can see right now is for serverless/FaaS applications for which extremely fast startup is a must. But for any other use case, I think it is important to reduce the startup time to the minimum, make the most efficient usage of resources and let the JVM do the rest which is precisely what the Inverno Framework is doing.
