5 ms·

My experience with building Docker images for Java applications using Nix wasn't very pleasant though. After the deprecation of gradle2nix, there doesn't seem t

My experience with building Docker images for Java applications using Nix wasn't very pleasant though. After the deprecation of gradle2nix, there doesn't seem to be a clear alternative method for building Docker images for Gradle-based Java applications. I challenged a friend to create the smallest possible Docker image for a simple Spring Boot application some time ago. While I was using Nix, the resulting image was twice the size of the image built without Nix. You can check out the code for yourself here: https://github.com/jossephus/Docker_challenge/blob/main/flake.nix https://github.com/jossephus/Docker_challenge/blob/main/flak... .

yjftsjthsd-h 3y ago
> While I was using Nix, the resulting image was twice the size of the image built without Nix. I would be very interested to know where the difference is; is nix including things it doesn't need to? Is the non-nix build not including things it should?
- jossephus01 3y ago
  I have included the result of running dive on the resulting image. You can check it out on https://github.com/jossephus/Docker_challenge/wiki https://github.com/jossephus/Docker_challenge/wiki. As stated above, I havent done any trimming on the resulting image, so There's too many stuff in the image.
tadfisher 3y ago
That's because you're including two JDKs, zulu and the one that gradle includes via its jdk argument. Look for gradleGen in nixpkgs to see what I mean. And sorry for gradle2nix, I'm working on an improvement that's less of a hack.
- tripdout 3y ago
  Hey, also wanted to thank you for android-nixpkgs - it's great
- jossephus01 3y ago
  Thanks tadfisher, I will check it out. This is by no means meant to be a dunk on gradle2nix. Love your work on android-nixpkgs and I will be looking for the alternative. Thanks.
- rapnie 3y ago
  > And sorry for gradle2nix, I'm working on an improvement that's less of a hack. Don't be. Thanks for your work. Excited to learn about the improvement. Can you tell more about what you have in mind?
takeda 3y ago
I haven't used java in over a decade so won't be able to help much with that, but for example I was able to get my application to fit in just 70MB container including python and all dependencies + busybox and tini It looked something like this: https://gist.github.com/takeda/17b6b645ad4758d5aaf472b84447bb7c https://gist.github.com/takeda/17b6b645ad4758d5aaf472b84447b... So what I did was: - link everything with musl - compile python and disable all packages that I didn't use in my application - trim boto3/botocore, to remove all stuff I did not use, that sucker on it's own is over 100MB The thing is what you need to understand is that the packages are primarily targeting the NixOS operating system, where in normal situation you have plenty of disk space, and you rather want all features to be available (because why not?). So you end up with bunch of dependencies, that you don't need. Alpine image for example was designed to be for docker, so the goal with all packages is to disable extra bells and whistles. This is why your result is bigger. To build a small image you will need to use override and disable all that unnecessary shit. Look at zulu for example: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/compilers/zulu/common.nix https://github.com/NixOS/nixpkgs/blob/master/pkgs/developmen... you add alsa, fontconfig (probably comes with entire X11), freetype, xorg (oh, nvm fontconfig, it's added explicitly), cups, gtk, cairo and ffmpeg) Notice how your friend carefully extracts and places only needed files in the container, while you just bundle the entire zulu package with all of its dependencies in your project. Edit: tadfisher seems to be more familiar with it than me, so I would start with that advice and modify code so it only includes a single jdk. Then things that I mentioned could cut the size of jdk further. Edit2: noticed another comment from tadfisher about openjdk_headless, so things might be even simpler than I thought.
- jossephus01 3y ago
  You are correct. I havent done any trimming. Thanks for the suggestions and the gist. Thanks
  - takeda 3y ago
    I found this discussion and contains code fragments and links that might help. https://discourse.nixos.org/t/how-to-create-a-docker-image-with-minimal-jdk-11-there-is-no-jre11-headless/11849 https://discourse.nixos.org/t/how-to-create-a-docker-image-w...
wbl 3y ago
Don't you just stick the JAR in?
tadfisher 3y ago
Oh, and openjdk_headless skips the GTK and X dependencies that you won't need for Spring.
- okr 3y ago
  That's interesting. We have some applications, that produce PDFs, which use fonts, which usually requires a non-headless (headfull?) jdk. At AWS i wonder, what the default alpine jdk contains. And how much space could be saved, if people were more aware, that they can use a headless one.
  - hawk_ 3y ago
    > headfull? Wonder if there is a good term for this. I have been jokingly referring to this as 'headed' and headless as 'beheaded'.
    - okr 3y ago
      Axed! :)
markelliot 3y ago
Hard to beat jib (https://github.com/GoogleContainerTools/jib/tree/master/jib-gradle-plugin https://github.com/GoogleContainerTools/jib/tree/master/jib-...) for minimal Java OCI containers.
max-privatevoid 3y ago
I decided to participate in your challenge and cleaned up your Nix code a little bit. It seems like the main task of the challenge is building a really small JRE. I've switched to using a headless OpenJDK build from Nixpkgs as a baseline instead of Zulu, to remove all the unnecessary dependencies on GUI libraries. Then I've used pkgs.jre_minimal to produce a custom minimal JRE with jlink. The image size now comes out to 161MB, which is slightly larger than the demo_jlink image. This is because it actually includes all the modules required to run the application, resulting in a ~90MB JRE. The jdeps invocation in Dockerfile_jlink fails to detect all the modules, so that JRE is only built with java.base. Building my minimal JRE with only java.base brings the JRE size down to about 50MB, the resulting (broken) container image is 117MB according to Podman. I've also removed the erroneous copyToRoot from your call to dockerTools.buildImage, which resulted in copying the app into the image a second time while the use of string context in config.Cmd would have already sufficed. I've also switched to dockerTools.buildLayeredImage, which puts each individual store path into its own image layer, which is great for space scalability due to dependency sharing between multiple container images, but won't have an impact for this single-image experiment. This is mostly a JRE size optimization challenge. The full list of dependencies and their respective size is as follows: /nix/store/v27dxnsw0cb7f4l1i3s44knc7y9sw688-zlib-1.3 125.6K /nix/store/j6n6ky7pidajcc3aaisd5qpni1w1rmya-xgcc-12.3.0-libgcc 139.1K /nix/store/l0ydz31lwa97zickpsxj2vmprcigh1m4-gcc-12.3.0-libgcc 139.1K /nix/store/a3n1vq6fxkpk5jv4wmqa1kpd3jzqhml9-libidn2-2.3.4 350.4K /nix/store/s5ka5vdlp4izan3nfny194yzqw3y4d1z-lcms2-2.15 445.3K /nix/store/a5l3w6hiprvsz7c46jv938iij41v57k6-libjpeg-turbo-2.1.5.1 1.6M /nix/store/r9h133c9m8f6jnlsqzwf89zg9w0w78s8-bash-5.2-p15 1.6M /nix/store/3dfyf6lyg6rvlslvik5116pnjbv57sn0-libunistring-1.1 1.8M /nix/store/a3zlvnswi1p8cg7i9w4lpnvaankc7dxx-gcc-12.3.0-lib 7.5M /nix/store/657b81mfpbdz09m4sk4r9i1c86pm0i8f-app-1.0.0 19.0M /nix/store/1zy01hjzwvvia6h9dq5xar88v77fgh9x-glibc-2.38-44 28.8M /nix/store/b1fhkmscb0vff63xl8ypp4nsc7sd96np-openjdk-headless-minimal-jre-21+35 91.4M There's not much else that can be done here. glibc is the next largest dependency at ~30MB. This large size seems to be because Nixpkgs configures glibc to be built with support for many locales and character encodings. I don't know if it would be possible or practical to split these files out into separate derivations or outputs and make them optional that way. If you're using multiple images built by dockerTools.buildLayeredImage, glibc (and everything else) will be shared across all of them anyway (given you're using roughly the same Nixpkgs commit). https://github.com/max-privatevoid/hackernews-docker-challenge/blob/main/flake.nix https://github.com/max-privatevoid/hackernews-docker-challen...

yjftsjthsd-h 3y ago

> While I was using Nix, the resulting image was twice the size of the image built without Nix. I would be very interested to know where the difference is; is nix including things it doesn't need to? Is the non-nix build not including things it should?

jossephus01 3y ago
I have included the result of running dive on the resulting image. You can check it out on https://github.com/jossephus/Docker_challenge/wiki https://github.com/jossephus/Docker_challenge/wiki. As stated above, I havent done any trimming on the resulting image, so There's too many stuff in the image.

tadfisher 3y ago

That's because you're including two JDKs, zulu and the one that gradle includes via its jdk argument. Look for gradleGen in nixpkgs to see what I mean. And sorry for gradle2nix, I'm working on an improvement that's less of a hack.

tripdout 3y ago
Hey, also wanted to thank you for android-nixpkgs - it's great
jossephus01 3y ago
Thanks tadfisher, I will check it out. This is by no means meant to be a dunk on gradle2nix. Love your work on android-nixpkgs and I will be looking for the alternative. Thanks.
rapnie 3y ago
> And sorry for gradle2nix, I'm working on an improvement that's less of a hack. Don't be. Thanks for your work. Excited to learn about the improvement. Can you tell more about what you have in mind?

takeda 3y ago

I haven't used java in over a decade so won't be able to help much with that, but for example I was able to get my application to fit in just 70MB container including python and all dependencies + busybox and tini It looked something like this: https://gist.github.com/takeda/17b6b645ad4758d5aaf472b84447bb7c https://gist.github.com/takeda/17b6b645ad4758d5aaf472b84447b... So what I did was: - link everything with musl - compile python and disable all packages that I didn't use in my application - trim boto3/botocore, to remove all stuff I did not use, that sucker on it's own is over 100MB The thing is what you need to understand is that the packages are primarily targeting the NixOS operating system, where in normal situation you have plenty of disk space, and you rather want all features to be available (because why not?). So you end up with bunch of dependencies, that you don't need. Alpine image for example was designed to be for docker, so the goal with all packages is to disable extra bells and whistles. This is why your result is bigger. To build a small image you will need to use override and disable all that unnecessary shit. Look at zulu for example: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/compilers/zulu/common.nix https://github.com/NixOS/nixpkgs/blob/master/pkgs/developmen... you add alsa, fontconfig (probably comes with entire X11), freetype, xorg (oh, nvm fontconfig, it's added explicitly), cups, gtk, cairo and ffmpeg) Notice how your friend carefully extracts and places only needed files in the container, while you just bundle the entire zulu package with all of its dependencies in your project. Edit: tadfisher seems to be more familiar with it than me, so I would start with that advice and modify code so it only includes a single jdk. Then things that I mentioned could cut the size of jdk further. Edit2: noticed another comment from tadfisher about openjdk_headless, so things might be even simpler than I thought.

jossephus01 3y ago
You are correct. I havent done any trimming. Thanks for the suggestions and the gist. Thanks
- takeda 3y ago
  I found this discussion and contains code fragments and links that might help. https://discourse.nixos.org/t/how-to-create-a-docker-image-with-minimal-jdk-11-there-is-no-jre11-headless/11849 https://discourse.nixos.org/t/how-to-create-a-docker-image-w...

wbl 3y ago

Don't you just stick the JAR in?

tadfisher 3y ago

Oh, and openjdk_headless skips the GTK and X dependencies that you won't need for Spring.

okr 3y ago
That's interesting. We have some applications, that produce PDFs, which use fonts, which usually requires a non-headless (headfull?) jdk. At AWS i wonder, what the default alpine jdk contains. And how much space could be saved, if people were more aware, that they can use a headless one.
- hawk_ 3y ago
  > headfull? Wonder if there is a good term for this. I have been jokingly referring to this as 'headed' and headless as 'beheaded'.
  - okr 3y ago
    Axed! :)

markelliot 3y ago

Hard to beat jib (https://github.com/GoogleContainerTools/jib/tree/master/jib-gradle-plugin https://github.com/GoogleContainerTools/jib/tree/master/jib-...) for minimal Java OCI containers.

max-privatevoid 3y ago

I decided to participate in your challenge and cleaned up your Nix code a little bit. It seems like the main task of the challenge is building a really small JRE. I've switched to using a headless OpenJDK build from Nixpkgs as a baseline instead of Zulu, to remove all the unnecessary dependencies on GUI libraries. Then I've used pkgs.jre_minimal to produce a custom minimal JRE with jlink. The image size now comes out to 161MB, which is slightly larger than the demo_jlink image. This is because it actually includes all the modules required to run the application, resulting in a ~90MB JRE. The jdeps invocation in Dockerfile_jlink fails to detect all the modules, so that JRE is only built with java.base. Building my minimal JRE with only java.base brings the JRE size down to about 50MB, the resulting (broken) container image is 117MB according to Podman. I've also removed the erroneous copyToRoot from your call to dockerTools.buildImage, which resulted in copying the app into the image a second time while the use of string context in config.Cmd would have already sufficed. I've also switched to dockerTools.buildLayeredImage, which puts each individual store path into its own image layer, which is great for space scalability due to dependency sharing between multiple container images, but won't have an impact for this single-image experiment. This is mostly a JRE size optimization challenge. The full list of dependencies and their respective size is as follows: /nix/store/v27dxnsw0cb7f4l1i3s44knc7y9sw688-zlib-1.3 125.6K /nix/store/j6n6ky7pidajcc3aaisd5qpni1w1rmya-xgcc-12.3.0-libgcc 139.1K /nix/store/l0ydz31lwa97zickpsxj2vmprcigh1m4-gcc-12.3.0-libgcc 139.1K /nix/store/a3n1vq6fxkpk5jv4wmqa1kpd3jzqhml9-libidn2-2.3.4 350.4K /nix/store/s5ka5vdlp4izan3nfny194yzqw3y4d1z-lcms2-2.15 445.3K /nix/store/a5l3w6hiprvsz7c46jv938iij41v57k6-libjpeg-turbo-2.1.5.1 1.6M /nix/store/r9h133c9m8f6jnlsqzwf89zg9w0w78s8-bash-5.2-p15 1.6M /nix/store/3dfyf6lyg6rvlslvik5116pnjbv57sn0-libunistring-1.1 1.8M /nix/store/a3zlvnswi1p8cg7i9w4lpnvaankc7dxx-gcc-12.3.0-lib 7.5M /nix/store/657b81mfpbdz09m4sk4r9i1c86pm0i8f-app-1.0.0 19.0M /nix/store/1zy01hjzwvvia6h9dq5xar88v77fgh9x-glibc-2.38-44 28.8M /nix/store/b1fhkmscb0vff63xl8ypp4nsc7sd96np-openjdk-headless-minimal-jre-21+35 91.4M There's not much else that can be done here. glibc is the next largest dependency at ~30MB. This large size seems to be because Nixpkgs configures glibc to be built with support for many locales and character encodings. I don't know if it would be possible or practical to split these files out into separate derivations or outputs and make them optional that way. If you're using multiple images built by dockerTools.buildLayeredImage, glibc (and everything else) will be shared across all of them anyway (given you're using roughly the same Nixpkgs commit). https://github.com/max-privatevoid/hackernews-docker-challenge/blob/main/flake.nix https://github.com/max-privatevoid/hackernews-docker-challen...