Reduce dependencies from Full JRE to compact1 profile

[avkonst]

We have got few applications written in Scala. Now we would like to reduce footprint of deliverable binaries. Two reasons: be able to deploy to embedded IoT-like devices and reduce size of docker images.

Resolution of this ticket will also make very positive effect on the size of Java 9 modules required by scala-library, when scala becomes ready for Java 9 jlink tool.

We downsized binaries of compiled OpenJDK JRE from 90MB to 13MB (we achieve it with minimal1 variant of stripped libjvm and compressed compact1 profile of rt.jar). This is big gain, but scala libraries have got unfortunate dependencies which trigger Full JRE requirement instead of compact1.

There 3 places of interest:

• scala.beans (org.scala-lang.scala-library-2.12.2.jar) -> java.beans (which is only avaialble in Full JRE profile of rt.jar)
• scala.sys.process (org.scala-lang.scala-library-2.12.2.jar) -> java.lang.management (which is available in compact3 profile of rt.jar, FYI: compact3 profile is 2 times bigger in size than compact1)
• scala.reflect.internal (org.scala-lang.scala-reflect-2.12.2.jar) -> java.rmi (which is available in compact2 profile of rt.jar, FYI: compact2 profile is about 50% bigger than compact1)

I wonder if it is possible to reduce these dependencies?

Maybe there is a way to disable part of functionally by testing dependency class availability in runtime and using reflection to invoke it?

Alternatively, it maybe possible to split 2 scala jars into more pieces, for example: libs required only for compilation and libs required for runtimes with various profiles (like scala-library-compact1, scala-library-compact2, scala-library-compact3, scala-library-extra instead of a single big one).

I am keen on to contribute the change if you could advise what can be done.

Here are the details and commands, which I used to analyse this:

>  jdeps -profile org.scala-lang.scala-library-2.12.2.jar | egrep -v ".jar$"
org.scala-lang.scala-library-2.12.2.jar -> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar (Full JRE)
   scala (org.scala-lang.scala-library-2.12.2.jar)
      -> java.io                                            compact1
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.lang.ref                                      compact1
      -> java.lang.reflect                                  compact1
      -> java.util                                          compact1
      -> java.util.concurrent.locks                         compact1
      -> java.util.stream                                   compact1
   scala.annotation (org.scala-lang.scala-library-2.12.2.jar)
      -> java.lang                                          compact1
   scala.annotation.meta (org.scala-lang.scala-library-2.12.2.jar)
      -> java.lang                                          compact1
   scala.annotation.unchecked (org.scala-lang.scala-library-2.12.2.jar)
   scala.beans (org.scala-lang.scala-library-2.12.2.jar)
      -> java.beans                                         Full JRE
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.lang.reflect                                  compact1
   scala.collection (org.scala-lang.scala-library-2.12.2.jar)
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.util                                          compact1
      -> java.util.concurrent                               compact1
   scala.collection.concurrent (org.scala-lang.scala-library-2.12.2.jar)
      -> java.io                                            compact1
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.util                                          compact1
      -> java.util.concurrent                               compact1
      -> java.util.concurrent.atomic                        compact1
   scala.collection.convert (org.scala-lang.scala-library-2.12.2.jar)
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.util                                          compact1
      -> java.util.concurrent                               compact1
      -> java.util.function                                 compact1
   scala.collection.generic (org.scala-lang.scala-library-2.12.2.jar)
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.util.concurrent.atomic                        compact1
   scala.collection.immutable (org.scala-lang.scala-library-2.12.2.jar)
      -> java.io                                            compact1
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.util                                          compact1
      -> java.util.regex                                    compact1
   scala.collection.mutable (org.scala-lang.scala-library-2.12.2.jar)
      -> java.io                                            compact1
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.lang.reflect                                  compact1
      -> java.util                                          compact1
      -> java.util.regex                                    compact1
      -> java.util.stream                                   compact1
   scala.collection.parallel (org.scala-lang.scala-library-2.12.2.jar)
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.util                                          compact1
      -> java.util.concurrent                               compact1
      -> java.util.concurrent.atomic                        compact1
   scala.collection.parallel.immutable (org.scala-lang.scala-library-2.12.2.jar)
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.util.concurrent.atomic                        compact1
   scala.collection.parallel.mutable (org.scala-lang.scala-library-2.12.2.jar)
      -> java.io                                            compact1
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
   scala.collection.script (org.scala-lang.scala-library-2.12.2.jar)
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
   scala.compat (org.scala-lang.scala-library-2.12.2.jar)
      -> java.lang                                          compact1
      -> java.lang.reflect                                  compact1
      -> java.nio.charset                                   compact1
      -> java.util                                          compact1
   scala.concurrent (org.scala-lang.scala-library-2.12.2.jar)
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.util                                          compact1
      -> java.util.concurrent                               compact1
      -> java.util.concurrent.atomic                        compact1
   scala.concurrent.duration (org.scala-lang.scala-library-2.12.2.jar)
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.util.concurrent                               compact1
   scala.concurrent.forkjoin (org.scala-lang.scala-library-2.12.2.jar)
      -> java.lang                                          compact1
      -> java.util                                          compact1
      -> java.util.concurrent                               compact1
   scala.concurrent.impl (org.scala-lang.scala-library-2.12.2.jar)
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.util                                          compact1
      -> java.util.concurrent                               compact1
      -> java.util.concurrent.atomic                        compact1
      -> java.util.concurrent.locks                         compact1
   scala.io (org.scala-lang.scala-library-2.12.2.jar)
      -> java.io                                            compact1
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.net                                           compact1
      -> java.nio                                           compact1
      -> java.nio.charset                                   compact1
      -> java.text                                          compact1
   scala.math (org.scala-lang.scala-library-2.12.2.jar)
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.math                                          compact1
      -> java.util                                          compact1
   scala.ref (org.scala-lang.scala-library-2.12.2.jar)
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.lang.ref                                      compact1
      -> java.util                                          compact1
   scala.reflect (org.scala-lang.scala-library-2.12.2.jar)
      -> java.lang                                          compact1
      -> java.lang.annotation                               compact1
      -> java.lang.invoke                                   compact1
      -> java.lang.reflect                                  compact1
   scala.reflect.macros.internal (org.scala-lang.scala-library-2.12.2.jar)
      -> java.lang                                          compact1
   scala.runtime (org.scala-lang.scala-library-2.12.2.jar)
      -> java.io                                            compact1
      -> java.lang                                          compact1
      -> java.lang.annotation                               compact1
      -> java.lang.invoke                                   compact1
      -> java.lang.ref                                      compact1
      -> java.lang.reflect                                  compact1
      -> java.util                                          compact1
      -> java.util.stream                                   compact1
   scala.runtime.java8 (org.scala-lang.scala-library-2.12.2.jar)
      -> java.io                                            compact1
      -> java.lang                                          compact1
   scala.sys (org.scala-lang.scala-library-2.12.2.jar)
      -> java.io                                            compact1
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.security                                      compact1
      -> java.util                                          compact1
   scala.sys.process (org.scala-lang.scala-library-2.12.2.jar)
      -> java.io                                            compact1
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.lang.management                               compact3
      -> java.net                                           compact1
      -> java.util                                          compact1
      -> java.util.concurrent                               compact1
   scala.text (org.scala-lang.scala-library-2.12.2.jar)
      -> java.io                                            compact1
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
   scala.util (org.scala-lang.scala-library-2.12.2.jar)
      -> java.io                                            compact1
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.util                                          compact1
      -> java.util.jar                                      compact1
   scala.util.control (org.scala-lang.scala-library-2.12.2.jar)
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
   scala.util.hashing (org.scala-lang.scala-library-2.12.2.jar)
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
   scala.util.matching (org.scala-lang.scala-library-2.12.2.jar)
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.util                                          compact1
      -> java.util.regex                                    compact1

and:

> jdeps -profile org.scala-lang.scala-reflect-2.12.2.jar | egrep -v ".jar$" | grep -v "not found"
org.scala-lang.scala-reflect-2.12.2.jar -> /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar (compact2)
   scala.reflect.api (org.scala-lang.scala-reflect-2.12.2.jar)
      -> java.io                                            compact1
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
   scala.reflect.internal (org.scala-lang.scala-reflect-2.12.2.jar)
      -> java.io                                            compact1
      -> java.lang                                          compact1
      -> java.lang.annotation                               compact1
      -> java.lang.invoke                                   compact1
      -> java.lang.ref                                      compact1
      -> java.lang.reflect                                  compact1
      -> java.rmi                                           compact2
      -> java.security                                      compact1
      -> java.util                                          compact1
      -> java.util.concurrent                               compact1
   scala.reflect.internal.annotations (org.scala-lang.scala-reflect-2.12.2.jar)
      -> java.lang                                          compact1
   scala.reflect.internal.pickling (org.scala-lang.scala-reflect-2.12.2.jar)
      -> java.io                                            compact1
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
   scala.reflect.internal.settings (org.scala-lang.scala-reflect-2.12.2.jar)
      -> java.lang                                          compact1
   scala.reflect.internal.tpe (org.scala-lang.scala-reflect-2.12.2.jar)
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
   scala.reflect.internal.transform (org.scala-lang.scala-reflect-2.12.2.jar)
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
   scala.reflect.internal.util (org.scala-lang.scala-reflect-2.12.2.jar)
      -> java.io                                            compact1
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.lang.ref                                      compact1
      -> java.lang.reflect                                  compact1
      -> java.net                                           compact1
      -> java.security                                      compact1
      -> java.security.cert                                 compact1
      -> java.util                                          compact1
      -> java.util.concurrent                               compact1
      -> java.util.concurrent.atomic                        compact1
      -> java.util.regex                                    compact1
   scala.reflect.io (org.scala-lang.scala-reflect-2.12.2.jar)
      -> java.io                                            compact1
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.lang.reflect                                  compact1
      -> java.net                                           compact1
      -> java.nio.charset                                   compact1
      -> java.nio.file                                      compact1
      -> java.nio.file.attribute                            compact1
      -> java.util                                          compact1
      -> java.util.jar                                      compact1
      -> java.util.zip                                      compact1
   scala.reflect.macros (org.scala-lang.scala-reflect-2.12.2.jar)
      -> java.lang                                          compact1
      -> java.lang.invoke                                   compact1
      -> java.net                                           compact1
   scala.reflect.macros.blackbox (org.scala-lang.scala-reflect-2.12.2.jar)
      -> java.lang                                          compact1
   scala.reflect.macros.whitebox (org.scala-lang.scala-reflect-2.12.2.jar)
      -> java.lang                                          compact1
   scala.reflect.runtime (org.scala-lang.scala-reflect-2.12.2.jar)
      -> java.io                                            compact1
      -> java.lang                                          compact1
      -> java.lang.annotation                               compact1
      -> java.lang.invoke                                   compact1
      -> java.lang.ref                                      compact1
      -> java.lang.reflect                                  compact1
      -> java.net                                           compact1
      -> java.nio.charset                                   compact1
      -> java.util                                          compact1
      -> java.util.concurrent.atomic                        compact1
      -> java.util.concurrent.locks                         compact1

[SethTisue]

highly relevant here: scala/scala#5677 (with more such work planned before 2.13 ships, and participation highly welcome). so for example scala.sys will move to a module, which will fix the compact3 requirement you found.

I'm curious about the java.rmi dependency. what's that doing, have you looked yet...?

[SethTisue]

I think the Scala.js and Scala Native projects would appreciate work in this direction as well (hello @sjrd, @densh)

[avkonst]

No, I have not looked into rmi dependency, but I did look into java.beans dependency. And it turned out it is very strong dependency based on inheritance from java.beans.SimpleBeanInfo.

[gourlaysama]

The java.rmi dependency is brought in by compiler support for the scala.remote annotation, but it was deprecated in 2.12, so it should be possible to cut that dependency in 2.13.

[adriaanm]

I'll submit a PR that removes all dependencies on @remote for 2.13.0-M3, so that we can remove it entirely once we compile the compiler with M3.

[adriaanm]

Yay! Library and reflect now only depend on the compact1 profile: scala/scala#6164

[avkonst]

Great! Big step forward.