Add Java 8+ API desugaring support

Support
1

Hi,

@Noisyfox & @kisg, Moe currently support a partial set of java 8 features and it is based on android 6 (api 23).
As you may know, google has released a while ago the desugaring support which basically allow developers to make use of a greater set of java 8 apis even across on old android apis without complaint by just adding these simple lines in the gradle (android project):

android {
…
// Configure only for each module that uses Java 8
// language features (either in its source code or
// through dependencies).
compileOptions {
** sourceCompatibility JavaVersion.VERSION_1_8**
** targetCompatibility JavaVersion.VERSION_1_8**
** }**
//…
}

desugar_diagram
Java 8 language feature support using desugar bytecode transformations.

Here is the problem, Moe cannot find java 8 classes that some of my libraries make use of. I am able to run my project from android api 8 until android api 30 without any issues. However, moe still complain about these errors:

dex2oat W 36007 1194688 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/LongUnaryOperator;
dex2oat W 36007 1194688 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/Consumer;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/BiFunction;
dex2oat W 36007 1194692 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/BiFunction;
dex2oat W 36007 1194691 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/BiFunction;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/BiFunction;
dex2oat W 36007 1194691 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/BiFunction;
dex2oat W 36007 1194692 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/Function;
dex2oat W 36007 1194691 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/Consumer;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/BiConsumer;
dex2oat W 36007 1194691 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/Function;
dex2oat W 36007 1194692 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/BiFunction;
dex2oat W 36007 1194691 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/BiFunction;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/BiFunction;
dex2oat W 36007 1194691 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/Supplier;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/Supplier;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/Supplier;
dex2oat W 36007 1194691 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/Supplier;
dex2oat W 36007 1194692 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/BiFunction;
dex2oat W 36007 1194692 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/Function;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/Function;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/BiFunction;
dex2oat W 36007 1194692 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/BiConsumer;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/BiFunction;
dex2oat W 36007 1194691 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/Function;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/BiFunction;
dex2oat W 36007 1194691 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/BiFunction;
dex2oat W 36007 1194692 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/BiFunction;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/beans/ConstructorProperties;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/beans/Transient;
dex2oat W 36007 1194692 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/beans/XMLDecoder;
dex2oat W 36007 1194691 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/beans/XMLEncoder;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/lang/FunctionalInterface;
dex2oat W 36007 1194691 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/lang/invoke/LambdaForm$Hidden;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/nio/file/FileSystemNotFoundException;
dex2oat W 36007 1194692 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/nio/file/Files;
dex2oat W 36007 1194688 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/nio/file/Paths;
dex2oat W 36007 1194692 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/nio/file/spi/FileSystemProvider;
dex2oat W 36007 1194691 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/nio/file/Path;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/nio/file/StandardCopyOption;
dex2oat W 36007 1194691 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/concurrent/CompletableFuture;
dex2oat W 36007 1194692 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/concurrent/CompletionException;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/concurrent/CompletionStage;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/concurrent/atomic/LongAdder;
dex2oat W 36007 1194691 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/BiConsumer;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/Consumer;
dex2oat W 36007 1194692 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/BiFunction;
dex2oat W 36007 1194691 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/LongUnaryOperator;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/Function;
dex2oat W 36007 1194691 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/util/function/Supplier;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Lkotlinx/serialization/json/JsonArray;
dex2oat W 36007 1194691 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Lkotlinx/serialization/json/JsonNull;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Lkotlinx/serialization/json/JsonObject;
dex2oat W 36007 1194691 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Lkotlinx/serialization/json/JsonPrimitive;
dex2oat W 36007 1194688 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Lkotlinx/serialization/json/JsonElement;
dex2oat W 36007 1194692 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Lkotlinx/serialization/json/JsonElementsKt;
dex2oat W 36007 1194693 /Users/noisyfox/Documents/projects/moe/aosp/art/compiler/driver/compiler_driver.cc:2033] Failed to resolve class Ljava/nio/file/CopyOption;

I not able to workaround theses errors using proguard as I am using some key features of these libs. These libs are working just fine across all android devices!

Please, Is there any way you could include Java 8+ API desugaring support in the next release so many java/kotlin libs could successfully compile?

Cheers

2

Hi,

Technically it is possible to add desugaring support, it would basically replace the Retrolambda component of the compilation process. However, this in itself will not solve the missing classes issue. To get these classes in class library, we would need to update the MOE runtime to a newer version of Android (right now it matches Android 6). This is a substantial amount of work:

  • updating the port of ART to iOS to use e.g. Android 11 ART
  • the new version of ART will require us to update the rest of the runtime as well, including moving to the OpenJDK based class library. Most of the tasks here are relatively straightforward porting work, but the biggest issue I can recall was with ICU. Unfortunately iOS does not export the full ICU API, but the class library is using ICU extensively. In the current version we have a partial ICU “emulation” that uses the iOS APIs and implements just enough so the old Apache Harmony based class library can work. This needs to be updated for the OpenJDK version, or we have to include the ICU database in each app that will increase the application size substantially.

Another option is looking at adding GraalVM (SubstrateVM) as an alternative Java runtime to MOE. Since Nat/J is using the standard JNI API, this should be feasible without major issues.

Best Regards,
Gergely

3 Likes
3

Thanks for your update. Actually having GraalVM to replace the Java runtime to Moe is not a bad idea! You can have quite a lot of benefits to do so. It would probably increase the speed and reduce the footprint of MOE apps. Also, I am wondering if you can take advantages of GraalVM to apply more recent java version or even apply addition polyglot languages on top of it. That would an interesting experiment to create MOE apps :slight_smile:

Kind regards

2 Likes