MOE Community Edition Gradle Plugin & SDK 1.7.2

[Noisyfox]

Changelog

• Javadoc added to SDK classes/methods, generated from Apple’s document.
• Code comments are copied from header files to java files as javadoc comments when generating bindings from header/framework.
• Fix a bug that bindings of multiple categories of the same class from different frameworks are generated to the same package.

Notice

The comment/doc generation is not perfect and sometimes it could grab the wrong comment from source code, or it might miss some comment. If you find anything wrong with the generated comment please let me know, I’ll try my best to fix it. However keep in mind it’s difficult to make every comment work properly due to the all possible comment format…

Install / Upgrade

Simply change you build script to use the new plugin and SDK:

buildscript {
    dependencies {
        classpath group: 'org.multi-os-engine.community', name: 'moe-gradle', version: '1.7.2'
    } 
}

[ekuleshov (Eugene)]

Great stuff @Noisyfox!

Curiously, some of the core comments are different from the apple online docs. I wasn’t been able to locate the original .h files for those. Not a big deal, I’m just wondering what’s the cause of that. E.g. UIColor.labelColor() has javadoc like:

/**
 * Foreground colors for static text and related elements.
 */

And the Apple docs have:

The color for text labels that contain primary content.

Also no docs for UIColor.initWithRedGreenBlueAlpha(…).

I was able to regenerate bindings in the mlkit sample project and most of the comments came trough.

Some didn’t. E.g. MLKTextRecognizer.h has docs on the MLKTextRecognitionCallback typedef:

/**
 * The callback to invoke when the text recognition completes.
 *
 * @param text Recognized text in the image or `nil` if there was an error.
 * @param error The error or `nil`.
 */
typedef void (^MLKTextRecognitionCallback)(MLKText *_Nullable text, NSError *_Nullable error)
    NS_SWIFT_NAME(TextRecognitionCallback);

It is generated in java like:

@Runtime(ObjCRuntime.class)
@Generated
public interface Block_processImageCompletion {
	@Generated
	void call_processImageCompletion(MLKText text, NSError error);
}

I’m guessing the callback code is generated from the method parameter and not from the typedef itself.

That also explains why methods like Globals.dispatch_after() and Globals.dispatch_async() are using different types for the callback function that all correspond to the same dispatch_block_t type.

[ekuleshov (Eugene)]

Another minor glitch with the javadoc comments.

It seems like there are few variations used in Swift code for declaring parameters and these get into the javadoc as is:

• :param: name - example
• \param name - example - this is somehow converted from the Swift’s - name comment
• returns: - example - seems like converted from Swift’s - returns: or - Returns:

It would be great to replace above :param:, \param and returns: with javadoc’s @param and @return accordingly when generating bindings comments.

[ekuleshov (Eugene)]

Stumbled upon an issue with generated bindings:

The following Objc code in MDC iOS code (v. 112.1 - the last version to support ios 9.0):

/**
 Bound to @c delegate on the @c defaultManager instance.
 */
@property(class, nonatomic, weak, nullable) id<MDCSnackbarManagerDelegate> delegate;

produces this in the generated Java bindings:

/**
 * Bound to @c delegate on the @c defaultManager instance.
 */
@Generated
public static void setDelegate_static(@Mapped(ObjCObjectMapper.class) MDCSnackbarManagerDelegate value) {
    Object __old = delegate_static();
    if (value != null) {
        org.moe.natj.objc.ObjCRuntime.associateObjCObject(this, value);
    }
    setDelegate_static_unsafe(value);
    if (__old != null) {
        org.moe.natj.objc.ObjCRuntime.dissociateObjCObject(this, __old);
    }
}

Note this is used in the generated static method, so code does not compile. For the time being I commented this out, not sure what to use instead of this or if generated method should not be static.

[Noisyfox]

@ekuleshov

Curiously, some of the core comments are different from the apple online docs. I wasn’t been able to locate the original .h files for those. Not a big deal, I’m just wondering what’s the cause of that. E.g. UIColor.labelColor() has javadoc like:

/**
 * Foreground colors for static text and related elements.
 */

And the Apple docs have:

The color for text labels that contain primary content.

Because the docs are collected directly from SDK’s header file which looks like this:

The same applys to UIColor.initWithRedGreenBlueAlpha(…)

[Noisyfox]

@ekuleshov

I’m guessing the callback code is generated from the method parameter and not from the typedef itself.

That also explains why methods like Globals.dispatch_after() and Globals.dispatch_async() are using different types for the callback function that all correspond to the same dispatch_block_t type.

Yes. I was thinking of merging all blocks from the same typedef to a single interface however it was a bit too difficult, and it is also a breaking change.

[Noisyfox]

I think the reason is the delegate property is expected to be an instance property, and currently natj doesn’t handle a static delegate properly. I’ll see if I could fix it.

[Noisyfox]

@ekuleshov

• :param: name - example
• \param name - example - this is somehow converted from the Swift’s - name comment
• returns: - example - seems like converted from Swift’s - returns: or - Returns:

There are just way too many different comment formats and I can not handle all of them correctly, because this could introduce false positives.

[ekuleshov (Eugene)]

@Noisyfox

There are just way too many different comment formats and I can not handle all of them correctly, because this could introduce false positives.

Some simple logic should work. E.g. when the first 2 patterns are at the beginning of the line inside comment block. The 3rd one is probably more prone for false positives

[ekuleshov (Eugene)]

@Noisyfox

I think the reason is the delegate property is expected to be an instance property, and currently natj doesn’t handle a static delegate properly. I’ll see if I could fix it.

BTW, speaking of generating static methods, there are some ancient issues related to Java generic values returned from the static methods.

For example, the NSArray class is declared like this:

public class NSArray<_ObjectType> extends NSObject
...

But then the .alloc() and a few other init methods are generated like this:

public static native NSArray<?> alloc();
public static native <_ObjectType> NSArray<?> array();

So, we have to explicitly cast the returned values in the Java code, e.g.:

NSMutableArray<NSLayoutConstraint> constraints =
  (NSMutableArray<NSLayoutConstraint>) NSMutableArray.array();

It will be much cleaner if these methods are generated like this instead:

public static native <T> NSArray<T> alloc();
public static native <T> NSArray<T> array();

This way Java compiler will do the type inference and we won’t have to cast:

NSMutableArray<NSLayoutConstraint> constraints = NSMutableArray.array();

Similarly, classes like NSLayoutConstraint have static methods like:

public static native NSArray<? extends NSLayoutConstraint> constraintsWithVisualFormatOptionsMetricsViews(
        String format, @NUInt long opts, NSDictionary<String, ?> metrics, NSDictionary<String, ?> views);

That basically require force-cast of the return value in Java and to avoid that cast the above method need to be like this:

public static native <T extends NSLayoutConstraint NSArray<T> constraintsWithVisualFormatOptionsMetricsViews(
        String format, @NUInt long opts, NSDictionary<String, ?> metrics, NSDictionary<String, ?> views);

Also, some other non-static methods are also off. E.g. the .initWithObjects(..) isn’t using class’s type _ObjectType:

public native NSArray<?> initWithObjects(
  @Mapped(ObjCObjectMapper.class) _ObjectType firstObj, Object... varargs);

I’d expect it to be like this:

public native NSArray<_ObjectType> initWithObjects(
  @Mapped(ObjCObjectMapper.class) _ObjectType firstObj, _ObjectType... varargs);

Generally all of the above makes you write a lot of not type-safe code and it is an error prone due to casting.

[Noisyfox]

Not everything can work due to how Java’s generic work. For example, I cannot make alloc() generic (i.e. public static native <T> NSArray<T> alloc()) because then this will clash with the alloc() in NSObject:

Java requires the static method in chile class that has the same signature as the superclass to also have the same number of generic type peremeters, otherwise it cannot determine which method to invoke.

---

As for this one:

@ekuleshov

Similarly, classes like NSLayoutConstraint have static methods like:

public static native NSArray<? extends NSLayoutConstraint> constraintsWithVisualFormatOptionsMetricsViews(
        String format, @NUInt long opts, NSDictionary<String, ?> metrics, NSDictionary<String, ?> views);

That basically require force-cast of the return value in Java and to avoid that cast the above method need to be like this:

public static native <T extends NSLayoutConstraint> NSArray<T> constraintsWithVisualFormatOptionsMetricsViews(
        String format, @NUInt long opts, NSDictionary<String, ?> metrics, NSDictionary<String, ?> views);

You have a misunderstandment here. The type parameter of NSArray is marked as __covariant, which means it should ALWAYS be used as NSArray<? extends SomeType> in Java. By marking the type parameter as __covariant, it allows you do things like

NSArray<UIView*> *viewArray = [[NSArray alloc] init];
NSArray<NSObject*> *objArray = viewArray;

the equivalence in Java is:

class A { }
class B extends A { }

List<? extends B> subList = new ArrayList<>();
List<? extends A> supList = subList; // <- this is fine

List<B> bList = new ArrayList<>();
List<A> aList = bList; // Error: java: incompatible types: List<B> cannot be converted to List<A>

This is also the reason of ALL binding methods that take NSArray as parameter always declare the typs as NSArray<? extends SomeType> (with the only exception of NSArray due to how NSString is specially handled by MOE).

However in this case, it might be good enought to make the return type simply not wildcarded, e.g.

public static native NSArray<NSLayoutConstraint> constraintsWithVisualFormatOptionsMetricsViews(
        String format, @NUInt long opts, NSDictionary<String, ?> metrics, NSDictionary<String, ?> views);

because this __covariant thing only matters when you try to assign a value, so as long as the api parameter contains the correct wildcard then it should be perfectly fine, since this is perfectly valid:

List<B> bList = new ArrayList<>();
List<? extends A> aList = bList;

which means when calling an Objective-C method, the __covariant is still honored, and you don’t need to deal with the cast in most of the cases.

The reason of not using <T extends SomeType> NSArray<T> method(); is because this is totally wrong, because the original return type NSArray<? extends SomeType> means the returned array could contain object of type SomeType, but if you set T to a subtype, then when you get the item from the array, you will have a cast error.

---

@ekuleshov

Also, some other non-static methods are also off. E.g. the .initWithObjects(..) isn’t using class’s type _ObjectType :

public native NSArray<?> initWithObjects(
  @Mapped(ObjCObjectMapper.class) _ObjectType firstObj, Object... varargs);

I’d expect it to be like this:

public native NSArray<_ObjectType> initWithObjects(
  @Mapped(ObjCObjectMapper.class) _ObjectType firstObj, _ObjectType... varargs);

For the type issue of the varargs, I’m currently looking at it. Should be fixable.

[Noisyfox]

@Noisyfox

However in this case, it might be good enought to make the return type simply not wildcarded, e.g.

public static native NSArray<NSLayoutConstraint> constraintsWithVisualFormatOptionsMetricsViews(
        String format, @NUInt long opts, NSDictionary<String, ?> metrics, NSDictionary<String, ?> views);

because this __covariant thing only matters when you try to assign a value, so as long as the api parameter contains the correct wildcard then it should be perfectly fine, since this is perfectly valid:

List<B> bList = new ArrayList<>();
List<? extends A> aList = bList;

which means when calling an Objective-C method, the __covariant is still honored, and you don’t need to deal with the cast in most of the cases.

Hmmmm this seems to be undoable because it causes issue when overriding the method with a more specific return type.

[ekuleshov (Eugene)]

@Noisyfox

Java requires the static method in chile class that has the same signature as the superclass to also have the same number of generic type peremeters, otherwise it cannot determine which method to invoke.

How about static <T> T alloc() for both NSObject and NSArray?

Also, how about methods like one in NSMutableArray <_ObjectType> NSMutableArray<?> array() ?

@Noisyfox

You have a misunderstandment here. The type parameter of NSArray is marked as __covariant , which means it should ALWAYS be used as NSArray<? extends SomeType> in Java.

I’m not suggesting to remove covariance, but replace the wildcard typing with a generic parameter, i.e. replace ? with T locally declared for static methods with <T>.

Also note, there is some covariance mismatch in some MOE generated binding classes, so your example not always work. E.g. compare the following methods from NSMutableArray:

void addObjectsFromArray(NSArray<_ObjectType> otherArray)

and

boolean addAll(Collection<? extends _ObjectType> c)

The second one is covariant, and first one isn’t. But the addObjectsFromArray(someArray()) would have worked if someArray is declared as <T> NSArray<T> someArray()

@Noisyfox

Hmmmm this seems to be undoable because it causes issue when overriding the method with a more specific return type.

What methods those would be?

[Noisyfox]

@ekuleshov

How about static <T> T alloc() for both NSObject and NSArray?

This won’t work, because we have NSDictionary which takes two type parameters. Are you gonna make ALL classes’ alloc() methods take two type parameter?

@ekuleshov

Also, how about methods like one in NSMutableArray <_ObjectType> NSMutableArray<?> array() ?

This will be changed to <_ObjectType> NSMutableArray<_ObjectType> array().

@ekuleshov

I’m not suggesting to remove covariance, but replace the wildcard typing with a generic parameter, i.e. replace ? with T locally declared for static methods with <T> .

I’m not saying to remove the parameter (although I originally said I was trying to removing it, but then I found it impossible, but this is not the point here), I’m saying those two forms are not equivalence. You must keep the widecard in the result type, otherwise the covariant is missing.

@ekuleshov

The second one is covariant, and first one isn’t. But the addObjectsFromArray(someArray()) would have worked if someArray is declared as <T> NSArray<T> someArray()

I’m about to fix the first one, to change it to void addObjectsFromArray(NSArray<? extends _ObjectType> otherArray)

@ekuleshov

What methods those would be?

For example these three classes have a same property that returns different types:

AVAsset:
@property (nonatomic, readonly) NSArray<AVAssetTrack *> *tracks;

AVMovie:
@property (nonatomic, readonly) NSArray<AVMovieTrack *> *tracks;

AVMutableMovie:
@property (nonatomic, readonly) NSArray<AVMutableMovieTrack *> *tracks;

And if the return types do not have wildcard in Java:

public native NSArray<AVAssetTrack> tracks();
public native NSArray<AVMovieTrack> tracks();
public native NSArray<AVMutableMovieTrack> tracks();

You will get error:

tracks() in AVMovie cannot override tracks() in AVAsset: return type NSArray is not compatible with NSArray
tracks() in AVMutableMovie cannot override tracks() in AVMovie: return type NSArray is not compatible with NSArray

If you change it to

public native NSArray<? extends AVAssetTrack> tracks();
public native NSArray<? extends AVMovieTrack> tracks();
public native NSArray<? extends AVMutableMovieTrack> tracks();

then it works.

[ekuleshov (Eugene)]

BTW, while we are at this, few years back @kisg had an idea to replace these alloc() calls with a regular Java constructors. Maybe he could comment on the complexity of the changes needed for that.

@Noisyfox

@ekuleshov

How about static <T> T alloc() for both NSObject and NSArray?

This won’t work, because we have NSDictionary which takes two type parameters. Are you gonna make ALL classes’ alloc() methods take two type parameter?

The static <T> T alloc() declaration has only one parameter.

Unlike the current NSDictionary declaration NSDictionary<?, ?> alloc() or even <K,V> NSDictionary<K, V> alloc().

With <T> T alloc(), the compiler should take the assignment side type and would substitute it for T or you could specify it explicitly NSDictionary.<NSDictionary<String, Integer>>alloc().

@Noisyfox

@ekuleshov

Also, how about methods like one in NSMutableArray <_ObjectType> NSMutableArray<?> array() ?

This will be changed to <_ObjectType> NSMutableArray<_ObjectType> array() .

Awesome!

There probably other similar methods in NSArray, NSMutableArray, NSDictionary and NSMutableDictionary that returns wildcards instead of type params, like:

public static native <_KeyType, _ObjectType> NSDictionary<?, ?> dictionaryWithDictionary(
        NSDictionary<_KeyType, _ObjectType> dict);

@Noisyfox

@ekuleshov

I’m not suggesting to remove covariance, but replace the wildcard typing with a generic parameter, i.e. replace ? with T locally declared for static methods with <T> .

I’m not saying to remove the parameter (although I originally said I was trying to removing it, but then I found it impossible, but this is not the point here), I’m saying those two forms are not equivalence. You must keep the widecard in the result type, otherwise the covariant is missing.

I guess for non-static methods wildcards are unavoidable in most cases. But for the static ones, instead of having something like:

static <_ObjectType> NSArray<?> array();

It should be like:

static <_ObjectType> NSArray<_ObjectType> array();

This one is from NSArray

@Noisyfox

@ekuleshov

The second one is covariant, and first one isn’t. But the addObjectsFromArray(someArray()) would have worked if someArray is declared as <T> NSArray<T> someArray()

I’m about to fix the first one, to change it to void addObjectsFromArray(NSArray<? extends _ObjectType> otherArray)

Awesome!

@Noisyfox

@ekuleshov

What methods those would be?

For example these three classes have a same property that returns different types:

AVAsset:
@property (nonatomic, readonly) NSArray<AVAssetTrack *> *tracks;

AVMovie:
@property (nonatomic, readonly) NSArray<AVMovieTrack *> *tracks;

AVMutableMovie:
@property (nonatomic, readonly) NSArray<AVMutableMovieTrack *> *tracks;

And if the return types do not have wildcard in Java:

public native NSArray<AVAssetTrack> tracks();
public native NSArray<AVMovieTrack> tracks();
public native NSArray<AVMutableMovieTrack> tracks();

You will get error:

tracks() in AVMovie cannot override tracks() in AVAsset: return type NSArray is not compatible with NSArray
tracks() in AVMutableMovie cannot override tracks() in AVMovie: return type NSArray is not compatible with NSArray

If you change it to

public native NSArray<? extends AVAssetTrack> tracks();
public native NSArray<? extends AVMovieTrack> tracks();
public native NSArray<? extends AVMutableMovieTrack> tracks();

then it works.

That’s expected, because AVMovie.tracks() and AVMutableMovie.tracks() are overwriting the AVAsset.tracks() and there are more subclasses of the first two…

[Noisyfox]

@ekuleshov

With <T> T alloc() , the compiler should take the assignment side type and would substitute it for T or you could specify it explicitly NSDictionary.<NSDictionary<String, Integer>>alloc() .

Then what if you write:

NSDictionary<String, String> dict = NSObject.alloc();

This is obviously wrong but the compiler won’t tell you that.
This is even more dangerous if you are using Kotlin with the type inference. If the compiler cannot tell you whether you use the wrong type it’s very easy to get it wrong.

@ekuleshov

There probably other similar methods in NSArray, NSMutableArray, NSDictionary and NSMutableDictionary that returns wildcards instead of type params, like:

NSMutableArray and NSMutableDictionary are not covariant because they are mutable.

@ekuleshov

I guess for non-static methods wildcards are unavoidable in most cases. But for the static ones, instead of having something like:

static <_ObjectType> NSArray<?> array();

It should be like:

static <_ObjectType> NSArray<_ObjectType> array();

This one is from NSArray

This might work. Let me have a try.

[ekuleshov (Eugene)]

@Noisyfox

Then what if you write:

NSDictionary<String, String> dict = NSObject.alloc();

This is obviously wrong but the compiler won’t tell you that.
This is even more dangerous if you are using Kotlin with the type inference. If the compiler cannot tell you whether you use the wrong type it’s very easy to get it wrong.

Good point.

Though you have to cast the right side in most cases anyways. E.g. can’t really assign this to anything type-safe without casting:

public static native NSArray<?> alloc();

[Noisyfox]

@ekuleshov

Though you have to cast the right side in most cases anyways. E.g. can’t really assign this to anything type-safe without casting:

public static native NSArray<?> alloc();

At least you can’t do

UIView v = (UIView)NSArray.alloc();

I was thinking of adding a separate alloc function which contains the number of the type parameter in the function name to overcome the function hidden issue, e.g.:

class NSObject {
    public static native NSObject alloc();
}

class NSArray<T> {
   public static native NSArray<?> alloc();  // <- this hides NSObject.alloc();
   public static native <T> NSArray<? extends T> alloc_1();
}

class NSMutableArray<T> extends NSArray<T> {
   public static native NSMutableArray<?> alloc(); // <- this hides NSObject.alloc() and NSArray.alloc()
   public static native <T> NSMutableArray<T> alloc_1(); // <- this hides NSArray.alloc_1()
}

[ekuleshov (Eugene)]

@Noisyfox

At least you can’t do

UIView v = (UIView)NSArray.alloc();

Sure you can. Cast is an unsafe operation. It will fail at runtime.

@Noisyfox

I was thinking of adding a separate alloc function which contains the number of the type parameter in the function name to overcome the function hidden issue

Maybe everything named alloc() is part of the problem. Wouldn’t it be easier with method like anNSArray(), anNSDictionary(), anNSMutableArray(), etc instead of all those alloc() ones?

Anyhow, currently it works, but I’m just voicing out some pain that comes with it. Just recently I have to write some code for iOS Charts bindings. Ended up with this:

// NSArray<? extends ChartHighlight> highlighted()
NSArray<ChartHighlight> highlights = (NSArray<ChartHighlight>) chart.highlighted();

  // <_ObjectType> NSMutableArray<?> arrayWithCapacity(@NUInt long numItems)
  highlights = (NSArray<ChartHighlight>) NSMutableArray.arrayWithCapacity(count);

    // ChartHighlight alloc();
    // ChartHighlight initWithXYDataSetIndexDataIndex(...)
    highlights.add(ChartHighlight.alloc().initWithXYDataSetIndexDataIndex(..));
    ...

    // NSMutableArray<?> alloc()
    // NSMutableArray<?> initWithArray(NSArray<_ObjectType> array)
    highlights = (NSArray<ChartHighlight>)
        ((NSArray<ChartHighlight>) NSMutableArray.alloc()).initWithArray(highlights);

Note that I had to cast chart.highlighted() from NSArray<? extends ChartHighlight> into NSArray<ChartHighlight> or else the NSArray.add(..) won’t let to add a ChartHighlight.

And when assigning to the same NSArray variable, the NSArray.initWithArray() gets really ridiculous with casting.

Things just don’t add up nicely.

[Noisyfox]

@ekuleshov

@Noisyfox

At least you can’t do

UIView v = (UIView)NSArray.alloc();

Sure you can. Cast is an unsafe operation. It will fail at runtime.

No you can’t. You will get a compile error.

@ekuleshov

Maybe everything named alloc() is part of the problem. Wouldn’t it be easier with method like anNSArray() , anNSDictionary() , anNSMutableArray() , etc instead of all those alloc() ones?

This could work as an addition to the existing alloc() method (and perhaps mark alloc() as deprecated.