Build System, Dependency Management and IDE integration (was: Eclipse, Ivy and MOE)

Feature
1

We mainly using Ivy in our projects, the advantage with Ivy over Gradle is the ability to link dependencies that are avaliable in workspace, this allows us to develop sub-projects without publishing anything or changing the dependencies definition. We managed to run MOE in Eclipse but the problem that MOE depends on Gradle dependencies and ignores java build path in Eclipse (which contains Ivy libs which contain opened projects)

In the attached picture you can see that projects (GoogleMapsCore & GoogleMapsIOS) are avaliable in the workspace and any change to them should be reflected in runtime ! (it does in Android and Desktop)

So, how to make MOE use the same java build path defined in eclipse ?

Thanks in advance.

1 Like
2
  1. I forgot about attaching the picture
  2. I just test MOE without Ivy , and just adding manually a project in eclipse fails because of the same issue
3

Dear Alaa Murad,

yes, MOE is using the build path that was defined in Gradle (or Maven for that matter), and not the one that you altered in your IDE.

This is a tricky request, and it can be generalized as: “the MOE IDE plugins should implement a custom builder that will use the build settings in the IDE instead of the one defined in the Gradle / Maven build”. We already received a similar request from @priand, but it was about Maven, because M2Eclipse has some special hacks in place to work with projects open in your Eclipse workspace, and we were not using that, instead we are doing a regular Maven build during launch.

I have 2 conflicting thoughts regarding this:

  1. I have a great respect for the fellow engineers who are working on software projects with MOE. I always assume that they have a good reason for doing things the way they do, and this is why we try to keep MOE very flexible, so it can be adapted to the needs of its users (the developers). So please, do share your requirements and also the rationale behind those requirements, because we want to make the best tool for you.

  2. I think, that integrating the build systems into IDEs the way they are now was a huge mistake, and it led to a lot of wasted effort, man power and money over the last 10-15 years. Let me explain:

  • Vendor lock-in: When you use your IDEs build system in a non-trivial project (especially if you are using a custom IDE plugin for it), you basically locked yourself into that IDE. Even if you use an open-source IDE, you could be locked into an outdated version for 5 years, because the IDE vendor went on a mad crusade to implement CSS into the IDE framework… many thanks to Eclipse for teaching us this valuable lesson.
  • Parallel build systems: You actually have 2 build systems: one in your IDE, and another one on your build server (I assume that you use a build server and you don’t live in the dark ages of GUI driven builds). This will open the door (door?! a huge gate!) to all sorts of debugging issues, because you will use a different environment during development and in production. Not a good practice.
  • IDE build system limitations: In the end, the integrated build systems of the IDE were not designed for today’s complex projects. Actually they were not enough to handle JavaEE projects properly a decade ago… They were better than nothing, sure. They were even better than Ant / Make in most cases. But if you compare the IDEs build system to Gradle or to the Buck / Blaze / Bazel / Pants / etc. “family” then it will be clear what I mean by limited. JetBrains actually started baby steps in the right direction, and in a recent IntelliJ they included an option where they call Gradle tasks directly for building instead of trying to use their own build system.

So my recommendation is to avoid depending on the IDEs build system, but use your favourite build system as much as you can, even when you work in your IDE. Definitely don’t use a custom dependency management solution inside your IDE.

I know that this is probably not the answer you were hoping for, and I am very interested in your feedback.

I am also very interested in feedback from the whole community. What do you think? What are your requirements in terms of build systems, dependency management and IDE integration?

Best Regards,
Gergely

4

Dear Gergely,

First thanks for the extensive reply and I do confirm that my request is “the MOE IDE plugins should implement a custom builder that will use the build settings in the IDE instead of the one defined in the Gradle / Maven build”

And my point to support that, simply, it’s a plugin for eclipse, MOE already toke steps to be integrated with IDE (vendor-locking) , I almost fail to understand the point of the plugin at this point if we must do everything with gradle, and let me be frank that gradle itself is the biggest vendor locking pushed on many java developers by many projects (that are forced to learn Groovy to build java project !!!) but this issue is out of topic.

My request is simple, we have LibGdx (which has a project for every platform and a core which contains the common code) , we then build small libs as separate projects, for example we need to have GoogleMaps, so we create projects like (GoogleMapsCore, then GoogleMapsAndroid, GoogleMapsIOS, GoogleMapsHTML ,etc) now in main project we have all those defined as

<dependency org="org" name="googlemaps-core" rev="latest.integration" conf="default->default" branch="${dep.branch}" />
<dependency org="org" name="googlemaps-android" rev="latest.integration" conf="default->default" branch="${dep.branch}" />

The magic of ivy that you don’t have to publish (while developing) the project as it will check Eclipse’s workspace and use the provided source (it even let you defined your git branch !) , now all of our projects (android, desktop, html) works without dependancies on gradle or any other tools and this make it very easy to fix stuff on the fly by opening the lib as a source and modify the code, if everything works, then we just publish the new lib/jar and close the lib project and refresh ivy , then we test again against the real jar. In MOE, we can’t do that because of “vendor-locking” to gradle, so if MOE is anti-vendor locking, it should’t be depending on gradle, IDE is a choice that millions of developers make, and we get lots of benefits from it. In MOE project, we are forced to publish a new snapshot/jar for every code change while developing (this translate to hundreds of jars to fix small problem or create simple feature).

I think the current MOE plugin is almost like Facebook HTML5 hybrid app (didn’t deliver full mobile experience), as it only warps gradle commands. You should be going native with the plugin, then you have to be bias toward the IDE (and use the IDE’s classpath and build settings)

If this didn’t make sense (Ivy stuff), I can live with allowing me to add the project manually to Eclipse (Like the test project), otherwise, are you really expecting that we publish every single modification ? how to achieve modifying code of sub-projects on-the-fly ?

Thanks in advance.
Alaa

5

Dear Alaa,

it looks to me that you want the IDE to build the Java parts and then hand it off to the rest of the build system.

We are actually doing something very similar with Maven already: Maven builds the java classes, and then the build results are passed to Gradle to execute the rest of the build steps (proguard, retrolambda, dex2oat, xcodebuild).

A possible solution would be to customize your iOS module’s build.gradle to use the output of the IDE build system as the starting point instead of the source folders. The change could be restricted to only happen when the build is called from the IDE. The advantage is that the solution is not Ivy or IDE specific.

This is why our use of Gradle is not a vendor-lockin in the sense you are referring to it: It is possible to add new frontends to it, like our Maven support or the use of the IDE build output that you are requesting. In this case Gradle is just used as a task management / scheduling solution and not a complete build system.

Best Regards,
Gergely

6

Gergely , you actually gave me a nice hint, it’s not fully automatic but I can live with it.

sourceSets.main.java.srcDirs = [ "src/", "MY_LIB_PATH/src/"]

I added the source of my lib directly to srcDirs, I also had to comment-out the lib dependency in gradle (so I don’t have 2 copies of the same classes) , now I can edit my code and hit run :slight_smile:

This what I was missing, this is OK work around for now.

Thanks
Alaa