OpenCV Framework Entegration

Support
1

Hi i want to use opencv framework with moe,
i can manage to add from xcode that framework.

Framework contains not only .h files but .hpp files

when i use “Generate Bindings” or “Link third party framework to xcode” generated some java binding classes are missing.
i think only C header files are processing, not C++ header files are processing.

Am i rigth?

if so is there a way to generate java bindings from “.hpp” files ?

Thanks.

2

Hi Kurul,

Nat/J only supports generating bindings from C or Objective-C libraries. To use C++ support, you need to create the annotated binding classes / interfaces by hand, and also add the Nat/J C++ processor to the build procedure.

However, OpenCV comes with Java bindings: http://docs.opencv.org/java/3.1.0/
It should be straightforward to get this working with MOE. We did not test it, but most of it should work out of the box, only the hardware access parts (e.g. camera access) will need to be added, similar to how they did with the Android support.

We would be happy to help with the integration of OpenCV, just contact us.

Best Regards,
Gergely

3

Thanks for the quick tip,
I will try to integrate existing java bindins for the ios.

4

Hi, i’m currently trying the same but with moe-1.3.0-beta-2.

It seems that “Link third party framework to Xcode” doesn’t exist anymore - so I’ve done the steps from docs.opencv.org to integrate OpenCV into my Xcode project. Next I’ve added the java bindings from the OpenCV android SDK into my moe-project in Android-Studio but without the android specific package.

But when I try to initialize a simple OpenCV Matrix I get the following runtime error:

art E 10883 3295201 /Volumes/SSD/gh-moe-master-1.3.0-beta-2/aosp/art/runtime/java_vm_ext.cc:255] No implementation found for long org.opencv.core.Mat.n_zeros(int, int, int) (tried Java_org_opencv_core_Mat_n_1zeros and Java_org_opencv_core_Mat_n_1zeros__III)
2017-02-16 15:02:11.208787 ios-moe[10883:3295201] *** Terminating app due to uncaught exception ‘java.lang.UnsatisfiedLinkError’, reason: 'java.lang.UnsatisfiedLinkError: No implementation found for long org.opencv.core.Mat.n_zeros(int, int, int) (tried Java_org_opencv_core_Mat_n_1zeros and Java_org_opencv_core_Mat_n_1zeros__III)
at org.opencv.core.Mat.n_zeros(Native Method)
at org.opencv.core.Mat.zeros(Mat.java:891)
…

Best Regards,
Raimund

5

Hi,

There are a few points that you should check:

  • are you linking OpenCV statically or dynamically (e.g. in a framework)?
  • is the JNI code also linked into your app?
  • you have to make sure that the linker does not throw out your JNI symbols (because they are never called from the native code). You can check with nm for example. If you don’t see the symbols, then you have to link statically with the -force_load option.

If OpenCV (or its JNI part) is linked statically, and it is using JNI_OnLoad for initialization, then you will need to do the following:

  1. Change the JNI_OnLoad method to JNI_OnLoad_opencv
  2. Call org.moe.MOE.loadStatic() as follows from your main class for example, before you try to use OpenCV.
static {
   MOE.loadStatic("opencv");
}

Best Regards,
Gergely

6

Hi Gergely,

OpenCV (v. 3.2.0) comes as a “.framework” file which I directly dragged’n’dropped into the framework group of my Xcode project. So it is placed under the “Link Binary With Libraries” section in the build phase of my app target. I’ve also added some necessary system frameworks and a prefix header to include OpenCV in all cpp files. That’s all what I’ve done in the Xcode project.

The Mat class of the android java binding looks as the following shortened version:

public class Mat {
public final long nativeObj;
// javadoc: Mat::Mat()
public Mat() {
nativeObj = n_Mat();
return;
}
// C++: Mat::Mat()
private static native long n_Mat();
}

I’ve tried to annotate them like this:

@Runtime(CRuntime.class)
public class Mat {
public final long nativeObj;
// javadoc: Mat::Mat()
public Mat() {
nativeObj = n_Mat();
return;
}
// C++: Mat::Mat()
@CFunction
private static native long n_Mat();
}

With this implementation I get the runtime exception:
FAILURE: C callback not found for method n_Mat in class org.opencv.core.Mat!

Furthermore I don’t know how to add the Nat/J C++ processor to the build procedure.

Thank you for your time. :slight_smile:

7

The OpenCV Java bindings have a JNI side that you need to link to your project as well. I don’t think it will be in the OpenCV.framework, but you can check it with nm.

As the OpenCV Java binding is using standard JNI, you don’t need to use Nat/J annotations.

8

I didn’t find any OpenCV Java bindings in the framework, so I’ve used “javac” and “javah” to generate header files from my Java project and got now header files like this:

/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class org_opencv_core_Mat */

#ifndef _Included_org_opencv_core_Mat
#define _Included_org_opencv_core_Mat
#ifdef __cplusplus
extern "C" {
#endif
/*
 * Class:     org_opencv_core_Mat
 * Method:    n_Mat
 * Signature: ()J
 */
JNIEXPORT jlong JNICALL Java_org_opencv_core_Mat_n_1Mat__
  (JNIEnv *, jclass);

// …shortened…

#ifdef __cplusplus
}
#endif
#endif

I’ve added those files to my Xcode project and also added the “jni.h” file to my Xcode project.
Am I on the right way? What should I do next? When I get this to run I will write it down and post it here.

9

Not really, these C functions are already implemented by the OpenCV developers.

I checked in the source that they use a generator to generate the native side of the JNI bindings (some are handwritten like Mat.cpp):

https://github.com/opencv/opencv/tree/master/modules/java/generator

You need to include these files in your project. I assume that the Android bindings have a way to build these, you should find a way to build them for iOS as well.

10

Okay it seems that I don’t have enough experience to get rid of it now but eventually it is possible to implement all the OpenCV logic in the Xcode project and provide the functionality with a simple Objective-C class wich has self defined methods like:

  • (void)connectCamera;
  • (void)disconnectCamera;
    and callback methods like:
  • (void)cameraDidChangedPositionX:(float)x Y:(float)y Z:(float)z;
    and then connect only this Objective-C class over Nat/J with my android studio project?
11

Yes, this is also possible.

12

Here is just a simple way I made it working:

The situation:
OpenCV for iOS (downloaded version or build by yourself from sources) doesn’t contain the jni support layer needed fo Java integration.

Ways which didn’t work:
I tried to modify the OpenCV framework build process in order to include it - it didn’t work as intended and I gave up due to the complexity of the build scripts and the speed they are developing.
I tried to convince the OpenCV developers to add Java support to the iOS-framework - no reaction.

Thus I decided to do it pragmatically:
The idea is:

  • the jni-layer is a very thin one containing only wrapper code and conversion functionality
  • this layer can be added to the OpenCV-framework at that point in time when you are building your app

Prerequisites:
You need the jni support sources and an opencv-*.jar.

Solution:
If you download the OpenCV sources you will find some of the needed sources there:
opencv/modules/java/generator/src/cpp
(common.h converters.cpp converters.h jni_part.cpp Mat.cpp utils.cpp)
The missing sources will be created automatically if you build OpenCV for a platform where Java support can be activated easily - I did it on Linux.
Then you find the needed other sources there opencv.build/modules/java:
calib3d.cpp core.cpp features2d.cpp imgcodecs.cpp imgproc.cpp ml.cpp objdetect.cpp photo.cpp video.cpp videoio.cpp
The opencv-*.jar will be created in opencv.build/bin

Next steps:

  • add all these sources to the Xcode part of my MOE app
    (the following configuration changes must be done in the target-configuraton: Build Settings, Build Phases)
  • add OpenCV framework to Xcode part of MOE
  • add frameworks needed by OpenCV framework to Xcode part of MOE app: CoreMedia, CoreVideo, CoreGraphics, AVFoundation (sufficient for my case - maybe others could be needed too depending of OpenCV functionality used in app)
  • add needed framework and header search paths to Xcode part of MOE app
  • solve header include problems
  • in core.cpp comment out the following line: cv::setErrorVerbosity( … in a Method called Java_org_opencv_core_Core_setErrorVerbosity_10
  • remove features2d.cpp from Xcode sources (didn’t link and I didn’t need it -> an issue still to be solved)
  • create precompiled header project.pch containing the following lines:
    #import <Availability.h>
    #ifdef _ _cplusplus
    #import <opencv2/opencv.hpp>
    #endif
    #ifdef _ _OBJC _ _
    #import <UIKit/UIKit.h>
    #import <Foundation/Foundation.h>
    #endif
    (I had to add extra spaces in the underscore sequences to be displayed properly)
  • under Product/Perform Action compile pch-file
  • close XCode project
  • add the opencv-*.jar to the libs folder of your app
  • create a proguard.append.cfg in your ios module folder (if it didn’t exist already)
  • add the following line to it: -keep class org.opencv.core.CvException
  • build and run your MOE app with OpenCV based functionality :slight_smile:

Issue: The feature2d.cpp linking problem is still to be solved …