[Note] No up-to-date AR libraries?

Posted on by
2

Somehow there doesn't seem to be good AR libraries which are either working well or even up to date. It looks like there where some buzz between 2007 and 2009. Quiet some people developed libraries and applications using marker-based AR. Like the ARToolkit which was last updated in 2007. The successor (or at least an extended version) of that is ARToolkitPlus, which was actually updated in February this year. But well, both work just with markers, which I don't need at all. I thought I could hack the code a little to extend it to work with natural features (FAST corners or something like that), but the creepy C++ code is just ugly and would probably require too much time to go through. There is also AndAR, but this just uses ARToolkit internally and last update sometime in 2010, so no real help for me. NyARToolkit looked nice in the beginning. It's similar to ARToolkit (and I somehow guess that there is at least one person working on it who was also working on the other library) but a bit more lagging. I could work with that -- if the code wouldn't be almost completely commented in Japanese. Again, no help for me.

But there is more. Qualcomms QCAR augmented reality SDK is well up to date and working incredibly good. Though, there is a huge problem: If you want to track images with natural features you'd need to first upload your images, let them go through Qualcomms "Image Target System", download some binary file (which has most likely interest points from the image in it) and then compile everything into the Android application. I mean, come on! How useful is that!? So I cannot use QCAR either, because I want to dynamically load interest points and other information during runtime.

What is left? I guess I need to try it on my own. OpenCV has all the functionality I need (There is this Android-optimized version which compiles nicely with the NDK). I "just" need to use it. JavaCV is Java-based wrapper to the OpenCV API, it is not commented at all but hopefully it will be of any use for me.

If there is someone out there with (helping) suggestions, please drop me a line in the comments.

Review: „Multiple Target Detection and Tracking with Guaranteed Framerates on Mobile Phones“

Posted on by
Reply

The described related work tracks motion using optical flow algorithms. It seems that those produce satisfying results but not yet cover the full potential of a AR tracking system. Others use interest-point based algorithms which are commonly known as very computational expensive. SIFT descriptors are probably the most used ones although they might belong the the most expensive ones. Nevertheless, some improvements have been achieved with SIFT and also SURF algorithms.

Similar to PTAM Wagner et. al also use a separated detection and tracking system. The detection system tries to find known targets in the currently available camera image using a modified SIFT algorithm. Instead of calculating the kind of expensive Differences of Gaussian (DoG) they use a FAST corner detection over multiple scales. Memory consumption is then reduced by using only 36-dimensional features instead of the original 128-dimensions of SIFT. Found descriptors are matched with entries from multiple spill trees, which is a similar data structure like the k-d-tree used in the original SIFT.

-- unfinished --

javacv

Posted on by
2

JavaCV is a wrapper for a couple of libraries, including OpenCV. They also provide a pre-compiled OpenCV for Android. Looks promising. Will take a look at it. Wonder if this slows down the application somehow, because they are some more method calls because of the wrapper stuff I guess. But if it speeds up the development process I could live with that for now.

OpenCV-Android

Posted on by
1

At github I found an optimized version of OpenCV for Android. Different to the original Android version this one can be compiled with the Android NDK (except with those creepy work-arounds described at opencv/AndroidTrunk).

Unfortunately, the build process took like hours: because some variables needed by the linker aren't set on Windows (but obviously on Linux). So after looots of trying it worked finally.

Sum of Squared Differences

Posted on by
Reply

The Sum of Squared Differences (SSD) is a way of determining the correlation between two image regions. It is usually involved when motion compensation needs to be done. SSD is defined as followed:

\sum\limits_{i,j\in W}(Image_1(i,j) - Image_2(x+i, y+j))^2

There are some variations of SSD. Like the Zero-mean Sum of Squared Differences (ZSSD):

\sum\limits_{i,j\in W}(I_1(i,j) - \overline{I}_1(i,j) - I_2(x+i, y+j) + \overline{I}_2(x+i, y+j) )^2

See also: