OpenCV and a webcam stream: a fail and a win

I managed to compile OpenCV with FFmpeg support for the Chumby. Unfortunately this did not solve my problem, because FFmpeg is not able to decode the MJPEG stream from my web camera via mjpg_streamer. I've found an another solution that works, though! More on that in the end of this post.

OpenCV with FFmpeg does work with video files (I tested with an AVI DIVX file) and might work with the stream generated by some other web camera, so I'm including some notes on how to repeat the build procedure:

• I got the latest FFmpeg source code using git clone git://git.videolan.org/ffmpeg.git. This was perhaps a mistake, because it turned out that compatibility with OpenCV 2.2.0 had been broken at some point. Fortunately I found patches that fixed this issue in the OpenCV source code.
• FFmpeg had to be configured with the --enable-shared option. I used the command 

~/chumby-sw/OpenCV-2.2.0/release$ sb2 ./configure --prefix=/home/<my-username>/chumby-buildroot/usr --enable-shared

• I had to apply another patch to OpenCV-2.2.0 source code to fix some linker errors (../../lib/libopencv_features2d.so.2.2.0: undefined reference to `cv::SIFT::SIFT(double, bool, bool, int, int, int, int)' etc.)
• I used the following commands to configure and build OpenCV:

~/chumby-sw/OpenCV-2.2.0/release$ sb2 cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/home/<my-username>/chumby-buildroot/usr -D BUILD_PYTHON_SUPPORT=OFF -D BUILD_NEW_PYTHON_SUPPORT=OFF .. 

~/chumby-sw/OpenCV-2.2.0/release$ sb2 make

The working solution I mentioned above is simply to decode the motion-JPEG stream generated by mjpg_streamer manually and use some functions from libjpeg together with OpenCV's cvCreateImageHeader and cvCreateData to convert the individual JPG images into OpenCV's IplImages. I found some sample code that I could easily modify to suit my purposes.

Working on getting things to work on the Chumby

Recently I've been working on a couple of things related to my robot project.

1. Getting Chumby and Arduino to talk to each other via USB serial. I've successfully completed a simple test where the Chumby sends an ASCII message to the Arduino, and the Arduino then shows the message on its LCD and echoes it back to the Chumby. Still to be done: writing a proper communication protocol that will be used by SHORT-E.
2. Trying to build and install a working version of the OpenCV computer vision library for the Chumby. It has not been easy. The current status is such that I managed to compile and install OpenCV, but for some reason it cannot load JPEG or PNG images even though I have built libjpeg and libpng for the Chumby and they work ok with other software (at least Fswebcam is able to use them). And even worse, OpenCV cannot seem to get proper frames directly from my webcam, even though it manages to trigger the camera to capture something.

I had to use OpenCV 2.2.0 because I couldn't configure the latest version not to support Python - and OpenCV's stupid CMake configuration scheme makes it next to impossible to cross-compile a version that does support Python. And I do have Python on my Chumby.

Currently I'm able to use OpenCV with BMP images, and I've found a piece of code that does JPEG to BMP conversion using libjpeg. I'm still hoping to get OpenCV to support my web camera by trying to compile a version of OpenCV that uses FFmpeg. First I'll of course have to build FFmpeg for the Chumby.

I'm hoping to be able to implement some sort of simple tracking functionality using OpenCV. E.g. have SHORT-E find and drive to a red ball or even follow a moving person. Hopefully Chumby's processing power will suffice for something like this.