Impressive
I would be happy with 15fps, at the moment I get about 3 - 5 with considerable lag on my setup which is a celeron 900 running Ubuntu 12.04 desktop.
The way I compensate for the lag so far is instead of attempting to track in realtime, when the object is noticed, it will tell the servos to move straight to that position where it last saw it and make that the centre of the screen. This is done by calculating an offset angle relative to the camera angle of view, I roughly calculated mine to be about +/- 11 and 8 degrees for horizontal and vertical. If it has moved away by the time it gets there, then hopefully it is still within the frame, if not, it goes looking.
The result is fairly jerky, but it matches what I see happening on the camera.
I’m hoping that the headless odroid running hardfloat ubuntu server will achieve better fps which should improve it a fair bit. (I compiled the kernel last week, odroid should arrive this week I hope.)
The other thing I could do is set up two threads. One simply tracks the object and populates the result which is read by a second thread that calculates the servo positions.
It wouldn’t be much faster, but it may free up some cycles to be used for the opencv side of things.