I decided to post here as a member since I have now spent many many hours without success to find what I'm about to ask.
I have an STM32F103 arm based board with 2.8" tft and a camera OV7670+fifo buffer memory bought cheaply from Ebay. Many hours I've struggled but I got it this far that I get qvga or qqvga (160x120 pixel) full color images from the OV7670 into my STM32 and show the image on the LCD. If somebody is interested, I'm willing to share the whole code.
Now...what my real goal is: not to get the stm32-ov7670 to work (i thought it was easy) but to do something with it. I'm open for anything, line following or maybe blob detection (follow a colored ball). The only thing I have now is 2-3 frames per second of 160x120 pixels in RGB565 format. I can depict the color values on the TFT, find specific pixel values etc etc but still....have no clue how to do something with it.
I've spent many hours with google and doing specific searches by using site:letsmakerobots.com. Either I see cool projects with absolutely no sourcecode or I find these cool openCV computer based projects which I find cool but I want to do something embedded.
Anyone, got a good suggestion? A lead to go on, like a good project you worked on? Maybe some good tutorial?
If you are developing in C/C++ then you can always bring the OpenCV platform into your embedded system, source code is available from Willow Garage… no promises on how efficiently it will run though =)
If you want to develop your own vision processing functions, Wikipedia has plenty of info under Feature Detection, although it can be more formal and math-heavy than it needs to be.
The first computer vision filter I remember learning about was the standard Sobel filter, I suggest you give implementing that a try. Get ready for a lot of matrix based functions, they’re everywhere in image processing.
Sadly there is no Delete for forum posts. What you type is here for prosperity. The best you can do is edit the comment to contain a ‘.’ or maybe even just a ’ '(space).
I am pretty sure the Sparkfun UGV contest utilizes traffic cones. Being able to home in on them for accurate cornering would be useful. You already mentioned line following. I have seen University robots that catch baseballs. I doubt 2-3 fps would be sufficient for that purpose though.
Tnx Birdmum, I got something to go on now! I am aware that my framerate is poor, I can crank it up when I do not display the frames on my TFT, that is probably what I do when I have an application fully working no debug required anymore.
I just want to learn the lowlevel stuff, I have 120x160 pixels and they represent info, now what to do with it. For a more powerful version I will revert to this openCV, I have a raspberry pi as well with an addon pcb STM32 (its called embedded pi)
Hi Telefox, another good idea which I was unaware of: I can bring in the OpenCV to my embedded system? Now that is something I will dig into tonight! I indeed develop in C.
The heavy math I’ve run into… thats not for me. I found tons of impressive 4-page papers of universities and I found 2 complete books in pdf about Robot Vision. As soon as the first Matrix calculations hit my head I felt at school again. That stuff I preserve for work in my free time I like to get my hands dirty and be as practical as possible.
I even found a book containing every imaginable vision/image processing algorithm there is, in C language. But here I stumbled over the problem not knowhing what to do with this algorythm. I will dig into the info you gave me and that of Birdmum. Hopefully I find some code in whatever language which I will dig into deep and translate to my system
There used to be an open source AVR Cam project, which used a camera that would send frame grabs to an on-board Mega 8 for processing and the microcontroller would talk with other hosts over serial (either computer or microcontroller). The output would be coordinates of the bounding boxes for the color the camera module is looking for. That was done on the camera module.
On your own microcontroller, having the bounding box, lets you determine the size of the blob and the center point coordinates of that box. Then, you can determine how far off-center the box is in the frame and move the robot or the camera (mouted on a head) to center the blob in the frame. If you know the height of the camera, the angle it is facing down to have the blob centered on the frame, you can calculate the distance to the object (represented by the blob) if the object is sitting on the floor, of course.
I have such an AVRcam module, but I haven’t been able with my limited programming knowledge to make good use of it. I also have an Arduino library, but it is limited and works awfully slow (slower than the camera can actually work).
Oh, it seems that the project’s page is still active, so you can take a look at the provided embeded code:
This is a big coincidence! Yesterday evening I was tracking down the lead I got from Birdmum and Telefox.
Telefox told me about openCV on embedded systems. When looking for that it quickly became clear that this would be a new struggle or challange (it depends how to look at it) because I found no one who had some compact embedded code availble (but I keep looking). While I searched the internet I found a post about Jrobot and checked it out. First impression is this is exactly what I need. So I already downloaded his AVR code and will take a look at it the coming days (its a big project).
I surely let you know what I do, I plan to write something about it on LMR as well. It frustrated me that there is no project or post that guides you through like a tutorial. I know its a lot of work thats probably why and the ones understanding how this all works might not be able at all to 100% put it on paper.
I will start with Jrobot and will go through the rest of your links as well. Man am I glad I put this post out, finally I’m making progress again
Nice, I’ve heard about the AVR Cam many times but I didn’t realise it was open source.
If you can get the AVR Cam code up and running maybe you could look adding some more processing options - colour-based blob tracking is nice, but it’s not exactly robust in real world conditions. I sometimes develop computer vision programs for work so I can give you some help if you want to try out some basic pattern/object recognition… could be the sort of thing you are looking for.
Haha TeleFox i just am starting to get the hang of this! I got wandered off the AVRcam because I checked the CMUcam as well, this one I heard about while looking into the other info.
I found an exact example of what I try to do as my “hello world” in image processing: find a color. Red in this example.
But I’ll keep your offer in mind for sure, If you are developing your own vision programs you are my master!
Right now i would be extremely happy if I can follow the red colored object with my stm32-ov7670 combination. My end goal would be doing basic object detection indeed but that will probably not succeed with the STM32, I think I need a bigger gun for that
I was digging into this piece of code which was written to follow the reddest object it detected. I think I mentioned it here in the discussion, its from the CMUcam website. There I saw some pixel magic which made me test my camera to see if its colors were ok. So I grab a qqvga frame of pixels and for a specific pixel at x,y I show its value on the TFT screen and also I take it apart in RGB565 components R,G and B. Then what I saw keeps me puzzled for some time now.
Black is not black, it should give me 0x0000 or R=0,G=0,B=0. Instead I get R=2,G=4,B=2. And I have 2 boards with the same OV7670 camera (and same code loaded) with same results. Since then I tried everything I could think of to make Black BLACK aah… To no avail… First I thought maybe some databits “stick” or something but if I take random measurements I learned that every of the 16 bits becomes 1 or 0 at a time. Just not all 0’s for Black. (The same goes for my white, its not decimal 65535 but 63000 something)
I tried whitebalance settings, black color corrections, R gain G gain B gain. Etc etc etc. Then just some OV7670 register settings from about 8 different sources which I collected during my searches. Does not change it.
I have seen some guys working with YuV getting the correct black values for their setup, that is decimal 0,128 for each pixel. I can not use this because my TFT is rgb565 and I have no code yet to pump it to a PC via the UART or something
So I come to you once again, is there anybody on the forum that can explain this behaviour. It seems to me if black is not completely black then red is not entirely red, right? So…my setup+code is useles until I fix this? Right? Or not?
Unless you have a pretty stable lighting environment it’s very erratic. I managed to get my AVRcam clone working a couple months ago but found it was too finicky for my uses. Slight differences in brightness was enough to make recognizing more than one color difficult at best.
I still have a few AVRcam boards for the clone I designed awhile back if you’re interested. They’re not pefect but they do work. For one the MAX232 serial converter doesn’t work for some reason but it speaks TTL just fine. Who uses serial these days anyways? lol I was using the ubiquitous FT232 converter board for communications with my computer via USB. Worked great.
It was kind of difficult and more expensive to find the C3088 board back then but they’re more available now. I recommend aliexpress if ebay doesn’t have any listed. IIRC they were something like $50 with shipping back then but I see they can be had for ~$30-40 now.
It took some time to figure out how to get the ATmega8 to work right but if you have difficulties I can offer advice, for sure.
If you want the naked board + switch I can sell you a set. What say… $10 incl. S&H? I’m able to provide pictures and whatnot if you like. The power switch I used was a unique model and it doesn’t seat 100% perfect due to my bad package design in eaglecad. It does work though (it just sits really high). Knock off a dollar if you don’t want it.
Or if you want you can buy the assembled cam module I have. I would have to tally up what it cost me but I would be willing to let it go for that amount plus shipping if you’re interested. I have a few unassembled sets so it wouldn’t break my heart
The cam is adequate for tracking one color but they are fussy about lighting and such. Tracking more than one color was a task in varying lighting conditions. You have to set the color ranges that it’s looking for and sometimes they overlap. Yellow in the dark looks a lot like orange in really bright light, for instance. Someone has a linefollower on this board that uses an AVRcam and it seems to work well in some applications.
It seems you have sort of the same issues with a totally different camera. Tnx for your offer, its a good offer but I stick to the ov7670 for a while. Not only because I bought 2 (with fifo) and 2 without but also the amount of effort/time it took me to get it to work…i’m determined to get this to work. Or live with its limitations. Maybe I do what bdk6 suggests. If I just ignore the noise error and put some boundaries then it might work.
I also have a camera from 4d systems, that one was supposed to be simple via the UART but somehow the camera was broken all along. Here I spent also many hours and in the end I got the 60 dollars back I paid. Its not even the money thats worst, its the time!!
Maybe when I’m ready to flush 2x OV7670 down the toilet I’ll dig into your AVR cam clone
It sounds like you’re on your way with the 7670. I’m excited to see success on your side. You have some big brains backing you and I see victory in your camp sometime in the future.
The offer is standing if you find you would like to tinker with an AVRcam after all. The bits aren’t going anywhere
Haha no I won’t flush it, had a good night sleep so back to business tonight after work. I’m an electronics developer so used to put my teeth in stuff
I’m going to let this thing do what I want. And if I get it right I plan to write an article about it and put it somewhere here on LMR. As we speak there are still people buying this cheap tempting camera and end up confused. The time it took me to get the SCCB interface working, pfff and then to get data out of it. I even had to route the wiring in a specific way: power supply + databus wires bundled together and all others from sccm/vsync line etc in another bundle and keep it seperated because they interfere. It results in some spacy infrared-like pictures on the TFT (like Predator haha). Not so handy if you are debugging and try to find out what settings work for my TFT and STM32.
…,.I also have a raspberry pi with the new Omnivision HD camera waiting for me to unpack. So if this totally flops then I’ll get into that. But I was waiting actually for some code/library/projects to pop up, I figured maybe on the Pi some other people can start it and I hop in later when I see some successful projects
The best you can do is edit the comment to contain a ‘.’ or maybe even just a ’ '(space).
