My original intent was to use this hardware as an opportunity for the kids at my sons robotics club to learn how to evaluate something and to hone their critical thinking skills. We ended up with a younger set of kids mostly middle school and freshmen in high school who just didn’t yet have the skills to do the review justice so I felt it made sense for me to do it.
The result then is that this took a lot longer since I am a busy guy. I apologize how long it has taken. I keep thinking that I should play with it a bit more and try different use cases, but I never get to it. So, the result is I have been sitting on this over a year.
Overview
The Pixy CMU Camera is a sensor that can be configured to view colored objects. It is easy to setup, easy to use and within minutes one can be identifying colors and where they are on the screen. This information can be transmitted to a processor such as an Arduino or other similar devices.
The Pixy CMU Camera makes it very easy to track and identify solid colored objects in consistent lighting conditions.
Before It Arrived
Even after 16 years as a software engineer, I love anything shiny and new. So it was with great excitement that I looked up the info on the Pixy CMU Camera. I have some experience playing with vision programming on OpenCV, but I am certainly no expert on it. I played with OpenCV and was frustrated at how any change in light would make the color change for OpenCV. I was impressed that their documentation stated very clearly that their advanced algorithms had figured out a way to ensure that across varying light conditions, it identified colors consistently. I was never able to get OpenCV to do that. I also liked the interface. The directions stated that the object should be in the middle of the camera’s view, press a button on the side, and it would now know to look for that color of object. Using an Arduino or other MCU, it would give a pixel position and object number back to the MCU using SPI or other protocol. I couldn’t wait to get my hands on it!
First impressions when opening the box were that the item was very small, compact and thin. The box it came in was slick with nice graphics which gave the impression of a very professional release. Someone had obviously put some thought and hard work into packaging which seemed to bode well for the actual device. Inside was some nice mounting hardware which would make it a simple addition to all but the smallest robots. Any robot would consider the Pixy CMU a handsome addition to their robot.
Testing
I installed PixyMon on my Windows computer and plugged in the Pixy to the usb port. PixyMon is their tool to help configure the Pixy so it knows what to identify and look for. One points it at an object about 6-12” away, aims to put the desired object in the center, holds the button down on the side of it and waits for the Pixy’s LED to become the color of the object for which you are looking. Release the button and the Pixy will know that is the color object for which it is intended to search.
It took about a half hour of tinkering, starting and restarting PixyMon before PixyMon finally recognized the Pixy CMU. From reading other people’s experiences with the Pixy, they didn’t have these issues so it probably is something specific to the setup on my computer. With my laptop which runs Windows 10, it seems to take some fiddling, plugging and unplugging the Pixy from the USB port for it to recognize that the Pixy is connected, but I am always able to get it to work. Other times, I plug the Pixy in, startup PixyMon and it comes right up. According to the instructions, it can take a while to get good at setting up the Pixy and that eventually one may not need to use PixyMon to setup the Pixy. PixyMon also has a set of parameters that can be tweaked which will enable it to give better search results.
I tried a number of different objects with varying levels of success. Objects with multiple colors were by far the least successful searches since it hones in on one color only. The Pixy can be trained to find particular colors but seems to get confused with multiple colors especially with colors that are different shades. The Pixy did the best with objects that are a solid color.
One object I tested was the dark red trackball from my mouse. I placed it on a white notebook to ensure it would have better contrast. After a bit of playing with it, I finally got the Pixy to find the object. I was looking at this in my dining room which has a dimmer, so I tried to dim the lights to see what the difference in recognition is. Here is a screenshot from PixyMon with the lights full on. The number of squares on the red ball indicate how well the Pixy identifies the object, that is the more squares the better recognition of the object.
The dimmer lights definitely had an impact on the Pixy finding that red ball. In the last example with the lighting at its dimmest, it couldn’t find the object at all.
Brightest:
Middle brightness:
Least bright:
I tried several different objects with varying colors to test the Pixy’s efficacy. After several hours of playing with the Pixy and the dimmer in my dining room, it appeared that the lighter colored objects were less sensitive to light changes (kind of makes sense doesn’t it?). The Pixy’s recognition was definitely sensitive to changes in lighting, but definitely an improvement over just using OpenCV looking for blobs of particular colors as I have in the past. With OpenCV, just about any changes in light would bring about a color change and make the blob unidentifiable since OpenCV would treat it as a different color.
I also downloaded the Arduino sample code and pretty quickly had the little robot I am working with following the red ball with the light in the dining room on full. I changed their code only slightly and had my little bot chasing it around in just a few minutes. The integration and samples into Arduino, Raspberry PI and BeagleBone Black appear to be outstanding. I only worked with the Arduino but other comments I have read online speak to how easy the integration is with the RasPi and BB.
One thing I wanted to comment on was how surprised I was at how “washed out” the images the camera returned. The camera that comes with the Pixy seems to have a poor color saturation. I used a $5 camera I had bought a few months ago and compared the color saturation. As you can tell, there is a substantial difference in the color returned. I used the brightest lights on the dimmer for the below image.
It seems to me that if the device uses color as a way to identify objects on the screen, that a better color saturation on the camera would lend to possibly better and more accurate results. There may be a technical reason why they used that camera for the Pixy CMU Camera, but it seems a strange choice.
Conclusion
From a technology perspective, the Pixy CMU Camera is a pretty nifty device. The interface to set it up is easy to use and works well within certain parameters. The programming samples, at least for the Arduino, are easy to use, and I had it working within an hour. It took longer to read the directions than it did to get it working with the Pixy.
The Pixie CMU Camera has a use case where it makes sense. If you know ahead of time that the light will be consistent, that the objects will be a solid color and what those colors will be, this is an excellent option. The interface makes it very easy to setup for situations exactly like that. For instance, an M and M sorter, ball sorter, LEGO sorter or working with anything where the light doesn’t change drastically and the objects to be found are SOLID colors. For instance, someone at our local Nashua Robotics Group had a robot that used a Pixy that would find and pick up a solid colored blue ball. It was a situation where the lighting didn't change and the object was a solid colored object so it worked very well.
So, within those parameters, I definitely recommend the Pixy CMU Camera. Its application is fairly narrow, so be careful with how you decide to use it.