Some time ago (maybe a year (!)) I remember saying that I would post info on my project. So here it is.
Because I’m a lazy grad student, I’m just going to post the link to my research profile wiki page. There’s not a whole ton of information there, so if you have any questions, feel free to ask them here (or PM, or email).
Cool, was the camera used for object recognition?
No, the camera was just sent back to the user interface. It’s designed to be a remote user interface, so the person can’t see the robot directly. All they have is sensor data. So the camera provides a “gripper view” for close inspection.
The question probably is what type of sensor mapped the objects and how involved is the mapping process to generate sufficient data to generate a 3D representation of the objects.
The way the 3D scan is created is really rather simple, but it has its problems. We used a SwissRanger SR-3000 time-of-flight ranging camera behind the robot arm. Basically, the picture that it takes tells you depth at each pixel. It also gives you the xyz position of each pixel.
The tough part is lining up the 3D scan with the robot arm in the interface. And that’s especially hard since the sensor distorts the scene by about 2cm in some places, and that’s almost the distance between gripper fingers.
For the experiment I ran, I just tried to eyeball the alignment, but the alignment was bad enough that people weren’t able to grab the objects half the time without using the gripper camera (i.e. only the 3D scan). Since then we’ve put in a calibration routine that does a slightly better job, but it’s still not perfect. We also recently got a stereo camera in hopes that there will be less distortion, but haven’t integrated it in yet.