Leap Motion Controller and Advanced Human Input Devices

Leap Motion Gesture Controller

RobotShop recently received samples of the new Leap Motion gesture input device and see incredible potential for use with manipulating virtual objects as well as for control or physical robots. When you first see the Leap Motion device on its own, it's almost impossible to know what it does. This little device, once connected to a PC, has the ability to track your hands in three dimensional space. Simply place your hands in the air above the device and the location of each of your joints, your palms, and the location of both hands (above the sensor) are picked up and displayed on the monitor. People have already programmed the Leap Motion controller for use with robots, including Lynxmotion robotic arms: Technology which may have been considered "advanced" and cutting edge only a few years ago has become commonplace today, leaving us in awe of what may be released next. The computer keyboard and mouse have served us well up to now, and it's difficult to change something which so many people are so familiar with. However, there are other options being released, like the Leapmotion sensor which allow for new kinds of intuitive user control and input. We'd like to take this opportunity to talk about a few of the other advanced human input devices. Although most of the devices below were designed for use with computer software programs like 3D CAD and gaming, it is easy to see how they have potential to be adapted for intuitive robotic control. Although the trend in robotics seems to be to make robots more autonomous, being able (at least to a certain extent) to "manually" control a robot is always desirable.

PS2 / Gaming Control

The Playstation 2 gaming console has come and gone, but the handheld PS2 remote control has continued to be a valuable tool for robot control due to the number of buttons present, low price and ease of use / integration. Although it was originally designed solely for the Playstation, third party manufacturers have made USB versions of the PS2, and also adapters which can connect standard wired or wireless devices to a computer. These are not necessarily advanced, but because of their ease of use, they are popular for sending analog and digital signals to a computer.

Air Mouse / Keyboard

As computer based home entertainment systems became popular, using a standard keyboard and mouse from the comfort of a sofa became cumbersome. Manufacturers seem to have resolved this by creating a small wireless keyboard with integrated mouse functions. The mouse aspect is either using a track pad or ball (as you would find on a laptop), or by incorporating an accelerometer / gyro or IMU where moving the keyboard up/down and left/right controls the 2D position of the mouse. These devices are becoming increasingly popular with single board computers like the Raspberry Pi since they do not require any "hacking", are compact and tend to be inexpensive.

3Dconnexion Mice

Unlike a normal computer mouse which uses a trackball or infrared sensor, the 3Dconnexion series of input devices allow for intuitive 3D navigation. It's advertised by the manufacturer as the "world's first wireless 3D mouse". Several of these devices include a 6 degree of freedom IMU to track movement. Not only are devices like this great for use in virtual environments, 3D CAD and object manipulation, there is also the potential for using the device to control objects in the real world.

Falcon Force Feedback 3D Controller

The Falcon Force Feedback 3D controller takes a different approach to input devices by using three arms connected to the actual input device. Motors in each of the arms provide resistance and vibration, giving you feedback. Consider adapting this device to control a robotic arm and getting actual force feedback.

NeuroSky MindWave Mobile EEG Sensor

No longer the realm of science fiction, the NeuroSky line of EEG sensors measure brain activity and even know when you have blinked! They are not quite at the stage where they knows your thoughts, but commercial applications for this technology have included controlling a real UAV, computer games and more. As this technology advances, it's hard to image (pun intended) what it won't be able to do!

Webcams / Cameras / Kinect

Long gone are the days when webcams were used solely for online video; as computers and software have become more powerful, inexpensive webcams have been used not only track objects, but also to scan 3D objects! The Microsoft Kinect was originally designed to track a gamer's motion for a more immersive gaming experience. It uses a camera and a grid of IR dots to get a sense of 3D space. Fortunately Microsoft designed it with a standard USB connector, and the community quickly hacked the signal to be able to display what the camera "sees". To be used as an input device, the computer compares what it sees (in 3D) to a simple human shape and tracks the motion. Enthusiasts have not only been able to track a human operator's motion and transfer that to a robot, but also to mount the sensor on a robot so it gets a real-time 3D map of its environment. https://www.youtube.com/watch?v=_VbT8a0OJsE