TouchTable Multi-Touch Mouse Free Interface for Windows

Category: 
TouchTable Multi-Touch Mouse Free Interface for Windows

Written as a project for a PhD research project, TouchTable allows Windows users to use any flat surface as their own mouse free, multi-touch interface with the use of a Kinect sensor.

With this software you'll be able to use your finger to control the mouse cursor, left mouse click by touching the surface with your index finger, right mouse click by sticking your thumb out while taping with your index finger and drag and drop by holding your index finger down and dragging it across the surface.

The developers blog presents the following logic behind the applications functionality:

  1. When first frame is processed, the depth map is stored as “environment depth map”. Basically, I remember the distance of each pixel in the view of Kinect as table pixel distance
  2. When a frame with a hand on it is processed, hand’s pixels are recognized based on comparing the actual distance of a pixel with the table pixel distance from previous step. If the distance is bigger than a threshold (f.e. 1cm), pixel is regarded as a hand pixel
  3. From the pixels of hand I extract hand contour (using OpenCV library)
    From hand contour I calculate the fingertips, based on the angle between neighboring pixels. If the angle is sharp, it is a hint that the pixel is at the fingertip sharp arc.
  4. I apply some heuristic to match detected fingertips to finger indexes. In this process, I try to first match fingers detected in previous frame based on their distance from the newly detected fingertips. I try to minimize the distance.
  5. I apply also some smoothing by mean averaging the fingertip coordinates from last few frames.

The non-commercial demo is free of charge and is available in both Windows 64 and 32 bit binaries and comes with the source code for plugins. You'll also need to have the OpenNI framework installed. Click HERE for installation instructions.

You can download the demo HERE