Install Keyboard Anywhere on Ubuntu Using libfreenect

If you've been following the Kinect hacking and development scene for as long as I have, there's a good chance you'll rememeber the Keyboard Anywhere hack. It was one of the first real eye openning Kinect hacks to hit the scene and for good reason. It combined the futuristic interface of an invisible keyboard that could be placed anywhere (hence the name) with a great memory from back in the day. I'm of course refering to that scene from Big where Tom Hanks and that other guy jam that song -- heart and soul I believe it's called -- on a huge floor piano.

Thanks to the generous nature of the developers, coupled with the great tutorial from instructables, we can now set this up ourselves! I took the time to set it up the other day and it works great! There were a few curve balls so I thought I'd place a guide on this site as well that follows the one from instrucatables but clarifies a few points. So without further adue, let's get started!

This guide used Ubuntu 11.10 installed in Parallels on a MacBook Pro. If you follow these steps to a T you should have no problems on either Parallels or on a dedicated Ubuntu machine.

Dependencies

First and foremost, you'll need to have libfreenect installed on your computer. I whipped up this quick guide the other day - Install libfreenect Drivers on Ubuntu. If things don't go according to plan, check out the original OpenKinect guide here

libfreenect comes with a Python wrapper that we'll need to install in order to get Keyboard Anywhere jiving with the Kinect. We'll install that in a second, but first we'll need to install a few dependencies.

sudo apt-get install cython python-dev python-numpy

Let the installation of complete and then navigate to the libfreenect wrappers directory. The Python wrapper setup file is located in /libfreenect/wrappers/python

Issue the following command to install it:

sudo python setup.py install

Next up, more dependencies!

sudo apt-get install pyqt4-dev-tools qt4-qmake libqt4-dev python-qt4-gl python-qt4-dev python-setuptools fluidsynth

Install PyQGLViewer

There are two ways to go about this. Easiest is to add the required repositories and install PyQGLViewer using apt, there's also the Manual Build and Installation of PyQGLViewer. If you decide to take that road, the developer left a comment on his instructables guide you may want to take note of here If the solution I documented here doesn't work, you may have to roll up your sleeves and do things the long way.

Issue the following commands in a Terminal in order to install PyQGLViewer:

sudo add-apt-repository ppa:christophe-pradal/openalea
sudo add-apt-repository ppa:christophe-pradal/vplants
sudo apt-get update
sudo apt-get install pyqglviewer

Install pyFluidSynth

Download pyFluidSynth from http://code.google.com/p/pyfluidsynth/
Extract it and navigate to the root directory and run the setup.py script

cd pyFluidSynth
sudo python setup.py install

Run Keyboard Anywhere

The finish line is near! Download Keyboard Anywhere from git hub at https://github.com/petermoz/keyboard-anywhere/ - click the ZIP button beside the https url

Extract the content and navigate to the root of the folder. Plug your Kinec tinto your machine and invoke the following command:

sudo python keyboard_anywhere.py

A box will eventually appear. If it looks to be a solid grey colour, that because you need to zoom right in in order to see the depth data. I was only able to do this once I tried it out on a different machine that just so happened to have a mouse with a center wheel/button! A lot of useful commands actually require a mouse wheel so if you have one lying around, I highly recommend you use it.

Usage

The developer couldn't have written it any better. This is was taken verbatem from the README file that accompanies Keyboard Anywhere:

Assuming all the dependencies are working ok, running the script will produce a seemingly empty grey window. This window is produced by QGLViewer, so you can press the 'h' key to access a help screen. A quick summary of that information is provided below:

  • Double click the middle mouse to centre the view (you should see some keys)
  • Press ENTER to start the displaying of kinect data (here's where it will crash if you didn't get freenect installed correctly, or your kinect isn't plugged in)
  • Left click and drag to rotate the view, right click and drag to move it
  • Scroll wheel zooms the view
  • Once you've figured out the view, and waved to yourself in live kinect data, press the 'f' key to display FPS information. If it's 30, that's great, you're keeping up with the kinect. If it's significantly less than that (say 10), playing the keyboard might not be much fun, and you'll need to try a faster computer.

Now, define where you want the keyboard to be:

  • Orient the view so you can see the surface on which you want to place the keyboard
  • Press ENTER to pause the kinect data
  • Press the '1' key
  • Holding SHIFT, left click a data point to set the lower left corner of the piano (Note: you must actually click on a point for it to work. You'll see a confirmation message when it does. If you're struggling to hit a point, try zooming out).
  • Press the '2' key
  • Again holding SHIFT, left click a data point. This sets the lower right corner
  • Press the '3' key
  • SHIFT + left-click another point. This will be used to set the upper corners of the keyboard. If this doesn't make sense, try it out and just see where the keyboard appears. It won't look right until you've set all three points.
  • To modify any of the points, just press '1', '2' or '3' and try again
  • Press ENTER to resume the live data. Chances are, the keyboard will intersect some of the data points (meaning that you'll hear notes, and the keys will light up green
  • To solve this, experiment with pressing and holding 'z' or 'SHIFT + z' to 'nudge' the keyboard away from the existing data until all the notes are off.

Hope this installation worked for you. If not, feel free to drop a comment and I'll see if there's anything I can do to help out. Enjoy jamming your virtual keyboard....anywhere!

Resource Guide Type: 
Groups audience: