Texture Mapping a 3D Object Using Kinect, RGBDemo and Blender

There aren't very many instructional guides or videos out there that go into detail when describing how to take a kinect captured image, import it into blender and then map a texture to it. Trust me, I spent a lot of time looking around. That goes double for Mac users. I've never used Blender before and right away I can tell you it doesn't boast the most straight forward and easy to use interface that I've come across. So, as a favour to all you Kinect hacking junkies out there that may have wanted to try out this sort of things, I whipped up this little tut with perdy pictures in hopes that it would help out a bit.

Two resource in particular allowed me to finally get it working. Props go to Brian Jepson's "From Kinect to MakerBot" guide that taught me the interface controls on a Mac --Pinch to zoom, rotate and some control key functions-- along with this YouTube guide.

If you're not familiar with using Blender, you're already at a huge disadvantage. The interface goes beyond intimidating as you're presented with so many options you won't know where to start. This guide will not get into describing the option or features, just how to blindly go from point A to point B such as I did. If you're interested in learning more, be sure to visit their site for tutorials and wiki's galore at http://www.blender.org/education-help

Installing RGBDemo

Nicolas Burrus created a wonderful tool for capturing and manipulating Kinect depth data called RGBDemo. There are a ton of options you can use to tweak your 3D captured image. You can save the file as a .ply file as well which you can then imported into Blender.

Download the Mac Binary installer from here - http://nicolas.burrus.name/index.php/Research/KinectRgbDemoV6?from=Resea...

Mount the DMG and drag the RGBDemo folder in the Applications folder.



Running RGB-Viewer and Creating a .ply File

Navigate to your RGBDemo folder in Applications. Ensure your Kinect is plugged into your computer and that it is also plugged into a power outlet. Locate and open up the rgbd-viewer application.

NOTE: You will need to have the OpenNI/NITE or libfreenect middleware installed before you'll be able to use RGBDemo. For more please visit the official site.

Open up the 3D View and Filters windows by click on the Show option.

Adjust the Depth Threshold Filter so that the background turns black and the object you plan on capturing are still visible.



In the 3D Viewer window, select Triangles and then SaveMesh when you're ready. This will create a current_mesh.ply file and an accompanying current_mesh.ply.texture.png image file in the RGBDemo folder in Applications.





Importing current_mesh.ply and current_mesh.ply.texture.png into Blender

Download Blender if you haven't already.

Import the .ply file by going to File -> Import and select current_mesh.ply from the RGBDemo folder located at /Applications/RGBDemo/current_mesh.ply





Continue to zoom in until you have located your imported current_mesh.ply

NOTE: Since I don't know if an actual command exists, I had to use the pinch method on my MacBook Pro's Trackpad. [Pinch IMG]



Once you've found it, you'll need to use the rotate trackpad motion [Rotate IMG] along with holding the option+two-finger-swipe motion in a up and down motion to rotate on the Z axis.

Do this until the depth image you took is centred and facing in the same direction it was captured -- i.e. front facing.



You'll now need to split the screen into two parts. One will be for the UV/Images Editor Type and the other will be the original .ply in Edit Mode.

NOTE: To do this, I had to click and drag the marked corners of each window. This will open up a duplicate that you can change around in order to view two separate screens that are being used simultaneously.

We'll now need to change a few values such as the Editor Type of the new window and the mode of the original 3D view. Change the window on the lefts editor Type to UV/Image Editor. Change the original 3D View on the right to Edit Mode apposed to Object Mode.



Now we'll need to create a new Material and Texture. Click the round orb like icon in the tools area located over to the right. I'm not even 100% sure if this step is necessary since all I did was change the name from Material.001 to something else.



To add a new Texture, you'll need to click the square red and white checkered icon beside the orb that represented Materials. Add a New Texture by clicking on the '+' sign. Give it any name you'd like. Change the Type to Image or Movie. Set the Mapping Projection to UV and finally, upload the ccurrent_mesh.ply.texture.png file located in /Applications/RGBDemo



Once everything is loaded up, our next move is to add the image to the UV/Image editor screen on the left and then switch from Edit Mode to Object Mode on the right along with changing the Viewport Shading to Textured.



Not quite sure to what to do from here. I guess if you were really into this sort of thing you could export or save the 3D texture mapped model you just created for use in another application such as Unity. Either way, that's the gist of it. I'm sure there will be tons of questions so please feel free to drop a comment and I'll do my best to help out.

Resource Guide Type: 
Groups audience: