Kinect Chapter 15. Using the Kinect's Microphone Array. This chapter is not included in the book.
The previous chapter was about speech recognition, with audio recorded from the PC's microphone rather than the Kinect. Frankly, a bit of a cheat, but still useful. This chapter shows how to capture sound from the Kinect's microphone array, so there's no longer any need to feel guilty about using an extra microphone. The trick is to install audio support from Microsoft's Kinect SDK which lets Windows 7 treat the array as a standard multichannel recording device.
Care must be taken when mixing this audio driver with OpenNI, but the payoff is that the microphone array becomes visible to Java's sound API. My Java examples start with several tools for listing audio sources and their capabilities, such as the PC's microphone and the Kinect array. One of the novel features of Microsoft's SDK is support for beamforming – the ability to calculate the direction of an audio source.
The PDF file for the draft chapter (792 KB). Navigation: Dr. ROS hints. Note: This page is still under constant construction as we update our codes into the official ROS repository.
Please check back often for changes or news. Read the ROS and related documentation A comprehensive list of readings can be found at UPenn MEAM 620 site. Essential Readme for the Erratic Robot Platform Hardware and payload Specifications Erratic Base Sensors ROS and software Others Kinect doing human tracking and pose estimation. Example Code repository linked to be created soon. Useful Links. Www.umiacs.umd.edu/~cteo/umd-erratic-ros-data/README_Kinect. Matt's Webcorner - Kinect Sensor Programming.
The Kinect is an attachment for the Xbox 360 that combines four microphones, a standard RGB camera, a depth camera, and a motorized tilt.
Although none of these individually are new, previously depth sensors have cost over $5000, and the comparatively cheap $150 pricetag for the Kinect makes it highly accessible to hobbyist and academics. This has spurred a lot of work into creating functional drivers for many operating systems so the Kinect can be used outside of the Xbox 360. You can find a decent overview of the current state of people working on Kinect here. I decided to hack around with the Kinect partially because Pat Hanrahan bought us a Kinect and partially becauase I wanted to see if it had a good enough resolution to be used for my scene reconstruction algorithm.
I also think the Kinect has pretty significant implications for cheap robotics projects that I might one day work on. Drivers Audio Most people talking about the Kinect are focusing on the depth sensor. Painting. Kinect 3D Hand Tracking. Note: This page regards the demo version alone.
Please, find a library version here. This work got the 1st prize at the CHALEARN Gesture Recognition demonstration competition (Check also this link). The competition was organized in conjunction with ICPR 2012 (Tsukuba, Japan, Nov. 2012). By downloading this demo you agree to the bounds and terms described in this license. Note: You can download the 3D Hand Tracking library here. Strict system requirements: PC with at least 1 GB of RAM64bit Windows OSCUDA enabled GPU card (Compute Capability 1.0 and newer) with 256 ΜΒ of RAM and the latest drivers in place Also, if you are interested in performing a live demo, make sure that you have installed the x64 version of your RGB-D camera driver. The demo itself is provided as an installable package of Windows binaries Kinect 3D Hand Tracking Windows 7 x64 This demo relies on a few 3rd party dependencies: Running the live Kinect Hand Tracking demo: Please, start it from the Start menu or the Desktop.