I’m currently experimenting a bit with the Kinect depth camera, which I intend to use for a prototype. I’ve played around with the device before on Windows using the Microsoft Kinect SDK. Setting up libfreenect on Windows is actually quite a pain, which was the reason why I started with the Microsoft SDK. As I wanted to code in Python, I’ve had a look at PyKinect and the Python Tools for Visual Studio (see also this PyCon US 2012 talk about building a Kinect game with Python). Unfortunately, the PyKinect samples seem to be outdated, and don’t work with the latest version of the Kinect SDK (version 1.0, released in February).
So I decided to see how far I would get with the Kinect and libfreenect on Ubuntu (12.04). Step 1: install the libfreenect demos to test if it works (this will also install libfreenect and its dependencies).
$ sudo apt-get install libfreenect-demos
So far so good. Unfortunately, the demo application wouldn’t start:
$ freenect-glview Kinect camera test Number of devices found: 1 Could not claim interface on camera: -6 Could not open device
Strange.. I did another check to see if my user was in the plugdev group (which
was the case):
$ groups jo jo : jo adm cdrom sudo dip plugdev lpadmin sambashare
Then I noticed that the Kinect’s LED kept blinking continously, which usually means there’s some sort of connection problem. It was correctly recognized, though:
$ lsusb ... Bus 002 Device 007: ID 045e:02b0 Microsoft Corp. Xbox NUI Motor Bus 002 Device 008: ID 045e:02ad Microsoft Corp. Xbox NUI Audio Bus 002 Device 009: ID 045e:02ae Microsoft Corp. Xbox NUI Camera
After a quick web search for the specific libfreenect-glview error message, I learned that recent versions of the Linux kernel prevent libfreenect from claiming the Kinect as they now include a driver to support the device as a regular webcam (see kinect.c), which is actually quite cool. This also means there should be a specific video device for the Kinect (/dev/video1 in my case).
The kernel modules are indeed loaded:
$ lsmod | grep -i gspca gspca_kinect 12936 0 gspca_main 28366 1 gspca_kinect videodev 98259 2 gspca_main,uvcvideo
Let’s try playing the camera stream using GStreamer and Video4Linux:
$ gst-launch-0.10 v4l2src device=/dev/video1 ! video/x-raw-yuv ! ffmpegcolorspace ! xvimagesink
That seems to work!
The Kinect actually has two image sensors: an RGB camera and a depth sensor, which consists of an infrared laser projector and a monochrome CMOS sensor. A depth map is created by projecting structured infrared light and capturing the resulting image from the monochrome sensor. As I wasn’t sure how to grab an image from the monochrome sensor, I contacted the author of the kernel module (Antonio Ospite). He told me it’s possible to get a monochrome image by specifying an image size of of 640×488 pixels (instead of the usual 640×480). Note that the kernel module currently only supports video streams from the monochrome sensor as unprocessed output.
If we pass that specific width and height to GStreamer, we get this:
It’s also possible to get the full-size (1280×1024) monochrome image. However, most webcam apps will just show the video stream from the RGB sensor for that resolution as you can’t specify that you want the specific Y10B format. To do that, you can use a separate program like qv4l2, which can be installed as follows:
$ sudo apt-get install qv4l2
To get the full-size monochrome image, run qv4l2, open the /dev/video1 device in raw mode (File > Open raw device), select 1280×1024 for the frame size, and “Y10B – Y10B” for the capture format, and click the red capture button:
OK, back to running the libfreenect demo. Someone over at superuser.com suggested to remove the modules (so that the user-mode libfreenect driver could
take over) and run libfreenect-glview again.
And indeed, after removing both gspca modules, the freenect-glview demo did work:
$ sudo modprobe -r gspca_kinect $ sudo modprobe -r gspca_main $ freenect-glview Kinect camera test Number of devices found: 1 GL thread Write Reg 0x0006 <= 0x00 Write Reg 0x0012 <= 0x03 Write Reg 0x0013 <= 0x01 Write Reg 0x0014 <= 0x1e Write Reg 0x0006 <= 0x02 Write Reg 0x0005 <= 0x00 Write Reg 0x000c <= 0x00 Write Reg 0x000d <= 0x01 Write Reg 0x000e <= 0x1e Write Reg 0x0005 <= 0x01 ...
The left side of the window shows a colored depth image, while the right side shows the RGB image from the camera. Of course, it would be quite cumbersome to remove the modules again every time you want to use libfreenect. One option is to blacklist the gspca module as follows:
$ echo "blacklist gspca_kinect" >> /etc/modprobe.d/blacklist.conf
Now that we've got that sorted out, I'll try out the libfreenect Python wrapper (or perhaps SimpleCV).
Update: Antonio Ospite pointed out in the comments that recent versions of libfreenect (0.1.2) can automatically detach the kernel driver. There's a PPA available by Florian Echtler with updated libfreenect packages for Ubuntu 12.04.
Avi Cohen !
March 31, 2012 — 20:53
I love you ! Thankssssssssssssssssssss
Shonn E
April 11, 2012 — 21:35
Thank you so much for figuring out the “Could not claim interface on camera: -6” problem Jo. I have been fighting this problem for a couple of weeks and searching blogs and help sites and nothing I ever tried worked. I had tried “sudo modprobe -r gspca_main” but it wouldn’t execute the command because it said gspca_main was in use. I had not found anything that stated you had to do the “sudo modprobe -r gspca_kinect” as well … and to do it first! This instantly solved my problems, and freenect-glview works now. I had tried creating a blacklist, but the person whose instructions I was following said to blacklist “gspca_main” not “gspca_kinect”. When I did so, it actually prevented my Ubuntu load from booting up! I have a dual boot set up where I can boot to Windows7 or Ubuntu 11.10, and after setting up the blacklist of gspca_main it would just sit and spin while trying to load and never come up. I had to boot to a sort of “safe mode” and remove the blacklisted file to get back up and running.
Jo Vermeulen
April 12, 2012 — 14:10
You’re very welcome! I mostly wrote this post to document how to get libfreenect to work on Ubuntu, so I wouldn’t forget myself I’m glad you got it to work eventually, at least it’s easier to work with libfreenect on UNIX systems than on Windows
anthonyoliver
May 2, 2012 — 19:49
One of the core SimpleCV (http://simplecv.org) developers here. Just upgraded to ubuntu was happy they are packaged now, but seems not quite. Thanks for the help and mention for simplecv.
Jo Vermeulen
May 3, 2012 — 00:44
You’re welcome! Thank you for mentioning this blog post on Twitter
Antonio Ospite
May 11, 2012 — 14:52
Recent libfreenect can detach the kernel driver itself, we have this version 0.1.2 in Debian, but I don’t know what version ubuntu is shipping.
Jo Vermeulen
May 11, 2012 — 15:40
Thanks for pointing that out, Antonio!
It seems version 0.1.2 is available in the current development release of Ubuntu (12.10 — Quantal Quetzal). Version 12.04 ships with version 1:0.0.1+20101211+2-3 which is a snapshot from late 2010, I guess. I just found a PPA by Florian Echtler with up-to-date packages though. I’ll try that and see whether it works!
TiagoBarufi
May 22, 2012 — 21:03
beautiful post and tutorial. Thanks for sharing!
Jo Vermeulen
May 24, 2012 — 12:17
I’m glad you found it useful!
Stefán Stefánsson
May 29, 2012 — 15:51
Hi and thanks for the writeup.
I installed the 0.1.2 version of libfreenect but now I’m having permission problems. I have to run all the programs as root (sudo) to get them to connect to the kinect.
I don’t know why this is happening because before I installed the new version (when I got it running by removing the kernel modules as you suggested) I was able to run the demo programs as a regular user.
-Stefan Freyr.
Jo Vermeulen
June 7, 2012 — 13:48
Indeed. I also have to run the libfreenect demos as root to be able to use the Kinect. I guess that’s because the program needs permissions for removing the gspca kernel module.
Alex Wallar
June 22, 2012 — 04:20
Hey man,
This post was super useful! I have been bugged by this disappointment for days but now everything seems to work
How did you get so good at using Linux, is there somewhere I should start?
Jo Vermeulen
June 25, 2012 — 11:22
By learning from others and by reading the built-in documentation (e.g., manpages), I guess. A more detailed answer might be a good idea for a next blog post
Homar6
July 23, 2012 — 22:38
You’re my master… thanks!!
Donald Woods
August 8, 2012 — 09:21
Agreed with you))
essaybase.co.uk
August 10, 2012 — 16:15
Thank you very much for your help! Now I have all figured out!
Rick
September 6, 2012 — 04:43
Thanks for the tip on gspca_kinect – I had the same problem with 12.04, and your advice
resolved the issue. Real time saver – thanks!
Chris
September 27, 2012 — 12:58
http://essay-writing-service.co.uk/
research paper
October 1, 2012 — 14:45
You’re my master… Now I have all figured out!
dirtydevil
November 29, 2012 — 07:26
Hi, Thanks for such a detailed post for Kinect Beginners.
Though my freenect demos are working but on trying lsmod | grep gspca , I am not getting any gspca_kinect or anything. Can you tell me please why I am not getting this. Plus I am getting only one /dev/video0 which is of my integrated Webcam . I am not getting any file for kinect like /dev/video1 . I think without it I can’t use openCV or any other processing. Thanks for help in advance. Please Do reply ASAP.
Alex M.
December 30, 2012 — 00:04
very nice tutorial.
any ideas why after removing both gspca modules, the freenect-glview demo did NOT work giving the “Could not open device” error? I am on Ubuntu 12.04
Carl Nobile
May 8, 2013 — 17:07
I have just done an update to my Ubuntu 12.04 and the Kinect stopped working. I had compiled my own opencv and libfreenect, so I uninstalled them and put on the distribution versions and the same issue persists. Has anybody run into this problem?
When I run glview I now get:
$ glview
Kinect camera test
Number of devices found: 1
Could not open motor: -3
Could not open device
~Carl
Jeremy Blair
May 25, 2013 — 20:22
Nice tutorial, but I’m getting this message when I try to do freenect-glview
Kinect camera test
Number of devices found: 1
send_cmd: Bad magic 00 dc
freenect_fetch_zero_plane_info: send_cmd read -1 bytes (expected 322)
freenect_camera_init(): Failed to fetch zero plane info for device
Could not open device
Not quite sure what it means, any help would be appreciated.
Daneel Ellinggton
July 7, 2013 — 12:16
i’ve got this issue too, what can i do ?
Jyeo
October 2, 2013 — 01:03
I’m getting the same issue, anyone have any suggestions?
Fatfreddy
October 10, 2013 — 10:45
Same problem, can’t open it as regular video device, as dev/ video1 or video0
ZB
May 7, 2014 — 07:04
Thanks bud, this was great – I still run into the issue of not being able to open though on Fedora 19
Young Zig Zag
October 21, 2014 — 03:24
I am currently having this issue. Did you ever resolve it?
Pratik Anand
January 31, 2015 — 22:00
You have to disconnect and then reconnect the usb of kinect. It be remounted as /dev/video0 (in my case)
berg
October 18, 2017 — 21:50
You can do:
sudo modprobe gspca_kinect
sudo modprobe gdpca_main
but will then loose freenectivity.