Electronic – How does the Microsoft Surface table detect objects

infraredlcdtouch-paneltouchscreen

I saw that the Surface table (now called PixelSense) detect objects.
The strange thing is that it's basically an IR screen with LCD on bottom.
How does it "scan" them then?

Here's a video
http://www.youtube.com/watch?v=I0QBivma6j8

Best Answer

drawing

A step-by-step look at how PixelSense works:

  1. A contact (finger/blob/tag/object) is placed on the display
  2. IR back light unit provides light (though the optical sheets, LCD and protection glass) that hits the contact.
  3. Light reflected back from the contact is seen by the integrated sensors
  4. Sensors convert the light signal into an electrical signal/value
  5. Values reported from all of the sensors are used to create a picture of what is on the display
  6. The picture is analyzed using image processing techniques
  7. The output is sent to the PC. It includes the corrected sensor image and various contact types (fingers/blobs/tags)

Source: Microsoft Pixelsense

Also, take a look at a similar conceptual implementation (i.e., IR-camera-based) by Johnny Lee (who in fact is now in the Microsoft team AFAIK): Tracking Fingers with the Wii Remote