Canesta’s 3D “CanestaVision” camera technology will be of particular appeal to game developers because it allows them to develop applications that can repeatedly and reliably sense body movements and gestures without any external constraints, such as strict control of room lighting, the need for high contrast or texture-free clothing or backgrounds, or arduous user training and tagging procedures.

Canesta accomplishes this by providing a continuous stream of color video frames that contain not only the familiar RGB (red-green-blue) picture elements, but also depth information – the distance from the camera to each pixel in the scene. This representation is sometimes called “RGB+Z”, where “Z” refers to the camera distance to an artifact in 3D space.

In a game application, the stream of RGB+Z frames are intercepted by Softkinetic’s middleware, which performs additional processing that maps gestures and body movements into specific commands or events, and then presents the game developer with a simple API (application program interface).

Canesta’s technology works independent of lighting conditions and is not impacted by nearby objects that blend into the background due to similarities in texture or color, and does not require a laser. Because of such capability, a hand gesture, for example, can instantly be discriminated from moving objects further away, as the camera – and ultimately the game software – “knows” exactly where that hand is in space. For immersion applications or games, where the player’s moving image is actually lifted out of his or her living room and inserted into the on-screen action in a “virtual” green-screen kind of a way, such fine-grained 3-dimensional discrimination is crucial.

“With four years of research and development, Softkinetic’s iisu technology is able to capture both full body movements, as well as slight move of the hand, opening the door for developers to create new and unlimited interface options,” said Michel Tombroff, CEO of Softkinetic. “The combination of Canesta’s 3D color video technology and Softkinetic’s iisu middleware will enable game developers using the tools to focus entirely on their application and the gameplay.”

Canesta created the entire category of electronic perception technology – inexpensive, single chip 3D electronic vision – in 1999, and announced the first, low-cost depth-sensing chips not long thereafter. Since then, developers in several key industries, particularly automotive, have been readying applications that depend upon robust depth sensing from the tiny 3D cameras. For computer gaming applications, or other immersive applications such as “alternative reality” video chat – where the on-camera participant is “dropped” into any still or moving-image background – the technology is unsurpassed. Some of the features that are of particular interest to game developers are: --     High-quality, primary 3D data. The unique ability of Canesta's 3D sensor to measure the physical distance to each "pixel" in the scene, rather than attempt to calculate it from edges, changes in contrast, or by triangulation with a second sensor and/or structured-light projection (collectively "derived" 3D data), is the reason that the camera is not fooled by the environment -- a fact-of-life that limits other 3D camera attempts to carefully controlled use cases. Instead, features in the scene will yield the same depth reading whether light, dark, textured, or identical to the background.
   
  The result is that the camera, as mentioned, is highly robust in bright, dark, and changing lighting environments. It -- and the game or application -- will work in any typical environment, such as a living room, and is not fooled by challenging conditions such as halogen lamps, shiny objects in the backgrounds (such as doorknobs), or sunlight streaming through an open window. Even the extreme case -- a gamer dressed entirely in "hospital" white, standing in front of a featureless white wall -- can be reliably and accurately discriminated -- even in light so low that human eyes would lose the subject.
   
--   High frame rate, low latency, and no drift. The availability of comparatively fine-grained, primary 3D data at up to 60 frames per second means that the application -- especially when assisted by Softkinetic's middleware -- can feature fast and repeatable detection and tracking. The high frame rate and low latency come from the fact that the sensor (or supporting software) is not spending time on notoriously-complex image-processing computations -- necessary for derived 3D solutions; zero drift is a feature of both the inherent stability of the CMOS-based sensor, and the lack of a second "triangulating" sensor or projection that always must be in strict calibration with the first.
   
--   Easy to manufacture in high volume. This is in particular contrast to triangulation-based sensors that must be held to comparatively strict manufacturing tolerances, and packaged in materials that attempt to minimize thermal and mechanical drift.
   
--   Inexpensive, standard CMOS 3D camera technology. The use of mainstream, state-of-the-art CMOS fabrication techniques for the 3D imaging sensor conveys permanent, consumer-oriented cost advantages and technical benefits over more exotic or boutique processes.
   
--   Easy to use Applications Program Interface (API.) Developers have access to the iisu Software Development Kit (SDK) that includes technical consulting, 3D graphics and an application development service. Developers can use C, C++ or C# in synchronous or asynchronous modes.

Some of the most interesting applications for the technology are game control, where hand movements can move or select objects; body kinetic uses, where avatars move on the screen in response to the players movements; physical training, where the players movements are analyzed in real time to provide a basis for computer-assisted instruction; “virtual chroma key” (or “depth key”), where a user is pulled from his or her living room and inserted in a background, either for games or for video chat; device control, where hand gestures are translated into common remote control and button-push actions; and hand-gesture-based computer operating system interfaces, where virtual or actual desktops are conducted and manipulated like a symphony orchestra.

“The availability of fast and accurate depth information based upon robust technology that has been heavily researched and broadly developed over 8 years is a real plus for the gaming industry,” said Jim Spare, president and CEO of Canesta. “If you couple that with the development simplifications that result from Softkinetic’s middleware, you attain what is a comfortable, evolutionary approach for game and application developers that results, in fact, in a revolutionary experience for end users. When we pioneered Electronic Perception Technology in 1999, gaming was one of the key drivers, and we are delighted that its time has come.”

About Softkinetic

Softkinetic is the leading provider of natural interfaces that transform the way people interact with the digital world. Softkinetic enables a fully immersive, transparent and intuitive user experience by providing a 3D real-time gesture recognition middleware to Interactive Digital Entertainment, Consumer Electronics, and Serious Games companies. Softkinetic, iisu, and Interface Is You are trademarks or registered trademarks of Softkinetic S.A. For more information on Softkinetic please visit www.softkinetic.net.

About Canesta

Canesta is the inventor of revolutionary, low-cost electronic perception technology that enables ordinary electronic devices from video games to computers, industrial and medical electronics devices, automobiles or other vehicles, and even factory automation equipment, to perceive and react to objects or individuals in real time. When given true, 3-dimensional depth perception with Canesta's unique CanestaVision™ electronic perception chips and software, consumer, automotive, industrial, military, and medical products can gain functionality and ease of use not possible in an era when such devices were blind.

Canesta believes future applications of electronic perception technology are virtually as broad as the imagination. They may include immersive and gesture-controlled computer and gaming applications, automotive safety improvements such as intelligent airbag systems that can sense the size and position of an occupant to control deployment and avoid injury, low-false-alarm security systems that can detect the difference between an intruder and normal activity, such as a pet moving or child visiting the bathroom at night, or robotic tools that can successfully operate in a dynamic, rather than static environment.

Canesta was founded in April 1999, and is located in San Jose, CA. The company has filed in excess of forty patents, 26 of which have been granted so far. Investment to date exceeds $58 million, from Carlyle Venture Partners, Honda Motor Company, Hotung Capital Management, JP Morgan Partners, Korea Global IT Fund (KGIF), Venrock Associates and others