A Japanese research group prototyped a gaming system by combining a pico projector and image recognition technologies and exhibited it at CEDED 2010, which took place from Aug 31 to Sept 2, 2010.
The research group consists of researchers at a graduate school of the University of Tokyo and a graduate school of Keio University.
The gaming system uses a small device made by combining a projector and a camera. The user holds the device in one hand and plays a game. The image of a character is projected by the projector, and the user can control the character by moving the device.
For example, when the device is moved to the right and left, the character moves accordingly. When the device is jerked upward, the character jumps. And, when the device is moved upward at a slow speed, the character moves upward as if it is flying.
The character acts differently depending on a graphic or an object on which it is projected. For example, when a character is projected on a horizontal line and the device is moved in a horizontal direction, the character walks on the line. And, when the character jumps and there is a graphic or an object above the character, the character bangs its head against it.
Such operations are realized by analyzing images taken by the camera in real time to recognize (1) the user's motions, (2) the distance between the plane of projection and the device, (3) the character bumping into an object and (4) the colors and shapes of objects. Rapid movements and the direction of gravitational force, which cannot be recognized by the camera, are detected by an acceleration sensor.
Furthermore, the gaming system is capable of recognizing colors such as red and blue. When the character moves to the image of red fire, it catches fire. And it is possible to extinguish the fire by moving the character to the image of a blue water droplet.
Gaming system could be realized with handset
The gaming system uses a pico projector, a camera and an acceleration sensor that are commercially available. Because the arithmetic processing required for image recognition can be easily carried out by an application processor equipped in a mobile phone, the research group considers that the system can be applied to any handset equipped with a projector, an acceleration sensor and a camera.
However, to realize the gaming system with a handset, the direction to which an image is projected has to be the same as the direction that the camera is aimed at. In other words, the projector lens and the camera lens must be arranged in the same direction (in parallel).
There are some handsets equipped with a projector in the market. But their projector lenses and camera lenses are arranged perpendicular to each other, making it impossible to apply the gaming system. If a mirror is placed near a camera lens or a projector lens to change its optical path, it may become possible to realize the gaming system.