Sony Corp announced the current state of the development of SmartAR, which is software for realizing augmented reality (AR), at a press conference in Tokyo Oct 29, 2012.
Sony has been improving the software to display AR images on mobile devices without lags since the previous announcement that the company made May 19, 2011 (See related article).
SmartAR is software that can extract the feature points of an image taken by a camera, match them with registered images and recognize the structure of a space while estimating the movement of a camera. It does not require "markers," which work as triggers that are used to select information or video to be superimposed. Also, it can display an AR image as if a virtual object is placed on a real object.
As for the recognition of a space or an image and the process of displaying information to be superimposed, a frame rate of only about 15 fps had been achieved with a personal computer at the time of the announcement in 2011. However, as a result of the improvement, Sony realized a frame rate higher than 60fps with a mobile device.
It is now possible to display AR images that accurately follow input images, and users feel that the AR images are more natural, said Masaki Fukuchi, researcher at the comapny's Intelligent Systems Research Laboratory. Also, the improved software uses information obtained with an acceleration sensor and gyro sensor to accurately estimate the orientation and location of a mobile device.
After the improvement of the software, it was employed for the main function of Sony Computer Entertainment Inc's "PlayStation Vita" portable game console, and several games using SmartAR were released. Moreover, the software is now used to promote the sales of the Sony Group's movie and music software. And Okinawa Churaumi Aquarium plans to start distribution of Churaumi AR Camera, an AR application for smartphones, Nov 1, 2012.
As functions related to SmartAR, Sony added (1) the "LiveAction AR," which displays a 3D photographed image such as human image so that it seems to exist regardless of the angle from which it is viewed, and (2) a large-scale image recognition technology that transmits image information extracted from an image taken by a camera to a server and matches it with registered images in the server.
The first function has already been employed for "Nogizaka 46 AR," an application for promoting CDs of "Nogizaka 46," a Japanese idol group. And the second function will be used for a campaign that Culture Convenience Club Co Ltd will start to promote the rental and sales of DVDs and Blu-ray discs in November 2012.