TrackCap:
TrackCap overview: Our system enables using smartphones for 3D interactions in applications on head mounted displays (HMD). It consists of a planar tracking target attached to the HMD, a tracking software on the smartphone that estimates the pose of the phone relative to the HMD, and a network interface that transmits the pose to the HMD. This system enables interactions of the phone with virtual objects that are placed in world coordinates. In this example, we interact with virtual objects by attaching a smartphone to a plastic toy gun. While previous approaches require holding the device in the field of view of the HMD’s camera (black), our system supports a large interaction space by using the camera on the phone for tracking (blue).
Abstract: The latest generation of consumer market Head-mounted displays (HMD) now include self-contained inside-out tracking of head motions, which makes them suitable for mobile applications. However, 3D tracking of input devices is either not included at all or requires to keep the device in sight, so that it can be observed from a sensor mounted on the HMD. Both approaches make natural interactions cumbersome in mobile applications. TrackCap, a novel approach for 3D tracking of input devices, turns a conventional smartphone into a precise 6DOF input device for an HMD user. The device can be conveniently operated both inside and outside the HMD's field of view, while it provides additional 2D input and output capabilities.
Acknowledgements: This work was enabled by the Competence Center VRVis, the FFG (grant 859208 - Matahari) and the EU FP7 project MAGELLAN (ICT-FP7-611526). VRVis is funded by BMVIT, BMWFW, Styria, SFG and Vienna Business Agency in the scope of COMET - Competence Centers for Excellent Technologies (854174) which is managed by FFG. Furthermore, Tobias is partially supported by Callaghan Innovation, host of the Science for Technological Innovation National Science Challenge, Seed Project 52421, and by the Marsden Fund Council from Government funding, administered by the Royal Society of NZ.
Video:
