Above: Google trains a VR headset to do 6DOF position sensing with AI assistance. Though it’s currently available in tethered VR headsets using position-tracking sensors for controllers, six degrees of freedom (6DOF) controller tracking is a holy grail of sorts for standalone VR headsets.
As discovered by Adobe’s Dimitri Diakopoulos and noticed by RoadtoVR, Google has come up with an inexpensive AI-assisted way to add 6DOF controller tracking capabilities to a standalone headset — full details of which are included in a recent Google research paper. The trick begins by using the same two fisheye cameras already found in the standalone VR headset to additionally track the positions of the user’s hands and arms. On the software side, it relies on a neural network trained with hand, arm, and controller positions to estimate the real-world controller’s unknown location within the user’s known grasp.
A dataset of 547,000 stereo image pairs was used to train the network, compiled from 20 users doing 13 movements in varied lighting. “Our key observation is that users’ hands and arms provide excellent context for where the controller is in the image,” wrote the Google researchers, “and are robust cues even when the controller itself might be occluded.
To simplify the system, we use the same cameras for headset 6-DoF pose tracking on mobile HMDs as our input. In our experiments, they are a pair of stereo monochrome fisheye cameras.
We do not require additional markers or hardware beyond a standard IMU based controller.” At the moment, Google is able to determine the controller’s position within a mean 33.5 millimeter error rate — only 1.32 inches — when it knows hand and arm positions. Presently, the system runs at 30FPS on a single mobile CPU core and doesn’t require any object to be glowing to track its location properly. Read more from venturebeat.com…
thumbnail courtesy of venturebeat.com