Computer vision is vital to enabling controller-free virtual reality interaction. The adjacent field of visual servoing involving mechatronics techniques allows for an affordable tracking system to be developed for real-time 3D facial mesh reconstruction and target triangulation. With the accessibility of open-source facial detection algorithms, a 2DOF pitch-yaw camera system was designed. Using state-of-the-art approaches, a velocity-controlled system with basic predictive control for out-of-bounds and occlusion compensation yielded a platform for further development of facial mesh generation.
The system was capable of 50 deg /s angular velocities with 82% of total tracking time spent within a ±1.8 deg boundary, equivalent to the size of a face at a distance of two meters, given a highly interactive presentation use case. A more dynamic test simulating a boxing match VR game yielded a total tracking time of 71%, with room for further improvements to the mechanical design and Kalman-based filtering.