Immersive Virtual Reality for Visuo-motor Integration Skill Assessment

Overview

A significant deficit affecting nearly half of children with hemiplegia is visual-motor integration, or eye-hand coordination. Children have difficulties integrating visual and motor information to effectively plan and execute movements. Visual-motor impairments are detrimental because they affect accuracy of reaching and grasping, which are movements involved in feeding, writing, and sports participation, among many other daily life activities. Although paper-and-pencil and touchscreen computer assessments exist, these fail to evaluate impairments under realistic, 3D conditions. This assessment barrier leads to significant gaps in knowledge the influence of these impairments on children's performance of functional activities. We will use immersive virtual reality (VR) delivered using a head-mounted display (HMD) to address this gap. Because it is fully visually immersive, VR makes interactions similar to real world performance. These features enable HMD-VR to offer more natural assessment conditions. HMD-VR may help us gain important new knowledge about functional movement deficits in children with hemiplegia. The purpose of this study is to evaluate low-cost HMD-VR as a realistic assessment tool for visual-motor integration deficits in children with hemiplegia. The long-term goals of our research program are to: 1) Inform clinical decision-making practices by providing families and clinicians with precise, accurate information about children's abilities; and 2) Generate new knowledge about visual-motor integration impairments to enhance the effectiveness of both virtual and conventional rehabilitation interventions. We will recruit 40 children with hemiplegia aged 7-16 years at GMFCS Levels I-III and Manual Ability Classification System levels I-II for testing sessions of seated paper-and-pencil, touchscreen computer and HMD-VR visual-motor integration tasks at 3 clinical sites We will measure feasibility using counts of enrollment, side-effects and protocol completion. Visual-motor integration is quantified in the paper-and-pencil task via standardized score and in touchscreen and HMD-VR tasks using equivalent temporal and spatial eye and hand metrics. This pilot study will generate descriptive estimates of differences in visual-motor performance under conditions of differing 3D realism. This work is the first step towards the ultimate goal of a valid assessment method informing new VR-based treatment options for children with hemiplegia.

In a private testing room at the testing site, children will complete descriptive functional sensory-motor tests with a registered physical therapist. They will then complete the paper-and-pencil visual-motor integration test (Beery-Buktenica Test of Visual Motor Integration) while seated comfortably. They will then undertake the visuomotor integration tasks using a touch screen computer, completing 5 trials of each of 3 target positions under visual-only, motor-only, and visual-motor integration conditions. The task will be displayed on a 20" HP Spectre touch-screen laptop with an RTX 960 graphics card. Eye-tracking will be undertaken using a Tobii Nano which integrates with Unity software. Kinematics of hand movement during reach to touch as well as head movements will be collected using an Orbbec Astra depth-sensing camera. Accuracy of touch is recorded by custom-written software tracking X-Y touch coordinates on the screen. All data collection modalities are synced and integrated using LabVIEW. Children will then complete the same tasks in the 3D HMD virtual environment. We will use the VIVE Pro EYE, the leading commercially-available immersive VR system, which has with a 110 degree trackable field of view and an embedded eye tracker. Arm movements are tracked by lightweight trackers attached via Velcro arm band to children's forearms and ManusVR motion tracking gloves worn on the hands. Head movements are tracked by position sensors in the HMD. Trackers and gloves enable upper extremity interaction with objects in the virtual world. An Alienware m15 gaming laptop with an NVIDA GeForce RTX 2060 graphics card will run the task. Following the visual-only, motor-only, and visual-motor integration tasks, children will complete a new visual-motor integration task involving virtual object transport, where they will grasp a virtual object and transport it to a new location in the virtual environment. Finally, they will complete the object transport task in a more audiovisually-complex virtual environment in the HMD.