Reducing Loneliness of Older Adults in Long Term Care Facilities Through Collaborative Augmented Realities

Overview

The goal of this clinical trial is to test the use of head-mounted display (HMD) augmented reality (AR) in older adults residing in communities or assisted care facilities. The main questions it aims to answer are: what is the feasibility, acceptability, and satisfaction of HMD AR vs two-dimensional audio-video communication? Older adults and their designated family member will complete HMD AR activities in 8 session visits over 4 weeks. Researchers will compare HMD AR to 2-D audio-visual (AV) comparison group on feasibility, acceptability, and satisfaction with the interactive communication technology.

For both interventions, the bandwidth within the participants' homes will be measured and a hotspot created if necessary to facilitate data streaming. For each session, a trained research assistant (RA) will be physically present at the participants' homes to troubleshoot for technological issues. The RA will observe the participant establish the Interactive Communication technology (ICT) connection, assisting as needed. The RA will remain within the room; the RA will monitor for technological issues and participants' reactions. HMD-AR Intervention Experimental Setup. Equipment includes a Kinect camera (Microsoft Corporation), a microphone, an HMD, and a laptop on which the data sharing and the activity ran and remotely displayed on the participant's HMD. The RA ensured that the participants were connected before moving to an unobtrusive viewing area. Upon completion of the session, the RA dismantled all equipment and cleaned the HMD with disinfectant wipes. AR Activities. Two collaborative AR activities were available tot he participants: fireplace decoration and checkers. If the fireplace activity was selected, participants could choose from a variety of virtual objects to decorate the fireplace mantel, hearth, or nearby wall. For the checkers activity, each participant sat at a table, and the checkerboard and checkers appeared as a virtual object in space. Participants were able to see each other's photorealistic avatars during both activities. Upon completion of each session, the participants would let the RAs know who turned off the AR display and stopped the data communication. Before the start of the experimental sessions, RAs provided one-on-one training and orientation to the HMD-AR to ensure that the participants were able to navigate the system. In addition, a hard-copy, 1-page quick reference guide was provided to each participant. Each older adult-companion pair participated in 2 sessions per week for 4 weeks (total of 8 sessions). Creation of photorealistic avatars. A mobile app, Polycam, was used to take multiple photos of the participant's face using a 360-degree view. Polycam constructed a 3D model from these photos that was then used to create a MetaHuman on Unreal Engine, matching the facial geometry and the facial features of the participant. However, MetaHuman Creator is limited in its ability to match the features exactly; therefore, the RAs added manual touch-ups to ensure the avatar looked similar to the participant. After creating the first version of the avatar, a video call was arranged with the participants to share their avatar with them and receive feedback. Participants were able to pick their avatar's clothing, hair, body shape, and accessories, and apply makeup. If a participant was not satisfied with the likeness of their avatar, the process was repeated, and more manual touch-ups were added. A second meeting was arranged with the participants to share the new version. 2D Audio-visual Intervention The 2D audio-visual sessions will take place in a dedicated room at the assisted living facility and in an area of the participants' choice for those living at home. RA will initiate the connection between the older adult and family member. In the unlikely event a family member does not have a device for the 2D audio-visual visits, a computer tablet will be supplied for use during the study. Upon completion of the session, the RAs recovered any loaned devices.