Multimodal Tracking of Therapy Activity and Effort During Rehabilitation

Overview

This study looks at whether everyday rehabilitation therapy sessions can be recorded and measured in a more complete, objective way. Today, therapists usually document sessions by hand, and the notes can vary a lot from one therapist to another. This makes it hard to track exactly what was done during a session and how much effort it took. In this study, adults receiving neurorehabilitation (for example after a stroke, or with Parkinson's disease, cerebral palsy, spinal cord injury, or ALS) wear two small, non-invasive wrist sensors during their normal therapy. The sensors measure things like heart rate, breathing, skin temperature, and movement. A camera records body movement during the session, and the therapist can turn on an audio recorder that captures their spoken instructions. After each activity, participants rate how hard it felt. Nothing about the therapy itself changes. The sensors, camera, and audio recording are used only to collect information - they do not guide treatment or affect any clinical decisions. Each participant takes part in three routine therapy sessions. The goal is to find out whether this combination of sensor, video, and voice information can be collected reliably during normal care, and to compare how the recorded effort matches what therapists and patients report. The study does not provide direct medical benefit to participants, but it may help make future therapy documentation more consistent, objective, and less time-consuming for therapists.

Rehabilitation therapy documentation is predominantly manual and therapist-dependent, with variation in structure, timing, and level of detail across practitioners. Existing records support clinical communication but do not systematically capture the temporal structure or quantitative characteristics of therapy sessions, such as movement intensity, physiological effort, and activity type. This feasibility study evaluates whether a multimodal logging approach can capture this information reliably during routine neurorehabilitation without disrupting clinical workflows. The approach combines three synchronized data streams. Wrist-worn, non-invasive CE-marked sensors continuously record physiological and movement signals (including photoplethysmography-derived heart rate and respiration, skin temperature, electrodermal activity, and acceleration). A single, fixed-position camera records observable movement during the session, from which automated computer-vision methods derive motion representations such as joint coordinates and abstract skeletal or mesh-based models; these derived representations contain biomechanical coordinate data only and do not retain identifying visual features. A therapist-controlled audio recorder captures spoken instructions and task descriptions to provide contextual annotation of session content. Each participant completes three routine therapy sessions within an approximately two-week period at a single rehabilitation center. Each session follows an identical structure: a five-minute baseline resting measurement, the routine therapy session with continuous sensor, video, and voice recording, brief self-reported effort ratings after each activity, and a five-minute recovery resting measurement. Therapy is delivered according to standard clinical practice; the recording systems are used only for data collection and provide no real-time output, feedback, or therapeutic guidance to the therapist or participant. Because the study is exploratory and observational, no hypothesis testing is planned. Analyses are descriptive and focus on data completeness, signal quality, and the practical integration of the recording systems into routine care, together with descriptive associations between therapist-reported effort, patient-reported effort, and sensor- and video-derived measures of activity and physiological load. Recordings are captured electronically, coded under a study ID, and stored on secure access-restricted institutional infrastructure.