This project adds to non-invasive BCIs for communication for adults with severe speech and physical impairments due to neurodegenerative diseases. Researchers will optimize \& adapt BCI signal acquisition, signal processing, natural language processing, \& clinical implementation. BCI-FIT relies on active inference and transfer learning to customize a completely adaptive intent estimation classifier to each user's multi-modality signals simultaneously. 3 specific aims are: 1. develop \& evaluate methods for on-line \& robust adaptation of multi-modal signal models to infer user intent; 2. develop \& evaluate methods for efficient user intent inference through active querying, and 3. integrate partner \& environment-supported language interaction \& letter/word supplementation as input modality. The same 4 dependent variables are measured in each SA: typing speed, typing accuracy, information transfer rate (ITR), \& user experience (UX) feedback. Four alternating-treatments single case experimental research designs will test hypotheses about optimizing user performance and technology performance for each aim.Tasks include copy-spelling with BCI-FIT to explore the effects of multi-modal access method configurations (SA1.3a), adaptive signal modeling (SA1.3b), \& active querying (SA2.2), and story retell to examine the effects of language model enhancements. Five people with SSPI will be recruited for each study. Control participants will be recruited for experiments in SA2.2 and SA3.4. Study hypotheses are: (SA1.3a) A customized BCI-FIT configuration based on multi-modal input will improve typing accuracy on a copy-spelling task compared to the standard P300 matrix speller. (SA1.3b) Adaptive signal modeling will allow people with SSPI to typing accurately during a copy-spelling task with BCI-FIT without training a new model before each use. (SA2.2) Either of two methods of adaptive querying will improve BCI-FIT typing accuracy for users with mediocre AUC scores. (SA3.4) Language model enhancements, including a combination of partner and environmental input and word completion during typing, will improve typing performance with BCI-FIT, as measured by ITR during a story-retell task. Optimized recommendations for a multi-modal BCI for each end user will be established, based on an innovative combination of clinical expertise, user feedback, customized multi-modal sensor fusion, and reinforcement learning.
For each specific aim, the development of new assistive technology BCI access methods will be evaluated in one or more experiments using alternating-treatments single-case research design (SCRD) with healthy controls and/or participants with SSPI. SCRD is ideal for examining small, heterogenous populations such as individuals with SSPI. It allows for detailed examination of performance trends and changes over time, and for participant-specific modifications to the intervention as part of an iterative design process. Because each participant serves as their own control, a sample size of five is sufficient to demonstrate and replicate an initial effect. Please see the Statistical Design and Power section for additional information about SCRD and data visualization and analysis. A total of 60 participants will evaluate the BCI advancements; 15 individuals with SSPI and 45 controls. Participants with SSPI who currently have a reliable means of communication, either using speech and/or an AAC device, will be enrolled. All participants will be within the ages of 18-89 years (NIH-defined adults), with an equal number of men and women. Healthy controls will be matched for age, gender, and education level. In SCRD studies, each participant serves as their own control, so participants will experience all of the baseline and intervention conditions included in each individual study, as described below. Condition order will be randomized in the alternating-treatments, controlled such that each participant completes an equal number of sessions with each intervention, with no more than two consecutive sessions with the same intervention. Blinding is not possible as each subject must know their condition in an alternating treatment design. All study visits with people with SSPI will be conducted in participants' homes by OHSU staff. Study visits with healthy controls will take place at the OHSU BCI laboratory. For all typing tasks, participants are seated approximately 75cm from an LCD display, set up for the BCI-FIT system. Depending on the user's customized BCI-FIT configuration (procedures described in SA1.1), one or more of the following control signals will be used in each typing session: EEG (ERP, Code or SSVEP), eye movements (gaze position or velocity), or binary switches. The experiments for SA1.3a, SA1.3b, and SA2.2 all involve copy-spelling tasks, in which participants will copy five common 5-letter English words of approximately equal typing difficulty (according to LM input), and correct mistakes by choosing the backspace character when appropriate. Individual signal models will be initialized to population models and will be personalized and refined with each acquired copy-spelling task data set. The experiment for SA3.4 involves a story-retell task, described below in the paragraph about that experiment. Experiment 1.3a will test the hypothesis that a customized BCI-FIT configuration based on multi-modal input will improve typing accuracy on a copy-spelling task compared to a standard P300 matrix speller. We will pilot test new multi-modal input features with control participants before every SCRD with participants who present with SSPI. It will include five participants with SSPI in an alternating-treatments SCRD and will concentrate on typing accuracy as the primary DV. An initial baseline phase will involve weekly copy-spelling sessions with each participant's existing access method. Three or more baseline sessions will be conducted until stable performance is observed, then the alternating-treatments phase will begin. Treatments consist of two different BCI-FIT configurations: 1) a multi-modal configuration chosen by a combination of the approaches described in SA1.1. (clinically-supported and performance data-supported) and 2) a standard P300 matrix speller. In weekly data-collection visits, participants will complete copy-spelling sessions with each BCI-FIT configuration, with counterbalanced session order. Participants complete at least five sessions with each configuration, more if needed to achieve stable performance. In Experiment 1.3b, it is hypothesized that adaptive individualized signal modeling will allow people with SSPI to type accurately during a copy-spelling task with BCI-FIT without training a new model for each use. This experiment will also include five participants with SSPI in an alternating-treatments SCRD with typing accuracy as the primary DV. In this study, no baseline is planned, as the comparison of interest is between versions of BCI-FIT with and without adaptive signal modeling. Initially, each participant will complete system optimization procedures described in SA1.1 and SA1.2 to identify their customized BCI-FIT configuration. During each visit, in the alternative treatments experiment, the participant will attempt three copy-spelling sessions with their customized BCI-FIT configuration, using three different model types: (1) a single calibration completed by the same user immediately before copy spelling; (2) multiple calibrations completed by the same user on previous days; (3) multiple calibrations completed by other users. Data will be graphed and analyzed separately (following procedures in the Statistical Design and Power section) to evaluate effects on performance with both system versions. The experiment in SA2.2 will test the hypothesis that either of two methods of adaptive querying will improve BCI-FIT typing accuracy for users with mediocre AUC scores. It will include five controls and five participants with SSPI, each with an AUC score in the range of 70-80%. (Based on pilot testing, adaptive querying is expected to provide the most benefit to users with this level of baseline performance.) The experiment will follow an alternating-treatments SCRD. In the baseline phase, participants will complete weekly copy-spelling sessions with BCI-FIT without adaptive querying. Each weekly visit will include two copy-spelling sessions with BCI-FIT either with or without adaptive querying techniques. Condition order will be counterbalanced such that conditions occur in random order (with no more than two instances of the same condition in a row) and participants will experience each condition an equal number of times (at least five times each, until stable performance is achieved). The experiment in SA3.4 will use an alternating-treatments SCRD experiment to test the hypothesis that language model enhancements, including a combination of partner and environmental input and word completion during typing, will improve typing performance with BCI-FIT, as measured by ITR during a story-retell task. This experiment will include five controls and five participants with SSPI, each paired with a partner to provide partner input (total enrollment of 10 dyads). In each weekly data-collection visit, participants will engage in two structured story-retell activities, one with and one without the enhanced language model features. Condition order will be counterbalanced such that conditions occur in random order (with no more than two instances of the same condition in a row) and participants will experience each condition an equal number of times (at least five times each, until stable performance is achieved). The story-retell activity will involve the participant watching a short video along with a communication partner, then using BCI-FIT to answer questions posed by a third person. The primary DV in this experiment will be ITR.
Study Type
INTERVENTIONAL
Allocation
RANDOMIZED
Purpose
BASIC_SCIENCE
Masking
NONE
Enrollment
55
Adding a personalized multi-modal access protocol to customize a BCI-FIT access method configuration for each individual end user, based on a combination of user characteristics, clinical expertise, user feedback, and system performance data in the software.
Adding a BCI-FIT adaptive signal modeling that employs transfer learning and on-line model adaptation techniques with noisy labels in the software of this brain-computer interface to eliminate the need for data collection exclusively for model calibration, as well as to address model drift issues associated with drowsiness, fatigue, and other human and environmental factors.
Adding BCI-FIT active querying techniques which are software-based optimal action control policies in the brain-computer interface developed with active and reinforcement learning techniques in order to perform efficient user intent inference to improve the entire speed-accuracy trade-off curve for alternative communication.
Adding vocabulary and location information (called partner and environmental input) to the language models in the brain-computer interface from a user's communication partner.
Oregon Health & Science University
Portland, Oregon, United States
Typing Accuracy
Correct character selections divided by the total character selections in a copy spelling task.
Time frame: 12 data collection sessions over 12 weeks (1 session/week) to assess change
Typing Speed
Correct character selections per minute in a copy spelling task.
Time frame: 12 data collection sessions over 12 weeks (1 session/week) to assess change
Information transfer rate
Time-averaged mutual information between intended and typed symbols from the alphabet, computed using probability distributions in accordance with a language model
Time frame: 12 data collection sessions over 12 weeks (1 session/week) to assess change
User experience
Responses to 10 items on the NASA TLX questionnaire about comfort, workload and satisfaction using the brain-computer interface system during all typing tasks
Time frame: 12 data collection sessions over 12 weeks (1 session/week) to assess change
This platform is for informational purposes only and does not constitute medical advice. Always consult a qualified healthcare professional.