Developing an Online Therapeutic Intervention for Chronic Pain in Veterans

The research study for this career development plan involves the development, iterative user testing, and evaluation of an online ACT treatment program for Veterans with chronic pain. This study will determine whether a larger-scale, more definitive VACT-CP full scale efficacy trial is indicated. Phase 1: Development Phase (15 months): I will develop a detailed VACT-CP protocol for individual online treatment. Treatment techniques will be based on previously-developed ACT treatments for chronic pain, as well as additional ACT training I will obtain during the early phase of the Rehabilitation Research and Development (RR\&D) Career Development Award (CDA-2). During this phase, mental health and other clinical staff providing pain care at the Veterans Affairs (VA) Bedford Medical Center (n = 10) will provide feedback and input on the intervention content. Phase 2: Pilot Evaluation of Intervention Components in Field-Based Iterative Usability Testing (21 months): The investigators will recruit Veterans (n=15) with chronic pain for usability testing, to assess the delivery mode, and utilize an Integrated Technology Acceptance Model to iteratively assess feasibility and usability of the VACT-CP online treatment. This portion of the project will focus on refining the delivery mode, and revising chronic pain intervention content as necessary. With my mentors, I will review the adequacy, utility of specific procedures, and intervention foci to revise the online treatment accordingly. Phase 3: Feasibility Pilot Randomized Controlled Trial (30 months): The investigators will assess the feasibility and acceptability using a Stage 1b RCT pilot test comparing VACT-CP (n = 20) to a wait-list control group (n = 20), including ease of recruitment, retention in each condition, treatment receptivity, attrition and retention in each condition, sustained participation, and the assessment process. The investigators will also describe preliminary data on the impact of VACT-CP vs. the control condition on pain-related functioning related to social, occupational, and physical functioning, emotional functioning, mental health, and quality of life. PHASE 1: INTERVENTION DEVELOPMENT The goal of this phase will be to 1) adapt ACT manualized treatment of chronic pain in Veterans into an online platform using the ECA as the intervention coach, and 2) gather qualitative feedback from chronic pain providers and content experts (n = 10 - 12) on the VACT-CP system for intervention refinement and potential benefits, barriers, referral concerns, and overall thoughts on the VACT-CP system. Initial Development of the VACT-CP System Procedure. The development of the intervention and its revision will be an iterative process over Phases 1 (Intervention Development) and 2 (Feasibility and Usability Testing) (specifications for intervention revision for Phase 2 are below) and will include: 1) reviewing existing ACT CP treatment manuals with mentors; 2) developing a preliminary online version of the intervention, with mentor input, 3) revising the intervention based on mentor and consultant feedback. Creation of the initial VACT-CP platform will include collaboration with mentors , as well as the Northeastern Personal Health Informatics team at Northeastern University led by Dr. Timothy Bickmore. Using input from this team of experts, as well as my own experience in educational technology and online course creation, I will create a basic prototype of the intervention including the main VACT-CP content components, alignment of session goals and outcomes, and the creation of a corresponding patient workbook with information on the system. This process will also include close mentorship between Drs. Kelly and Heapy to assess current ACT for CP manualized treatment, and adaption of this content to the online system. VACT-CP Treatment Content and Structure. The following components will be emphasized in VACT-CP based on previous manualized ACT-chronic pain treatment by Vowles and Sorrell Life with Chronic Pain: An Acceptance-based Approach Therapist Guide and Patient Workbook and utilizing ECA technology to create a personalized, therapeutic experience. Creation of the intervention will entail modification of the ECA image, creation of the intervention scripts and programming for the online user-experience. Participants will receive eight online modules, provided as weekly sessions that will feature an ECA (virtual therapist) as a treatment guide. Adaption to Online Platform. Treatment content for VACT-CP will be integrated into the Northeastern University platform created and maintained by the relational agent research group managed by Dr. Timothy Bickmore, which has already programmed and run several VA-specific, Institutional Review Board (IRB) -approved interventions. Dr. Bickmore has confirmed that both the budget and timeline for this project are appropriate, and thus feasible. This website will be available to Veterans for free for this project, and Veterans will be able to re-access the any module where they left off, or restart a module if preferred after logging out using their de-identified username and password provided by research staff. For the purposes of pilot testing with providers and Veterans, the user interface will also first be mocked-up using PowerPoint (PPT) slides, and following feedback from providers will then be adapted to the current ECA online platform (Phases 1 and 2). This platform will include elements such as personalized conversations with the embodied conversation agent, starting with a mindfulness exercise at the beginning of each module, and moving through major session components. Each session will incorporate different interactive psychoeducation experiences for each session, goal setting and tracking, and finally end with weekly homework aligned with the goals of the next session. Planning for and addressing potential safety concerns online is a major concern, and consultation for the inclusion of safety procedures and information has been provided by Drs. Kelly, Heapy, and Greene. Consequently, in addition to the modules, information regarding the Veteran Crisis Hotline will be incorporated into the website with the addition of a button providing phone number information and the PI's number will be included within the website as well should a Veteran need to access clinical staff. The investigators will also assess with Veterans whether they were able to locate these resources early in the usability and pilot testing. Qualitative Provider Interviews Procedure. A sample of 10 current mental health providers (including social workers, physicians, nurses, and psychologists) who provide treatment for chronic pain at the Bedford VA will be interviewed for formative assessment of the potential benefits, concerns, and institutional Veterans Health Administration (VHA) dissemination issues for the VACT-CP intervention, using a semi-structured interview guide and a "think-aloud" strategy for intervention review. Sample. Providers will be recruited from presentations and emails to Primary Care/Behavioral Staff and the Mental Health Service Line at the Bedford VA, and scheduled for a time for a 45 minute to one hour interview during regular work hours, though potential participants will also have the option to interview over the phone. Participants will be consented for participating in the interview and having their participation audio-recorded. Interview Guide. This interview will include open-ended questions related to palatability of the intervention, feasibility of engagement with Veterans with chronic pain, interest or potential concerns with referring Veterans to such a program, and reactions to a short walk-through of the intervention. The purpose of these questions is to gather information from valuable stakeholders and potential referral-providers on VACT-CP system to address refinement of the program, possible benefits, barriers, referral concerns, and overall thoughts on the VACT-CP system Questions for palatability are based on the "organizational perspective" components of the Practical, Robust Implementation and Sustainability Model (PRISM). Interviews will be audio recorded and transcribed. Provider Interview Analyses Qualitative data analysis will use a thematic approach for the data related to the goal of ascertaining intervention palatability and feasibility and referral issues, and a narrative approach for data collected with the goal of intervention refinement. Two independent reviewers will code Section 2 of the interview transcriptions using open-ended coding. Each element will be summarized separately, and overall summary themes will be generated. Summary tables of key points will be generated, similar to previous formative assessments performed. These tables will ensure data saturation has been reached for both provider input. Two reviewers will analyze qualitative data using a modified grounded coding to identify themes using audio recordings after each interview. Intervention areas will be modified as feedback is given. Each set of interviews will take place with 3 - 4 participants, to allow for iterative adjustment of the intervention based on feedback. Number of changes needed for each intervention area will be tracked, and less intervention refinement will be anticipated by the end of the interview cycle. However, if intervention palatability is not achieved, further testing using other providers will be utilized. The generated information and themes will be utilized for development and refinement of the VACT-CP intervention, to learn from major stakeholders important areas for potential revision during intervention development. PHASE 2: FEASIBILITY AND USABILITY TESTING The goals of Phase 2 are to: 1) gain further feedback on the VACT-CP treatment by gathering information from Veterans via usability testing to; 2) assess Veteran interest, satisfaction, and usability of the online VACT-CP treatment; 3) iteratively address any technological concerns. This Phase will allow for feedback and preliminary support that the Phase 3 small-pilot RCT is merited. Given past research suggesting that similar interventions are usable and beneficial, the investigators anticipate that usability findings suggesting concerns or places for revision will not be such a strong barrier that the intervention will not be at all useful in this population. However, should Phase 2 data indicate that a small-pilot RCT is not merited, the usability information from this Phase will be utilized to inform further VACT-CP refinement until the usability bugs have been worked out and the system is ready for testing in a feasibility pilot RCT. This information will also be disseminated to inform future online chronic pain treatment development. Sample Information Sample Size: No single participant reviewer can identify all the usability problems in a system; typically 3-5 reviewers can identify 80% of the usability problems, with diminishing returns from additional reviewers per round of testing. The investigators will conduct 3 rounds of usability testing with 4-5 Veterans per round to assess and iteratively address usability concerns (n=12-15 total). Recruitment. Subjects will be 15 patients primarily recruited by referral from the Primary Care Behavioral Health at the Bedford VA Hospital. Eligible participants will have a current diagnosis of non-cancer chronic pain, defined as related to either Musculoskeletal pain or Joint Problems/Osteoarthritis or presence of chronic pain of at least mild to moderate severity. The investigators have decided to target joint and musculoskeletal pain as these have been shown to best respond to psychological, behavioral interventions, whereas neuropathic pain and headaches having less research supporting this. However, the investigators will not exclude for other diagnoses of pain, as this is typical and would not necessarily preclude Veterans from benefiting from the treatment. The investigators have verified that there are 2,605 Veterans who were seen over the past year (i.e., having had at least two appointments for pain) at the Bedford VA with chronic, non-cancer pain concerns, suggesting that the recruitment goal of 15 Veterans within 3 months to do a single-session usability test for Phase 2 is achievable. In addition, I will give presentations to primary care providers, social workers, and mental health clinicians at the Bedford VA to enhance recruitment. Using the VA medical record, Veterans meeting eligibility criteria for chronic pain seen within 12 months at the Bedford VA will also be sent letters inviting them to participate. Screening Procedures. Callers will be screened by phone. Eligible participants will be scheduled for an in-person assessment with the PI, who will confirm study eligibility and obtain informed consent. A research associate affiliated with the research study will then conduct clinician-rated assessments, and subjects will complete self-report measures. Feasibility and Usability Assessment Procedures Eligible Veterans will come to the Bedford VA to complete baseline demographic and health measures. Following baseline assessment, Veterans will be provided with an iPad that will walk them through different portions of the user interface and treatment set-up, during which time they will be asked questions and recorded as a usability interview. Specifically, different images for options of the virtual therapy guide, as well as a PowerPoint with information about the user interface will be shown. Next, Veterans will be provided with written instructions for use, and do a behavioral walk-through of how to use the online system. Patients will be given instructions on interacting with the system, and will then walk through a portion of the treatment using PowerPoint slides that will simulate the VACT-CP treatment and usability interface, and then complete the Usability Survey and System Usability Scale (SUS: see measures below). During this time, the Veterans will take part in a contextual interview wherein Veterans move through the system unguided and are observed and asked questions about their experience. After this, additional, open-ended questions will be asked regarding Veteran preferences. For purposes of feedback and iterative program development, the Veteran will only be provided with the part of the first VACT-CP session during this time. The participant will then be left in a secure, confidential room to walk through the VACT-CP system on their own, during which time their use of the system will be recorded. Approximately 4 - 5 Veterans will be recruited and participate in this usability testing at a time, with 3 different usability testing arms, in order to allow for time to iteratively address any functionality issues or concerns with the user interface. Following this walkthrough, participants will be provided with measures of demographics, usability and health information technology use, including a retrospectively worded version of the Usability Survey items (e.g., I enjoyed using the VACT-CP program) and a semi-structured qualitative exit interview. Participants will receive compensation for their in-person visit, which should take approximately 2 - 3 hours. Assessment Measures A broad range of reliable and valid measures will be used to provide an assessment of chronic pain difficulties, demographic information, and technology use, technology interest, and comfort. In addition, Veterans will be observed while interfacing with the VACT-CP system, recorded, and asked questions by the interviewer. This will consist of specific questions related to the different parts of the system the veteran interacts with (e.g., the ECA design, structure of the modules, feedback system) as well as more open-ended when not provided with specific cues (e.g., what are you trying to get to from this page, what could be better about how this page is set up, etc.) Questions will be open-ended in nature, and address the extent of any problems with accessing and using the VACT-CP treatment online, how they might perceive the usefulness or importance of VACT-CP focal areas of treatment, whether they would recommend this treatment to a fellow Veteran, and any problems or concerns they may have with using the VACT-CP program. Using this additional, open-ended interview protocol, the investigators will also inquire as whether they would be interested in using the VACT-CP treatment at home and the role of their virtual therapist in the patient's interest in the treatment, and about past strategies or tools the patient used to address their chronic pain issues and whether the VACT-CP treatment might seem more useful than prior options or strategies. Finally, participants will be asked to indicate what, if any, changes they would make to the treatment to assist in self-management of their chronic pain. This will also include assessing their interest in it complementary and alternative methods of pain management (stretching, yoga, progressive muscle relaxation, etc.) to assess if it would be useful and feasible to add these options to the platform later if desired. The interview will be video recorded and coded to look for similar themes, difficulties, revisions, and strengths of the system. Intervention Revisions During Phase 2, in which Veterans will interact with the VACT-CP online treatment, the investigators will: 1) gather feedback from Veterans regarding acceptability and usefulness of the treatment, 3) gather and analyze complementary data from SUS-8, 4) address any technological issues as they arise, and 5) examine preliminary data from this phase of the study. Modifications to the treatment will be based upon: 1) acceptability and feasibility of the online treatment, and 3) qualitative feedback obtained from patients, therapist, and mentors. Individual feedback sessions at the end of phase 2 will be held to gather information for refining the treatment manual and make it acceptable to patients. The investigators anticipate that the Veteran intervention reviewers might miss bugs or other user interface problems, and anticipate that some revisions to the online system and program may need to be made for Phase 3. For instance, one potential issue that may arise might be an inability for the Veteran to read the text on the tablet screen. After receiving this information during an initial feedback round, the researchers will amend the program with a potential solutions i.e. text size or text color being altered, with the revised version then being piloted in the next round of Veteran user testing. Some usability challenges will be addressed with introductory handouts/tutorial materials/orientation to the program at the beginning of the intervention session. In addition, large organizational issues will be discussed before moving on to platform refinement. Data Management Patient files will be kept in a locked cabinet and password protected files where data will be encoded with ID numbers. IDs will be unrelated to any identifying information. Video recordings will be stored on a secure, VA approved server. The majority of data analysis will consist of qualitative and observational methods, including coding qualitative interview data and coded video files. Data will be checked for inconsistencies, omissions and errors. As only 15 subjects will participate in Phase 2, only summary quantitative statistics will be computed to describe the sample and to determine outcome on the study measures. Data gathered through qualitative interviews and quantitative measures will be housed on a VA-approved cloud-based data storage server. Data Collection and Analysis Qualitative Combined Usability and Contextual Interview. The investigators will focus on usability and feasibility to address Aim 2. During 60-75-minute usability sessions held in private, secure rooms rooms within the Bedford VAMC, participants will assess the usability of the VACT-CP intervention using a "think-aloud protocol", verbalizing their thoughts as they navigate the website. After a period of exploration with the site, the research staff will use a pre-created protocol to ask Veterans to perform a series of tasks on the website (i.e., entering therapy goals, accessing the different modules, completing pre-session assessment, interacting with the virtual therapist). The usability testing space is already equipped with a computer, tablet, and a web camera. As a participant navigates the intervention website on a tablet or computer, their actions will be recorded through webcam electronically, and Dr. Reilly will observe the actions in real-time within the room, and a blinded observer/note taker. Drs. Waring and Quigley will consult on usability testing procedures. The video recordings and notes made by Dr. Reilly during user testing will be transcribed and analyzed using qualitative software available through the Bedford VAMC. A modified consensual qualitative research (CQR) approach will be utilized to code the transcribed videos, which Dr. Reilly has been trained in and utilized in past peer-reviewed publications. CQR can be particularly useful in allowing for a team of coders to identify themes arising in both structured and more open-ended interview moments. Dr. Reilly will develop an initial codebook for using in coding based on the key activities and questions in the user-testing protocol. Dr. Reilly and the research associate will then employ the codebook in identifying usability-themes within the videos, with an additional research staff identified to act as a code auditor. According to CQR, the following processes occur to allow for depth and breadth in describing the shared experiences of participants: (b) two coders utilize the initial codebook throughout the data analysis process to foster multiple perspectives; (c) consensus is researched between coders about the meaning of the data; (d) at least one auditor must check the work of the primary team of judges and minimize potential bias; and (e) domains and themes are reported in terms of the frequency of arising themes. Themes will then be categorized according to the following CQR groupings: general (include all or all but one of the cases), typical (include more than half of the cases up to the cutoff for general), and variant (at least two cases up to the cutoff for typical). Specifically, the themes generated by this qualitative data will be used to identify user performance measures, errors and difficulties, and tasks that do not meet acceptable criteria, and general trends and categories of users' behavior such as how well users can complete an assigned task on the VACT-CP system and where they are encountering problems. Using this method, the investigators will classify all general comments regarding usability as necessary to include in iterative development, while variant and typical will be discussed within the research group to decide upon potential ways to incorporate this feedback into intervention refinement. In addition, Veterans will complete a series of brief usability questionnaires following user testing and the qualitative interview. Usability Survey and SUS data will be input into REDCap, reported descriptively using statistical software, and used to describe Veteran's general perceptions and opinions of the technology's usability, both within the controlled testing setting and for future use within their home. This feedback will then be triangulated with qualitative data, and used to iteratively refine the VACT-CP program between the three waves of user testing. The investigators will complete this process with 4-5 Veterans per round; Dr. Reilly then summarize suggested changes, obtain input from the full mentoring team, then meet with Dr. Bickmore's staff to implement potential changes. Results will be published to inform other researchers about usability findings related to format feedback, user interface, interest, and perceptions of behavioral intervention content. PHASE 3: FEASIBILITY PILOT RCT The goals of the feasibility pilot RCT study phase are to conduct a Stage 1b RCT pilot trial to: 3a) explore relationships between Veteran traits, usability beliefs, and technology perceptions on intended and actual use of the VACT-CP system; 3b) describe differences in VACT-CP versus a wait-list control group on pre- and post-test measures of pain acceptance, pain level, pain-related functioning, and quality of life, treatment satisfaction with the VACT-CP system and ACT-related outcomes including pain acceptance, valued living and experiential avoidance; 3c) evaluative the relative feasibility and acceptability of the VACT-CP intervention procedure and waitlist control, including ease of recruitment, retention in each condition, treatment receptivity, attrition and retention in each condition, sustained participation, and the assessment process. A total of 40 Veterans will be enrolled to test the VACT-CP system within their home. Participant Selection and Recruitment Sample Size: Based on the previous recruitment of Veterans with chronic pain at the Bedford VAMC, the investigators estimate that approximately 65% of Veterans screened for participation by research staff, recruitment letters, flyer recruitment, and referrals from staff (N=60 enrolled participants) will meet the inclusion/exclusion criteria and be eligible for the pilot trial. Similar to Phase 2, the investigators will not exclude for other diagnoses of pain. Given this, the investigators will aim to recruit a total of 60 participants over fifteen months total to allow for the desired final 40 participants screened as eligible and randomized into the pilot RCT study, specifically aiming for recruiting 3-4 participants per month. Subjects for the proposed study will be recruited over the course of approximately 15 months and be screened according to their presence of chronic pain and other important. Subjects will be recruited from the Bedford VAMC, where 2,605 Veterans are currently (as of November, 2017) being seen at the Bedford VAMC for chronic, non-cancer pain concerns, suggesting that the investigators' final recruitment goal of 40 persons total enrolled (1.5% of the total Bedford population) is feasible. In addition, I will give presentations to primary care providers, social workers, and mental health clinicians at the Bedford VA to enhance recruitment. Using medical record data, Veterans meeting eligibility criteria for chronic pain seen within one month at the Bedford VA will also be sent letters inviting them to participate. Study Procedures and Assessment Treatment Procedures. Following recruitment, a research associate not otherwise involved in the study will screen participants prior to enrollment. Veterans will be randomized in a 1:1 ratio, with a research staff member utilizing and keeping the schedule. Veterans randomized to VACT-CP will receive 8 online-module based weekly sessions of treatment via personal computer or provided tablet with wireless accessibility at the Bedford VAMC. Veterans in the waitlist control will be provided with a list of common pain resources at the Bedford VAMC. The VACT-CP program will be housed on a secure web server through Northeastern University, with no Veteran private health information collected by the system. If during Phase 2 it becomes clear that more or less sessions are necessary (based on indicators previously discussed for usability and feasibility testing), the number of sessions for Phase 3 will be changed accordingly. Veterans in both conditions will complete the assessment battery (including all measures below, except for the treatment satisfaction and working alliance inventory) at baseline. In addition, the full battery will be taken at mid-point of the intervention phase (4 weeks) and at the end, immediately following the treatment (8 weeks). Veterans will be tracked in terms of their post-referral treatment engagement (e.g., enrolling in any of the pain-resource options on the handout). In addition, a research associate will call the Veteran at weeks 3 and 6 and conduct a brief safety assessment and qualitative interview with Veterans assess for potential suicidality, usability issues, and participant health and well-being.