Investigating the Impact of Mode of Administration on Item Response

Overview

The Patient-Reported Outcomes Measurement Information System (PROMIS) is an NIH Roadmap initiative to develop a computerized system measuring patient-reported outcomes in respondents with a wide range of chronic diseases and demographic characteristics. In the first four years of its existence, the PROMIS network developed item banks for measuring patient-reported outcomes in the areas of pain, fatigue, emotional distress, physical function, and social functioning. During the item banking process, the PROMIS network conducted focus groups, individual cognitive interviews, and lexile (reading level) analyses to refine the meaning, clarity, and literacy demands of all items. The item banks were administered to over 20,000 respondents and calibrated using models based on item response theory (IRT). Using these IRT calibrations, computerized adaptive test (CAT) algorithms were developed and implemented. The network has designed a series of studies using clinical populations to evaluate the item attributes, examine their utility as CATs, and validate the item banks. More information on the PROMIS network can be found at www.nihpromis.org. This study is designed to examine how differences in modes of data capture affect psychometric properties and score differences and to evaluate the consistency of these results across three PROMIS health domains: emotional distress-depression, fatigue, and physical function. Four modes of administration will be compared: interactive voice response (IVR) technology, paper and pencil questionnaire, personal computer, and personal digital assistant (PDA). A total of 800 patients will be enrolled from three diagnostic groups: chronic obstructive pulmonary disease (COPD), depression, and rheumatoid arthritis. The study will test for equivalence across modes of administration, with the hypothesis that there are no mode effects; if mode effects are found, their magnitude across modes will be estimated. This network project will result in an improved understanding of the effect of assessment mode on patient-reported outcome (PRO) data. Guidance from this project can help in planning future PROMIS activities beyond the present PROMIS program.

This study is designed to systematically test the impact of mode of administration on patient-reported outcomes measures included in the PROMIS item banks. It is designed as a randomized cross-over study. Two non-overlapping alternate forms (Form A \[FA\] and Form B \[FB\]) with eight unique items each from three of the PROMIS domains (emotional distress-depression, fatigue, physical function) will be developed. Respondents will answer one of the forms by automated phone interview using interactive voice response (IVR) technology, paper and pencil questionnaire (PP), personal computer (PC), or personal digital assistant (PDA) technology. The other form will always be answered by PC. The order in which the forms are administered will be randomized. The two assessments will be separated by a short interval (e.g., 5 to 10 minutes), but will take place on the same day. The study is powered to evaluate equivalence within a score difference of +/-2.0 on a T-score metric (standard deviation of 10) with 85% power. Data for the IVR-PC, PP-PC, and PC-PC modes will be collected via Polimetrix (n=200 per arm, with random assignment to arm); data for the PDA-PC mode will be collected via Stony Brook (n=200). Respondents will have one or more of the chronic conditions studied in other Wave 2 studies (COPD, depression, or rheumatoid arthritis).