Eye Examinations as a Gold Standard to Evaluating Survey Instruments

Overview

Objective measures of health are relatively expensive; meanwhile, self-reports are critiqued to be subjective and not comparable. A middle ground approach is to apply the frontier work of vignette methods. The aim of the survey experiment is to explore the validity of survey instruments to identify students who need glasses, using objectively measured visual acuity as a gold standard. It is hypothesized that direct comparison against vignette (DCV) is significantly different from indirect comparison against vignette (ICV); meanwhile, self-assessment of vision (SAV) is significantly different from primed self-assessment of vision (PSAV). It is also hypothesized that DCV is a more valid survey instrument than ICV to classify students who need glasses, using objective visual acuity as a gold standard. Lastly, it is hypothesized that priming effect from vignettes improves the validity of self-assessment. It is planned to enroll 3,755 subjects in the survey experiment.

The schools will be visited to announce this project on survey experiment and measuring visual acuity, for which all participants will receive free eye examinations. After receiving consent/assent, two data collection strategies will be applied in this survey - DCV and ICV, to identify individual perceptions of vision. In the arm of ICV, the self-assessment of vision (SAV) is listed before the vignette questions; however, in the arm of DCV, the self-assessed vision (PSAV) is listed after the vignette questions to estimate priming effect. With randomization prior to administration of survey, 50% of the students will be enrolled in "direct comparison against vignettes" (DCV) and another 50% for "implied comparison against vignettes" (ICV). Each participant will self-administer the survey. The objective measure of vision will be set up in a vacant classroom with assistance from members of the Health Center Jingning County. DCV, ICV, SAV, and PSAV are estimated from randomized survey questionnaires. Objectively measured vision is from simplified Snellen chart. Measures - All subjects will be asked to respond to the questions about vision acuity without glasses or contact lenses for themselves and these hypothetical scenarios. Information collected through the survey includes self-assessed vision, vignettes, clusters (school, grade, and class), whether the subjects wear glasses or contact lenses, and two demographic variables (age and sex). Self-assessed vision - Data of self-assessed vision is collected by asking students to think about their own vision without glasses or contact lenses. Specifically, each student was asked to respond to this question: "at the present time, would you say your distance eyesight is excellent, good, fair, poor, or very poor?" Vignettes - The DCV questionnaire asks subjects to jointly evaluate their own vision and vignette situations; meanwhile, the ICV questionnaire asks subjects to conduct single evaluation of vignette questions. Even though subjects in ICV questionnaires were not asked to compare their visual acuity with the hypothetical person Zhang, the experimenter could conduct the comparison through data analysis. Accordingly this approach is named the "indirect comparison against vignettes" (ICV). For example, in the DCV questionnaire, the subjects will be asked to respond to the following question: Sitting in the last row in the classroom, \[Xiao Zhang\] does recognize teacher's faces clearly but cannot identify small hand writing on the blackboard clearly. Would you say your distance eyesight is: 1. Better than Zhang's 2. The same as Zhang's 3. Worse than Zhang's Comparatively, in the ICV questionnaire, the subjects will be asked to respond to the following question instead. Sitting in the last row in the classroom, \[Xiao Zhang\] does recognize teacher's faces clearly but cannot identify small hand writing on the blackboard clearly. Would you say \[Zhang\]'s distance eyesight is: 1. Excellent, 2. Good, 3. Fair, 4. Poor, or 5. Very poor. Two vignettes are designed in this study. More specifically, two hypothetical persons with common Chinese names, Xiao Wang and Xiao Zhang will be introduced in the survey, who would be mentioned as V1 and V2 in the following discussion. V1: In the dining hall, \[Xiao Wang\] has no difficulty to see students on the same table clearly but he cannot not see students on next tables clearly. V2: Sitting in the last row in the classroom, \[Xiao Zhang\] does recognize teacher's faces clearly but cannot identify small hand writing on the blackboard clearly. The two vignettes is designed in the ascending level of visual acuity and the vignettes is arranged in the questionnaires in this way: V1 and V2. The question in DCV is, "Would you say your distance eyesight is better than Wang's, the same as Wang's, or worse than Wang's?" and the question in ICV was, "Would you say Wang's distance eyesight is excellent, good, fair, poor, or very poor?" respectively. Objective visual acuity - The simplified Snellen chart only consists of letter "E" with different directions, which was approved by the National Bureau of Standards in China to be a national standard in testing visual acuity (GB115331989). It was approved by the Ministry of Health to be used nationwide in China on March 27th, 1989. The simplified Snellen chart is used to estimate visual acuity with the respondent standing at 5 meters. The measurement ranges from 4.0 to 5.3, an arithmetic sequence with 0.1 progression. The larger the number is, the better the respondent's vision is. Capability to respond to this eye chart is most likely to be independent from educational level. Furthermore, simplified Snellen chart also rules out of the difficulties of Snellen letters and common misidentifications(Mathew, Shah et al. 2011). Therefore, it is less likely than a standard Snellen chart to underestimate true vision due to low literacy rate. In the line with vision 4.0, only one letter "E" is presented, and in the line with vision 4.1, there are two "E"s. Except those two lines, the probability to overestimate true vision by 0.1 due to chance alone is smaller than 6.25% (i.e., 1/64), since a correct response to at least three "E"s in each line is required in this survey experiment. More specifically, the probability to guess the direction of one letter "E" correctly by chance is 1/4 and the probability to guess the directions of three "E"s correctly by chance is only 1/64. Using eye examination as a gold standard, the empirical research addressed the validity of vignette methods.