Diagnostic Precision and Accuracy Using the Teed Scoring System Versus the O'Neill Grading System

Overview

This validation study investigates the diagnostic precision and accuracy when grading otic barotrauma using two alternative gradings systems including the TEED and the OGS grading systems. Background There are two grading systems used to grade otic barotrauma. They include the Teed Score (TEED) and the O'Neill Grading System (OGS). Teed has been utilized more extensively but among physicians has been found to have limitations. The OGS is an alternative grading system with less classification categories than what the Teed classification system utilizes. The OGS system is tied directly to treatment decision making, specific to the respective grading level. Methods This study will utilize a sample of clinical examiners who will independently grade 120 digital color photos of actual patient tympanic membranes with and without evidence of eustachian tube dysfunction and middle ear barotrauma including variable stages or grades of otic barotrauma. They will use the TEED chart as a reference in selecting the appropriate TEED score for each of the photos. Simultaneously they will also utilize the OGS chart as a reference in selecting the appropriate OGS grade for each of the photos. The grading will be conducted by physicians, nurses, and technicians. This will allow for comparisons at varying provider levels. The comparisons will be made using the kappa statistics across the disciplines. This will allow comparison between observed agreement, and expected agreement due to chance for all reviewers. Research Hypothesis Ho: There is no difference in diagnostic precision and accuracy when grading otic barotrauma between clinical examiners using either the TEED or OGS grading systems. Ha: There is a difference in diagnostic precision and accuracy when grading otic barotrauma between clinical examiners using either the TEED or OGS grading systems.

Background Eustachian tube dysfunction (ETD) and middle ear barotrauma (MEB) are the two most common complications of clinical hyperbaric oxygen treatment (HBO2T) though these disorders can present in other settings. There are currently two alternative systems used to grade otic barotrauma in patients: the Teed Score (TEED) and the O'Neill Grading System (OGS). The TEED is the most commonly used grading system originally described in 1944 and contains five grades based on a single physician examination (1). There have been modifications to the TEED over the years included six grades, hence the TEED as modified by Macfie and Edmonds is commonly known as the Modified TEED Scale (2, 3). The TEED is not specific for the evaluation and treatment of middle ear barotrauma in patients undergoing HBO2T, pilots or airline passengers, soldiers exposed to explosive ordinance, recreational scuba divers, individuals driving to altitude, swimming injuries due to free diving to depths without equalization, loud sound pressure injuries, but rather to submariners practicing submarine escape training (1). Otoscopy is commonly performed on all patients who present with these complaints and to hyperbaric oxygen patients prior to treatment. Though it is often performed on patients having difficulty equalizing middle ear pressure during the compression and/or decompression phase of the hyperbaric treatments. Although the results from these otoscopic examinations are used to determine the diagnosis and proper course of treatment for the ETD or MEB, the TEED may not correlate with the consistency of diagnosis, the clinical approach to relieving symptoms, or the definitive course of treatment for the otic barotrauma (2,3). The OGS is a simplified classification system of ETD and MEB limiting the evaluation to three grade levels as opposed to six levels using the modified TEED classification system(4). Literature Review A background search using PubMed without any restrictions on study type, date, language, country of origin, was carried out and no validation studies of either the TEED or the OGS grading system could be found. No studies comparing either of the two systems for consistency or uniformity of diagnoses were discovered or available for retrieval. Specific Aims The goal of this study is to determine if using either the OGS or TEED will provide consistent and uniform results in grading barotrauma. The study will determine if the OGS and TEED can be used by all clinical examiners including those seeing patients for suspected barotrauma in the community setting including medical professionals who routinely see patients undergoing clinical hyperbaric oxygen treatments (hyperbaric physicians, nurses, and technologists), in order to determine if (1) both systems yield consistent results regardless of the system used for grading barotrauma and (2) whether there will there be consistent results among the various professionals (physicians vs nurses vs. CHT's) performing the otoscopic evaluation. Study Question and Research Hypothesis Is there a difference in precision and accuracy when scoring and grading ETD and MEB in patients between clinical staff members based on the type of scoring/grading system used? Ho: There is no difference in precision or accuracy when grading ETD and MEB between examiners using either the TEED or the OGS grading systems. Ha: There is a difference in precision and accuracy when grading ETD and MEB between examiners using either the TEED or the OGS grading system. Ho: There is no difference in diagnostic precision and accuracy when grading ETD and MEB based on training level of examiners using either the TEED or the OGS grading system. Ha: There is no difference in diagnostic precision and accuracy when grading ETD and MEB based on training level of examiners using either the TEED or the OGS grading system. Methods Inclusion criteria All patients photographed between March of 2013 and June of 2019 that developed symptoms undergoing hyperbaric oxygen therapy were selected and deidentified. This involved retrieving all documented photographic images of ETD and MEB to score/grade the degree of otic barotrauma. These patients all had baseline photographs taken of the external ear canal and tympanic membrane as part of the pre-treatment standardized protocol in our department. Baseline photographs (normal) are included in the retrieved images. The photographs were initially screened and any images that were blurry to the degree that rendered scoring/grading impossible were excluded. Images considered interpretable despite blurring or partial obstruction due to cerumen or hair were included. This was done to maximize the use of photos that reflect the varying quality of images encountered in everyday clinical treatment conditions using otoscopy. The candidate physicians, hyperbaric nurses, nurse practitioners, and physician assistants will not be limited to any specialty, however at the technician level the study was limited to those working in the hyperbaric medicine setting. Sample size The participants will review and independently grade a total of 120 otoscopic photos of tympanic membranes taken between March 2013 through June 2019. The photos include normal and abnormal images representing tympanic membranes with all levels of Eustachian tube dysfunction or middle ear barotrauma following hyperbaric pressurization that required a hold due to patient subjective ear symptoms. The photo images were taken from actual hyperbaric patients seen at our facility with the exception of 10 photos. The 10 photographs included from outside sources were necessary to ensure inclusion of all stages of otic barotrauma including severe stages. All participants in the study will be provided with instructions regarding the study both verbally and in writing. The study participants grading and scoring the ETD/MEB photos will have various levels of professional clinical training and will include physicians, nurses, and technologists. Participants will use the modified TEED placard commonly used in hyperbaric centers as a reference in selecting the appropriate TEED score for each of the photos. Simultaneously, they will utilize the OGS placard as a reference in selecting the appropriate OGS grade for each of the photos. The TEED will be used without the benefit of a baseline photo, which was not required when this system was developed in 1944 and subsequently modified. The OGS system will be used initially without the use of a baseline photo, this is similar to the intended use of the TEED. The OGS system will subsequently be graded separately with the benefit of a comparison baseline photo in accordance with the intended use in hyperbaric treatment centers where baseline photos can be readily obtained. Baseline photos could potentially be obtained in other settings beyond hyperbaric clinical settings e.g. pilots, navy divers, etc. Data Analysis There will be three levels of comparison of precision and accuracy in using these two scales. The first comparison will evaluate the consistency displayed using each grading system (TEED and OGS) independently without the knowledge of the baseline status of the tympanic membrane prior to trauma. The second level of comparison will evaluate both grading systems for precision and accuracy as compared to each other. The third level of comparison will have the reviewers compare a baseline photo of the patient's tympanic membrane taken prior to hyperbaric treatment and pressurization to the post treatment traumatic photo and provide an OGS grade as the OGS was intended to be used in hyperbaric patients or in other settings where an available baseline photo may be available. The analysis will include the use of a kappa statistic for comparison between observed agreement and expected agreement for each of the scoring/grading systems comparing clinical examiners reviewing the images (5). The Fleiss kappa statistic will be used for comparison since 3 or more clinical examiners observers will score and grade the photo images (6). A comparison of the kappa statistics with respect to scoring/grading system used, and level of training will be analyzed. Ethical Considerations The study uses de-identified archived photos of external ear canals and tympanic membranes. A protocol was submitted to the IRB and approved prior to the conduct of this study. Funding Sources None