Language Translation of Knowledge Mobilization Resources

Overview

Knowledge mobilization (KM) resources are tools designed to facilitate the use of research evidence in healthcare decision-making. These resources are created in various formats - including plain language summaries, infographics, and videos - to meet the needs of diverse end-users, such as healthcare professionals, policymakers, patients, and caregivers. They are intended to be easily accessible; however, individuals whose first language is not English may have difficulty understanding them. Thus, translating KM resources into other languages is essential to support health equity and accessibility, but it is often costly and time intensive. This study aims to explore whether artificial intelligence (AI) tools, specifically ChatGPT - an AI-based large language model developed by OpenAI - can effectively translate KM resources for members of the public whose first language is not English. The resource being evaluated offers guidance on preventing post-COVID-19 condition and has already been translated by a professional (human) translator into seven languages commonly spoken in Canada: French, Spanish, Ukrainian, Tagalog, Arabic, Chinese, and Punjabi. Using ChatGPT, AI-generated translations will be created in those same seven languages. For this study, participants - adults living in Canada whose first language is one of the selected languages and able to read English - will be randomly assigned to review either an AI-generated or a professionally translated version of a KM resource. They will then complete a questionnaire evaluating their understanding of the resource, as well as the readability and acceptability of the translation. This study will contribute to the Investigators' understanding of the potential use of AI for translating health information. The goal is to support equitable access to health information and promote citizen-centered care by reducing language barriers using innovative solutions.

Background and Rationale: KM resources play a critical role in bridging the gap between research and practice in healthcare. Effective communication of this information is essential to ensure the uptake and implementation of research evidence and health recommendations by the public. Canada is home to a highly diverse population. Language proficiency remains a persistent challenge for many newcomers. Inadequate health communication stemming from language barriers can lead to poorer health outcomes, reduced satisfaction with care, and increased healthcare inequalities. High-quality translations of health materials are therefore essential to promoting equitable access to health information and helping reduce disparities in health outcomes. While professional translation services have long been the standard for producing linguistically accurate translated materials, they are often resource-intensive, limiting the scalability and timeliness of current translation efforts. Recent advances in AI have generated interest in leveraging AI-powered tools to support the translation of health materials. However, concerns remain regarding the ability of AI powered tools to navigate the cultural, contextual, and emotional nuances of language that are vital in healthcare communication. No controlled trials have yet compared AI and professional translations specifically for KM resources aimed at the public. Empirical assessments of AI-generated translations are therefore essential for understanding its capabilities and limitations. Objectives: The overarching research question is: Can AI effectively translate KM resources for the public? For this study, the primary objective is to compare KM resources translated using an AI tool versus those translated by professional human translators. Specifically, the Investigators hypothesize that translations of KM resources produced by ChatGPT will be comparable to professional human translations in terms of understanding, readability, and acceptability. Methods: Patient and Public Involvement: Members of the Pediatric Parent Advisory Group (P-PAG) will provide input throughout the study. They will give input on study design, assist with pilot testing data collection instruments, advise on recruitment strategies, and promote the study within their networks. They will assist with interpreting results and will critically contribute to developing and disseminating findings (e.g., plain language summary of results). Trial Design: This study is a randomized controlled trial. Participants will be randomly assigned (1:1 allocation ratio) to receive either an AI-translated version or a professionally translated version of a KM resource. They will be asked to answer a set of questions about their understanding of the content of the resource, its readability and acceptability. Trial Setting: The trial will be conducted entirely online. Participants will complete the study remotely using their own personal devices (e.g., computers or tablets) with internet access. All materials, including study link, instructions, and questionnaires, will be delivered electronically. Eligibility Criteria: * 18 years of age or older * Living in Canada * First language is one of the selected languages (i.e. French, Spanish, Ukrainian, Tagalog, Arabic, Chinese, and Punjabi) * Able to read and complete a questionnaire in English * Access to an electronic device (e.g. computer or tablet), Internet and email Intervention and Comparator: Participants will be randomized to view a KM resource translated, either by AI (i.e., ChatGPT) or a professional (human) translator, into one of the selected languages. The English-version of the KM resource is available online (https://canpcc.ca/app/uploads/2025/03/Online-Resource-Sheet-GT1-Prevention-of-PCC-Final.pdf). Outcomes: The following outcomes will be used to evaluate the translated versions of the KM resource from multiple dimensions: knowledge acquisition (understanding), ease of text processing (readability), and confidence in the information (acceptability). The primary outcome will be: Understanding: Measured by participant responses to a set of 7 multiple-choice questions specifically designed to evaluate key content knowledge about post COVID-19 condition and related recommendations contained in the resource. The number of correct answers will serve as an objective indicator of understanding, assessing the ability of each translation method to preserve critical concepts. Secondary outcomes will be Readability and Acceptability. Harms: This study involves little to no risk to participants. Nevertheless, potential harms may include mild fatigue or frustration if participants find the text confusing or difficult to interpret. To minimize this, all materials will be pilot-tested for clarity and presented in a user-friendly format. Participants may withdraw from the study at any time without penalty. Participant timeline: Participants in this study will be involved in a single-session conducted online. All data will be collected immediately after the participant has reviewed the assigned translation. Each participant will complete the study in approximately 10-15 minutes. No long-term follow-up is planned. Sample Size: Due to the pilot nature of this study, the Investigators aim to recruit 50 participants per language group, with 25 participants randomized to evaluate the AI-translated version and 25 assigned to evaluate the professionally translated version. With seven language groups, the total target sample size will be 350 participants. This sample size follows guidance for exploratory and pilot studies that require no formal sample size calculation. With 50 participants per language group, the Investigators will be able to provide estimates of potential differences while allowing us to explore variations by language. Recruitment: Recruitment will begin once the study has received ethics approval. Potential participants will be recruited through a variety of online strategies to ensure broad and diverse enrollment across language groups. Recruitment efforts will include targeted electronic newsletters distributed via University of Alberta's undergraduate and graduate student digests, as well as outreach to international student groups. Additionally, a recruitment email will be sent to pre-established connections to interest-holder groups across Canada to inform them about the study and to ask them to share the study materials within their networks. This may also include asking them to share via traditional means of communication (e.g. e-newsletter, list serv). Potential participants (self-identified) will be able to access the study directly from the recruitment material by visiting the study link provided. The study link will include detailed instructions to complete the following: * Answer eligibility screening questions * Identify their first language * Review Study Information letter * Read KM resource to which they have been randomized (AI or professional translation in participant's first language) * Complete the questionnaire including demographics and responses to questions regarding the KM resource Incentives: Participants will be compensated $5 CAD. This incentive will come in the form of an electronic CAD gift card to the participant via Everything Gift Card. Recruitment will occur through publicly shared recruitment materials (e.g., email digests, electronic flyers, etc.) containing a study description and a link to the study. Interested individuals will self-identify as potentially eligible and access the study directly by clicking the link. Upon accessing the study link, participants will be presented with a brief set of screening questions to confirm eligibility. Those who do not meet the inclusion criteria will be automatically thanked for their interest and exited from the survey. At the end of the survey, personal information (first name and email) will be shared (if the participant chooses to do so) with the researchers on a separate form, unlinked to the survey responses, for the purposes of gift card distribution. Information collected will be housed separately from the study data in the Gift Card Log. Randomization and Blinding: Participants will be stratified by their first language then randomly assigned (1:1 allocation ratio) to one of two groups: the AI-translated KM resource or the professionally translated version. Randomization will be performed at the individual participant level using a computer-generated random allocation sequence to ensure unbiased group assignment. Participants will be blinded to the type of translation they receive. They will be informed that the study compares different translation methods but will not be told whether the materials they view were translated by AI or by professional translators. Data Collection and Management: Data will be collected and managed using REDCap (Research Electronic Data Capture). REDCap has an extensive privacy policy (https://help.redcap.ualberta.ca/policy-procedure/privacy) that has been reviewed by the study team and determined suitable for this study, and has been previously approved for research use at the University of Alberta. Data will be downloaded regularly and stored long-term on a secure server in the PI's faculty (Faculty of Medicine \& Dentistry) at the University of Alberta. Upon completion of the study, the data will be archived on the Faculty of Medicine \& Dentistry server for five years. Statistical Methods: The Investigators will collect demographic data (e.g., age, gender, ethnicity, education level, country of birth, health literacy, and proficiency in languages spoken) to describe the study population. The Investigators will use descriptive statistics (numbers, frequencies, means with standard deviations) to analyze and present demographic data. The Investigators will determine overall understanding by calculating a score (ranging from 0 to 7) for each participant based on the number of questions answered correctly. The Investigators will calculate a mean score for each group and compare groups (professional vs. AI translation) using a two-sample t-test assuming equal variances. Other outcomes will be compared between groups using independent t-test or chi squared tests dependent on the type of data (i.e., continuous or categorical). Statistical uncertainties will be expressed with 95% confidence intervals; p\<0.05 will indicate statistical significance. Analyses will be conducted based on intention-to-treat, i.e., all available data will be included and the Investigators will not know whether or the extent to which participants read the translated version of the KM tool.