The study assesses the potential of using computational models, specifically large language models, to simulate psychotherapeutic sessions, aiming to improve therapy outcomes and advance therapist training through innovative technology.
Health research has evolved significantly, increasingly incorporating computational models that improve our understanding and effectiveness of medical interventions. This shift from traditional to computational methods represents a major advancement in medical research, offering a more sustainable and innovative approach for conceptual advances and therapeutic discovery. In silico models, based on scientific simulation, use computational algorithms to mimic real-world systems or processes. This virtual environment allows researchers to explore phenomena impractical, unethical, dangerous, expensive, or impossible to study otherwise. Psychotherapy is widely acknowledged as a primary treatment for a variety of mental health conditions, from depression and anxiety to personality disorders, offering significant pathways to recovery and improved quality of life. Yet current methods have shown limited effectiveness, prompting a need for innovative research approaches. In silico psychotherapy research leverages computational simulations, large language models (LLMs), and generative artificial intelligence to explore and refine psychotherapeutic interventions. By simulating human-like conversations, this approach provides insights into therapy dynamics and holds promise for revolutionizing therapist training and expanding treatment techniques. This study aims to establish a proof-of-concept for simulating psychotherapeutic sessions using LLMs, focusing specifically on motivational interviewing. It involves the simulation of 512 psychotherapy sessions using LLMs as well as 8 real-world psychotherapy transcripts. By modeling human interactions, the study seeks to enhance healthcare delivery, therapist training, and personalized psychotherapy.
Study Type
INTERVENTIONAL
Allocation
NON_RANDOMIZED
Purpose
OTHER
Masking
SINGLE
Enrollment
520
The therapist large language model (LLM) is designed to show high levels of empathy, warmth, and genuineness. This setup aims to create a supportive and trusting therapeutic environment to improve patient engagement. High levels of these positive factors are linked to better psychotherapy outcomes and a stronger therapist-patient relationship.
The therapist LLM for this group is designed to show low levels of empathy, warmth, and genuineness. This setup aims to examine how a less supportive and empathetic therapist affects psychotherapy sessions. Lower levels of these positive behaviors can lead to reduced patient engagement and a weaker therapist-patient relationship, potentially hindering therapy outcomes.
University Hospital Basel
Basel, Switzerland
Simulation's Accuracy in generating Psychotherapeutic Dialogues
Assessment of the simulation's ability to accurately produce psychotherapeutic dialogues that adhere to the principles and techniques of motivational interviewing (MI), as determined by the average global scores of the Motivational Interviewing Treatment Integrity (MITI) code 4.2. The MITI code 4.2 includes various subscales, such as empathy and MI spirit, each scored on a scale from 1 to 5, with lower scores suggesting a need for improvement in MI delivery, while higher scores reflect stronger therapeutic skills and better patient outcomes.
Time frame: 12 months
Number of Errors/Deviations
The number of errors or deviations from expected psychotherapeutic practices is counted, providing a quantitative measure of simulation quality. This measure also serves as exclusion criteria from any other assessment.
Time frame: 12 months
Metric of Verbal Content (Therapist)
Assessment of the text metrics of the therapist, based on the number of sentences, words, syllables, characters, and lexical diversity.
Time frame: 12 months
Metric of Verbal Content (Patient)
Assessment of the text metrics of the patient, based on the number of sentences, words, syllables, characters, and lexical diversity.
Time frame: 12 months
Turn-takings
Assessment of the turn-takings, based on the number of exchanges between the therapist and patient within a session, indicating the dynamic interaction flow.
Time frame: 12 months
Improvement of Patient
Improvement of the patient is evaluated using an an observer-rated, circularly framed version of the Importance and Confidence Rulers, measuring the simulated patient's psychotherapeutic progress on a circular scale from 0 to 10. Lower scores indicate lower perceived importance or confidence, while higher scores suggest greater perceived importance or confidence in making the change.
This platform is for informational purposes only and does not constitute medical advice. Always consult a qualified healthcare professional.
Motivational interviewing techniques as applied during the sessions on which the transcripts are based.
Time frame: 12 months
Credibility of Patient Behavior
The credibility of the patient's behavior is estimated using a 0 to 10 scale indicating how likely the evaluator found that the participants are real humans or simulations, offering insight into the perceived authenticity of the simulated interactions. The credibility of the 8 real-world transcripts served as a comparison baseline/benchmark for this evaluation. Lower scores indicate lower authenticity of the patient large-language model's (LLM's) simulated behavior, while higher scores suggest higher authenticity of the patient LLM's simulated behavior.
Time frame: 12 months
Credibility of Therapist Behavior
The credibility of the therapist's behavior is estimated using a 0 to 10 scale indicating how likely the evaluator found that the participants are real humans or simulations, offering insight into the perceived authenticity of the simulated interactions. The credibility of the 8 real-world transcripts served as a comparison baseline/benchmark for this evaluation. Lower scores indicate lower authenticity of the therapist large-language model's (LLM's) simulated behavior, while higher scores suggest higher authenticity of the therapist LLM's simulated behavior.
Time frame: 12 months
Manipulation Check
The implemented level of psychotherapeutic common factors by the therapist-LLM is approximated using the Therapist Empathy Scale (TES) as rough manipulation checks. The TES rates the therapist's ability to understand and share a patient's feelings on a scale from 1 to 7. Higher scores indicate greater empathy, reflecting a stronger connection and understanding of the patient's emotions, while lower scores suggest less empathy.
Time frame: 12 months
Manipulation Check
The implemented level of psychotherapeutic common factors by the therapist-LLM is approximated using the Working Alliance Inventory Short Observer form (WAI-S-O) as rough manipulation checks. The Working Alliance Inventory uses 12 items evaluating the perceived therapeutic alliance between the patient and the therapist on a scale of 1 to 7.
Time frame: 12 months