A peer-reviewed study published in JMIR Formative Research examined whether conversational AI could help address that. The study is an independent evaluation of SimFlow.ai in real UK primary care training, with findings that are specific, honest, and worth reading in full.
This World Health Day, we are asking what it means to 'stand with science' in the world of healthcare simulation. When a new tool enters clinical training, what should we reasonably expect of it, evidence-wise? Science surrounds the industry; it is what simulation strives to replicate. So, it follows that the tools we use to practise exacting medical science should be subjected to that same objective scientific scrutiny – to ensure simulation is standing with science every step of the way.
Cost, logistics, and lack of standardisation at scale are fundamental challenges that communication skills training faces. These create consequential gaps in communication competencies that continually impact patient care. It is against this backdrop that a new peer-reviewed study evaluating SimFlow.ai has been published – welcoming both its findings of strength and its identified areas for improvement with equal regard.
The research serves as a meaningful, well-designed early-stage study into agentic AI scalability in medical education. The study used a cross-sectional design, running the SimFlow.ai platform across more than 70 simultaneous teaching practice interfaces in a single morning session, generating 47 questionnaire responses from medical students and GPs across a census sample of one UK medical school and its 70 associated general practice training sites.
Participants all completed the same standardised clinical scenario and evaluated the platform across five domains:
These domains were selected for their direct relevance to the key implementation questions that determine SimFlow.ai's efficacy as a supplementary training tool.
The study's own authors note its scope limitations plainly: single institution, cross-sectional design, and some collaborative survey responses that may carry group dynamic effects. An adjusted response rate of 61.3% and a pre-registered protocol on the Open Science Framework reflect the methodological care taken. A prospectively registered follow-up study is already underway.
The study identified four qualitative themes tied to the effectiveness and future directions of agentic AI in scaled medical education: clinical authenticity, interactional limitations, educational potential, and implementation considerations. Alongside these themes, the quantitative findings reveal a clear gap between the high accuracy of the AI's medical 'logic' and the ongoing challenges of natural, human-like conversation.
Participants with prior AI experience rated realism significantly higher — suggesting that how learners are introduced to AI simulation is an important variable determining their expectations and experiences. Onboarding design matters as much as platform design given this expectation effect.
These findings offer practicable implications for implementing AI simulation software into communication training settings. Perhaps the most significant of these sits in the relationship between the two fidelity scores: educational value was rated positively even where conversational realism scored only moderate. This finding directly challenges the assumption that simulation must feel completely real to produce learning.
This platform and research are a clear example of what evidence-into-action looks like in healthcare simulation education: an AI tool open to real training deployment, user scrutiny, peer-reviewed evaluation, and considered improvement — which is precisely the empirical standard of science and simulation.
SimFlow.ai is addressing a real and well-documented scale issue in communication skills training, in as considered a way as current AI technology allows. Its realism will continue to develop in step with the capabilities of that technology. What the research gives us, for now, is an honest account of where those capabilities currently sit.
At Sim & Skills, we believe the value of AI in healthcare education is directly proportional to the rigour and honesty of its evaluation. The research gives us a clear-eyed basis for understanding what SimFlow.ai currently does well and where development continues. We think that is the right foundation for any training provider considering the platform — and we offer it as the basis for you to reach your own view.
Sim & Skills supported Elevate Healthcare at HPSN UK 2025 for two inspiring days of innovation in healthcare simulation. From cutting-edge patient simulators to interactive workshops and thought-provoking talks, the...
How do you choose a new Catheterisation Trainer? With so much choice, it's hard to find the right model to meet your needs. Our handy comparison guide compares our top...
How do you choose a new Airway Management Trainer? With so much choice, it's hard to find the right model to meet your needs. Our handy comparison guide compares our...
How do you choose a new IV injection training arm? With so much choice, it's hard to find the right model to meet your needs. Our handy IV arm comparison...
Find out why Epilepsy training is so important. Discover how Buccal Injection Trainers are a safe and cost-effective way to train staff to administer life-saving Epilepsy medication.
Paediatric training is designed to meet the challenges and develop the skills needed to care for young children. You need the best paediatric patient simulators and task trainers to meet...
RECAP: Top 5 Airway Mangement Trainers: Airway Management Simulator with Feedback from BT Inc Airway Management Trainer Complete from Decent Simulators Airway Management Trainer (Basic) from BT Inc Advanced Airway...
Having the right manikin means you can achieve your learning outcomes, improve the confidence of your trainees and increase patient safety. But with so much choice, how do you select...
