A large language model writes by predicting the next likely word. It is very good at sounding right. It doesn't check whether it is right.
- Whose job
- Everyone who uses an AI tool
- Who usually leads
- Your digital lead, through training
- How often
- Every time you use one
How it works
A large language model (LLM) is trained on huge amounts of text. When you give it some words, it predicts which word is likely to come next, then the next, and so on. The UK government’s guidance puts it simply: LLMs “predict the next word in a sequence”, and don’t understand what they produce the way a person does.1
Try it. The odds below are made up, but the idea is real.
Illustration with made-up numbers
“The patient was started on ______”
Why it makes things up
Because it’s built to produce text that sounds plausible, an LLM can write something fluent, confident and wrong. The government’s guidance warns that output can look convincing while being factually incorrect.1 This is often called a “hallucination”. In a scribe, it might be a symptom the patient never mentioned, or a dose that was never said.
The other risk: us
The more often a tool is right, the less carefully people check it. This is called automation bias, and it’s why a person checking every output matters most once a tool feels reliable.
One UK study of AI in radiology found that, week by week over three months, radiologists agreed with the AI more and more. It was most marked in junior staff and out of hours, when people were busiest.3 So “a clinician checks it” isn’t a control on its own. It depends on time, training and workload, and those belong in your hazard log too. See the clinical safety basics.
Good habits
- Read it before you sign itEvery note, letter or summary, every time. NHS England’s guidance on AI scribes expects the clinician to review what the tool produces.2
- Watch for the specificDoses, dates, numbers, names and sides of the body are where errors hurt most.
- Approved tools onlyNo patient details in free or personal accounts. See your AI policy.
- Report the odd onesA strange output is a near miss. Report it so it can be fixed.
References
- Generative AI framework for HM Government. Cabinet Office
- Guidance on the use of AI-enabled ambient scribing products in health and care settings. NHS England
- Clinical safety and AVTs webinar. Newton’s Tree, April 2026. Haris Shuaib describes Newton’s Tree’s radiology study. Panel with Dr Keith Grimes, Dr Youssof Oskrochi and Haris Shuaib
Sources on this page last checked 6 November 2026.