The Efficiency Case Builds, the Trust Case Wobbles

The National Commission into the Regulation of AI in Healthcare, convened by the MHRA and co-chaired by Professor Alastair Denniston and Patient Safety Commissioner Professor Henrietta Hughes, has published its recommendations for a future regulatory framework. The centrepiece is a staged, "L-plates" style approach: AI models would launch under close supervision and tight constraints, prove themselves on real patients, and only then graduate to wider, less restricted use — replacing a single point-in-time approval with lifetime monitoring of every AI-enabled device.

The recommendations draw on one of the largest public engagement exercises the MHRA has run on this topic — around 12,000 contributors, including patients, clinicians, NHS leaders, developers and international input from Singapore and the United States, with deliberative research led by the Health Foundation. Alongside the staged-approval model, the commission proposes a searchable public record of safety incidents for individual AI devices and stronger MHRA enforcement powers, so continuous monitoring becomes an ongoing obligation for developers rather than a one-off filing.

Lifetime monitoring is not just a regulatory burden on suppliers — it changes what an NHS organisation is implicitly signing up to when it deploys an AI tool. If continuous performance and safety monitoring becomes the standard, provider organisations need their own version of it: a designated owner for each live AI tool's ongoing performance, not just its go-live business case. A commission recommendation is not yet law, but a safety case built as if it already were will age far better than one that isn't.

Proximie has launched what it describes as the largest surgical AI evaluation undertaken in the NHS, deploying its Intelligence Suite across eight NHS hospital sites and 104 operating theatres, catheterisation laboratories and endoscopy suites, with cloud and AI infrastructure from AWS and transformation support from Deloitte. The 12-month evaluation aims to cover more than 20,000 procedures.

The software's job is unglamorous but concrete: identify where delays accumulate between one procedure and the next, and use that data to schedule available theatre time more tightly — treating more patients in the theatres and with the staff already in place, rather than requiring new facilities. Proximie founder Dr Nadine Hachach-Haram says the evaluation is "built to find out how big an impact this makes across the NHS, at scale"; Deloitte's Sara Siegel frames the target plainly: operating theatres are "the most costly and resource-heavy part of a hospital."

A capacity play is still a clinical-adjacent decision: reshuffling theatre time changes whose case goes first, and by how much. The evaluation's honest test isn't whether throughput rises — it's whether the gains persist once the pilot's attention and novelty wear off, and whether the scheduling logic can be explained to the surgical teams and patients it reorders. Efficiency measured only in cases-per-day can hide who waited longer to make room for someone else.

A peer-reviewed review by University of Edinburgh researchers, published in a BMJ digital health and AI journal, finds that roughly 40% of UK GPs now use AI note-taking tools in consultations — and identifies critical gaps in what ambient scribes capture. Facial expressions, gestures and emotional states are lost in transcription, patient narratives are deprioritised in favour of clinical facts, and patients become more guarded about sensitive disclosures — substance misuse, domestic abuse, mental health — when they know they're being recorded.

The benefit case is real and sits in the same review: one Dudley clinic using the Heidi tool cut its patient-letter backlog from six months to two weeks. Lead researcher Dr Lucas Seuren doesn't dispute the efficiency gain — his concern is what's missing from the record it leaves behind: "the experiences of patients are poorly considered, and there are real risks that the patients' stories are lost." The review also flags a deskilling risk it calls cognitive offloading — clinicians losing memory recall and note-writing skill as the task is automated away from them.

Most AVT safety work to date audits transcription accuracy — did the scribe mishear or miscode something. This review points at a different failure mode entirely: the scribe can be word-perfect and still lose the parts of the encounter that carried the actual clinical or safeguarding signal, because those parts were never spoken plainly in the first place. An accuracy audit will not find that gap. Only asking patients and clinicians directly will.

Health Innovation Minister James Frith has told the Commons Health Committee that mistrust of Palantir, which runs the NHS Federated Data Platform under a seven-year, £330m contract, correlates with a surge in patients withdrawing their records from NHS research: 60,000 additional opt-outs between mid-May and mid-July 2026 alone.

The Federated Data Platform is the infrastructure sitting behind a growing share of clinical AI benefit claims: Palantir cites 110,000 extra operations enabled, a 15% reduction in long-stay patient discharge delays, and a 6.8% improvement in 28-day cancer diagnosis rates. Frith's warning to the committee was direct: "It may not be possible to realise the benefits of the 10-year health plan if patients stop sharing their data."

The UK Statistics Authority is now examining Palantir's own performance claims, and ministers are reportedly considering whether to invoke a break clause in the contract — with critics citing the company's work with the Israeli military and US Immigration and Customs Enforcement as the source of public unease. Whatever the outcome, the lesson for any organisation running clinical AI on shared data holds regardless of supplier: the benefit case and the trust case are the same dataset, measured two different ways, and a fall in one eventually caps the other.