BY MARTIN LUSH | GxP AI IMPLEMENTATION SPECIALIST
A REAL Scenario: One of my clients ( a member of my AI in Pharma Interest group was inspected by an inexperienced auditor using their own LLM as an assistant to critique deviation (and other) reports for ‘gaps’.
My client disagreed with many of the AI outputs, causing ‘friction’ with the auditor - who seemed convinced by the LLMs precise (and confident) output. My client wasn't prepared for this and didn’t know what to do. During multiple calls this is how it developed:
"Martin, are they allowed to do this?"
To my knowledge there's no regulation that stops an investigator from using an LLM to support inspection activities. I asked them about their own policy. They didn’t have one that prohibited use. They were also very nervous about challenging for fear of escalation. I said it wasn't a question of ‘if’ they challenged (they had to), it was more about ‘how’. We brainstormed concerns to help formulate a strategy.
If an investigator entered confidential GMP information into a public LLM (e.g. consumer version of ChatGPT etc.) without approved safeguards, this could create serious confidentiality and information governance concerns. Their in-house policy should make this clear (it didn’t).
Potential issues:
But is ‘confidentiality’ risk over stated?
Logically - agencies are unlikely to use public ChatGPT accounts for confidential inspection data. But how long before agencies leverage:
After all - a knowledgeable, AI literate inspector could dramatically improve their auditing efficiency and effectiveness and quickly! If I were in their shoes I would. Industry must be prepared.
Regulatory decisions must always be based upon evidence, facts, applicable regulations and scientific judgement. If an inspector can't explain:
Then there may be concerns regarding transparency, reproducibility and procedural fairness A company must legitimately challenge:
"Please explain how you arrived at that conclusion."
This may be the largest practical risk. An inexperienced inspector might ask:
"Review this deviation and identify deficiencies."
The model will search for deficiencies. A different prompt:
"Assess whether this deviation investigation demonstrates a scientifically justified conclusion."
will produce a completely different answer. The prompt becomes part of the inspection methodology. As humans we're also prone to 'anchoring' to the first output. This attachments is hard to shift.
Current regulatory thinking increasingly expects explainability. I 100% agree. So, if industry is expected to explain AI-assisted decisions, the same standard must apply to regulators. Should regulators be permitted to use less explainable systems than those they permit industry to use? No.
During the multiple calls that followed this is what we came up with (currently a work in progress):
Create a formal procedure for responding to AI-assisted regulatory observations. Questions should include:
Avoid: "The inspector is wrong." Instead ask: "Can we review the assumptions underlying that conclusion?" Focus on:
Less confrontation and more science is always better
If AI is being used to critique investigations, investigations must become more robust (a good thing) to include:
Essentially make the thinking visible. We should be doing this anyway The more transparent your reasoning, the harder it is for an AI-generated criticism to gain traction.
If concerns arise Ask:
The objective is not to challenge the use of AI but any reliance upon unverified AI output.
Most companies are currently preparing staff to use AI and NOT preparing them to defend against AI-assisted scrutiny. So, train SMEs to recognise:
Otherwise incorrect observations become institutionalised because nobody pushed back.
The real issue may not be whether inspectors use AI. If I was a regulator – I would. It's inevitable. After all, if AI can, when used correctly, review 5,000 deviations in hours it would be UNETHICAL not to use it.
The real issue is whether regulators develop standards for AI-assisted inspections analogous to the standards they expect from industry. If pharma must demonstrate:
Regulators must be held to the same principles when AI materially influences inspection outcomes. AI-assisted inspections are coming if not already here. The objective is not to resist them. it is to ensure there transparent, explainable, evidence-based, and subject to the same scientific rigor expected of regulated industry.
So, remember to include this scenario in your next Mock Inspection Exercise
