AI applications – where, what and why?

• What AI applications are you using and why? What are their real drivers? Costs, efficiency, process optimisation, prevention and risk reduction?
• Which models are you using? Are they explainable, validated, and trusted?
• SaaS providers: What checks, balances, controls and accountability exist?
  

If you were inspected tomorrow, could you answer these confidently?

Ethics, governance and guardrails

• Who created your governance? (Top-down often weak. Bottom-up builds stronger compliance.)
• Do you have a clear Terms of Reference? Make sure top three are there:
  
  • Patient / public safety and human oversight
  • Transparency, traceability and accountability
  • Bias, integrity, and data stewardship
• Have you Red-Teamed your AI policy? Show me the risks you've tested and mitigated. Please remember blanket bans (“We don’t allow LLMs”) are unrealistic. If staff have the apps, assume they use them.

Regulators are starting to see through governance “words” that never reach the shop floor. Are you ready? Are your structures and routines in place?

Operational discipline and experts (not just humans) in the loop

• What oversight structures, routines, and measurements are in place? Do you have fast feedback loops?
• Are your “experts in the loop” truly educated in AI foundations and are experts in products and processes. Are they capable of asking the right questions and critical analysis?
• What happens when experts disagree with AI outputs? (if they always agree - be worried. Very worried!)

Future plans and sandboxing

• With new AI models arriving weekly, what are your future plans and how are you testing safely?
• Do you have a sandbox for trial and failure-safe learning?

AI education (not just training): The BIG One

• Do staff understand machine learning, deep learning, neural networks (pros and cons) or just the shiny tools? From the Board to the Shop Floor.
• Can they apply critical thinking frameworks (Red Teaming, Six Hats, Cynefin, ABC of Critical Thinking) Test critical thinking by asking.

'Please explain' - always challenge outputs and critical thinking

Remember, most people will default to AI tools because they are easier. Look for misuse and abuse by challenging reports and investigations. Ask them to explain

• How did you get to this conclusion? Show me your ‘workings’
• What data did you use, what assumptions did you make and why?
• What other options did you consider?
• What is the basis of your risk assessment?

If they can't explain their reasoning, an LLM probably did it for them - badly.

Why this matters

Some regulators are already asking me these questions so make sure you are prepared. Please remember:

• Organisations that prepare now will build credibility and trust with inspectors.
• You will be ahead of future guidance rather than scrambling to catch up.

Over to you

What would you add to this list? Which of these questions would challenge your teams today? How are you preparing your people for the AI questions regulators will bring?