101
General manager, AI platforms at ArisGlobal.
ABSTRACT: Pharmacovigilance teams have largely stopped debating whether AI systems can be validated. The sharper question, addressed in new CIOMS XIV guidance, is how organisations keep proving that validation still holds once a system is live, updated and running at scale. ArisGlobal’s Jason Bryant unpacks what the guidance means in practice, building on a recent podcast conversation with Denny Lorenz, an active member of the CIOMS XIV working group.
The CIOMS Working Group XIV report on AI in pharmacovigilance, published at the end of last year (1), marks a pivotal point in AI operationalisation in a drug safety context. The comprehensive guidance and supporting framework is the collaborative output of regulators, academia and industry internationally, and it has been painstakingly crafted and refined.
Already in development before ChatGPT’s launch in late 2022, the original guidance assumed that organisations would be training their own machine learning models on proprietary data. Pre-trained, general-purpose GenAI models upended that assumption almost overnight, causing a rethink. The guidance now published transcends a particular technology, focusing instead on a series of widely-applicable principles, hopefully giving the recommended framework a decent shelf-life.
Already in development before ChatGPT’s launch in late 2022, the original guidance assumed that organisations would be training their own machine learning models on proprietary data. Pre-trained, general-purpose GenAI models upended that assumption almost overnight, causing a rethink. The guidance now published transcends a particular technology, focusing instead on a series of widely-applicable principles, hopefully giving the recommended framework a decent shelf-life.
In response to feedback elicited during a consultation period, the Working Group has also included provisions for how PV teams can convert the principles into something auditable. As a result, the published guidance includes detail on how the framework fits alongside existing quality and regulatory systems organisations, and includes specifics on lifecycle and governance, as well as what counts as practical evidence.
The 7 principles
At a high level, the CIOMS Working Group XIV provides guidance on:
- Risk-based approach: How much validation, monitoring and documentation a system needs should scale with what’s actually at stake if it goes wrong.
- Human oversight: The human role in designing, monitoring and reviewing AI systems, and how people catch and correct problems.
- Validity and robustness: Noting that a system needs to perform reliably for its intended purpose under real-world conditions, with checks that continue well past launch.
- Transparency: Openness about how a system works and performs, so people understand its risks and limits.
- Data privacy: The need for AI systems to protect personal data and respect people’s right to control their own information.
- Fairness and equity: The specification that AI shouldn’t disadvantage particular groups, which means identifying and correcting bias throughout a system’s lifecycle.
- Governance and accountability: Every AI system needs an owner, with clear roles and oversight in place so organisations answer for how it’s used.
Matching scrutiny to risk
Ad hoc AI use raises the same governance question as any purpose-built system. A reviewer drafting case notes with a general-purpose AI licence needs the same scrutiny as one relying on a dedicated extraction tool, even though currently only one of these is actually checked.
Human oversight is where that calibration tends to break down most often, something the guidance doesn’t alter. Too often organisations still treat a reviewer in the workflow as an all-encompassing guardrail, when oversight is just one necessary control among several. A reviewer might check every single case, for instance, but without the PV system master file being documented, data privacy being addressed, or performance metrics to highlight whether the reviewer is actually catching something meaningful. (Oversight only counts as evidence to support a risk-based approach if the reviewer’s own judgement is also being measured.)
Equivalent measurement must extend to the system itself, too. Validity and robustness, a separate principle in its own right in the CIOMS guidance, requires ongoing monitoring. This is because an AI system’s inputs will vary case by case (there will also be new prompts and subtly updated models over time), so validating once at launch isn’t enough. Poignantly, accountability for the safety judgement stays with a qualified PV professional throughout.
Gauging readiness
The CIOMS guidance also provides a governance grid — essentially a diagnostic aid for testing whether a desired use case is ready for production. Items to check against include whether there is a documented risk assessment, a described oversight process and a governance structure which allows for reassessment (vs a one-time sign-off). The grid doesn’t require each box to be ticked to move forward with a new use case however, as long as any gaps are identified and being worked on.
In terms of when a PV subject matter expert (SME) should become involved in a project, this was a point of contention during the Working Group’s deliberations. Some members argued that SME involvement should be kept to formal sign-off at production, while others pushed for input from the conceptual phase, alongside vendors from day one. This is a familiar tension in any regulated, technical discipline — whether specialist input is a late-stage check or a design constraint from day one. The published guidance builds early involvement into the governance grid, on the basis that an SME who understands a model’s limitations before it is built is likely to ensure a better-informed risk assessment.
That risk assessment work pays off later too, once a system has a production track record behind it. Where now, even cases with 200-300 distinct fields are all typically still reviewed as a matter of course, irrespective of how reliable the AI extraction is seen to be, CIOMS’ proposed risk- and evidence-based approach could be interpreted as paving the way for greater discernment around human oversight going forwards. For instance, once a system has racked up a full year of dependable performance on a specific field, organisations may feel they can start narrowing review down to only the low-confidence extractions and let the rest through. Regulators haven’t yet said whether they’ll actually accept reduced review once organisations ask for it, however. This remains an open question, and is why industry and regulators need to keep talking — through bodies like CIOMS itself — not just rely on the guidance and hope for the best.
In the meantime, the course has been set and companies have little choice but to follow where the industry is going. The EMA and FDA, which published their own joint principles for AI across the medicines lifecycle in January 2026 (2), are also seeking validation and oversight that scale with a system’s risk. They, too, want organisations to schedule monitoring throughout a system’s lifecycle rather than check it once and move on.
The upshot is that trusting an AI system should never be seen as something served by a single checkpoint, passed once and forgotten, so validation and governance need to provide for that.
About the author
Jason Bryant is general manager, AI platforms at ArisGlobal. He leads the build-out and scaling of the company’s AI-native platform, NavaX, across the life sciences industry, with a focus on bringing agentic AI into pharmacovigilance in a way that keeps governance and human judgement central to the process.
References and notes
* This article draws from a recent AI Exchange podcast discussion with Denny Lorenz, who has spent more than two decades working in Safety and Pharmacovigilance and is an active member of the CIOMS XIV working group.
- Council for International Organizations of Medical Sciences (CIOMS), ‘Artificial Intelligence in Pharmacovigilance’, CIOMS Working Group XIV report, Geneva, 4 December 2025. Available at: https://cioms.ch/working_groups/working-group-xiv-artificial-intelligence-in-pharmacovigilance/
- European Medicines Agency and U.S. Food and Drug Administration, ‘EMA and FDA set common principles for AI in medicine development’, 14 January 2026. Available at: https://www.ema.europa.eu/en/news/ema-fda-set-common-principles-ai-medicine-development-0
