Validation and Trust: The Governance of AI Solutions at Health Systems

Download Report

Health systems and hospitals are widely adopting AI solutions in administrative and clinical workflows, but there are significant gaps in strategy, validation, testing environments, and success measurement, according to a new report from the Center for Connected Medicine at UPMC and KLAS Research.

The report examines how health systems are adopting and governing AI and is based on a survey of more than two dozen health system leaders. The research reveals that while AI adoption has accelerated rapidly across the industry, many organizations are still working to establish the infrastructure, governance frameworks, and strategic foundations needed to support long-term success.

“The health care industry has moved remarkably quickly from discussing the potential of AI to actively deploying solutions across the enterprise,” said Rob Bart, MD, chief medical information officer at UPMC. “What’s emerging from this research is a clear recognition that implementation is only the first step. Health systems are now focused on building the governance structures, testing capabilities, and organizational strategies necessary to ensure AI delivers meaningful and measurable value.”

The findings paint a picture of an industry moving quickly to embrace AI technologies, particularly in administrative and clinical workflows, while simultaneously grappling with the complexities of implementation, validation, and oversight.

Key Findings from the Survey of Health System Leaders

Among the report’s key findings:

• More than 90% of health systems have deployed third-party AI solutions, demonstrating that AI has moved beyond experimentation and into mainstream use. 

• Clinical documentation (52%) is the most commonly cited area for AI deployment, followed by revenue cycle, coding, and billing (36%) applications. 

• Testing of AI solutions is nearly universal, with 92% of organizations reporting that they evaluate third-party AI tools before deployment. However, validation methods vary considerably, ranging from formal vendor testing protocols to limited pilot programs and other informal approaches. 

• Less than half of respondents (44%) report having a dedicated data platform or environment for testing AI solutions, highlighting potential challenges in evaluating performance, safety and scalability. 

• Most organizations’ AI strategies are evolving, with 63% indicating their approach to AI is developing or ad hoc rather than fully established or advanced. 

• Leading barriers to AI adoption include limited resources, insufficient time, and shortages of specialized talent, reflecting the realities of implementing emerging technologies in complex health care environments. 

• Respondents reported no strong consensus on the metrics used to measure AI success, underscoring the need for greater alignment around how organizations evaluate return on investment and value. 

The report explores how health system leaders are balancing the urgency to adopt AI-powered solutions with the responsibility to manage risk and validate performance that support trustworthy and effective use of the technology. 

UPMC’s Ahavi Data Platform for Testing AI Solutions

UPMC’s response was the development a real-world data platform called Ahavi that can validate and improve third-party AI models before deployment at UPMC. Ahavi leverages deidentified patient data to test AI solutions “in silico” to see their effect on wide-ranging applications without disruption to patient care.

“As AI adoption continues to expand, health care providers are recognizing that success depends on more than selecting the right technology,” said Ken Howard, vice president at UPMC Enterprises, the innovation, commercialization and venture capital arm of UPMC. “It also requires the right data infrastructure, governance processes, workforce capabilities, and evaluation frameworks to support responsible innovation.”

The report is intended to help health care executives, technology leaders, clinicians, and operations teams benchmark their organization’s AI maturity and better understand the challenges and opportunities facing the industry.

Next Steps

• Download a complimentary copy of the full report, “Validation and Trust: How Health Systems Are Testing and Governing AI Solutions,” by submitting the form below. 

• Learn more about the Ahavi platform and contact the team.