Tools to evaluate and validate AI solutions are essential for innovative health systems 

Center for Connected Medicine Technology Services

There’s an ocean of opportunity for innovation-focused health systems to develop and leverage third-party AI solutions, even as heavyweight vendors look to offer AI that’s integrated into electronic health record (EHR) systems.

That’s the view of Jeffrey Jones, Senior Vice President of Product Development at UPMC Enterprises, the innovation, commercialization, and venture capital division of the UPMC health system. Jones is leading an effort at Enterprises to build a series of tools that will allow UPMC to evaluate and validate new AI solutions for implementation and investment without disrupting the broader IT services at one of the nation’s largest integrated health systems. 

The work at Enterprises on these tools comes as health system leaders across the country consider their options for implementing AI solutions. A recent Center for Connected Medicine at UPMC (CCM) research report found that a significant number of health systems are likely planning to adopt AI solutions via EHR vendors due to the easy integration and the ability to work with fewer vendors. 

More than 70% of respondents in the CCM survey research said their organizations were planning to adopt AI solutions integrated with their EHR. Just 10% said their organizations would not be implementing AI solutions from an EHR vendor, according to the report, “How Health Systems are Navigating the Complexities of AI.” 

While there are many benefits of adopting AI solutions that are integrated into an EHR — including lower operational costs and ease of use for clinicians — EHR vendors aren’t likely to address every use case in health care where AI can drive improvements.  

“There is an ocean of opportunities for AI to improve health care, but for an investment division like UPMC Enterprises to take advantage of those opportunities we need to determine the viability of solutions and validate the efficacy of models across our patient populations without disrupting of UPMC’s IT services,” Jones said.  

That’s where the Technology Services team that Jones leads come in.  

“We are looking at areas where AI can ultimately accelerate the operations of the broader health system or positively impact clinical care, but that aren’t being delivered by way of the EHR. But without the tools we’re developing to assess the impact of the AI solution under consideration, we’d just be guessing,” he said.  

Potential of generative AI in health care 

The CCM research not only asked health system leaders about their plans for AI solutions integrated into EHRs but also about where they see generative AI adding value in health care. Generative AI solutions, such as OpenAI’s ChatGPT, are creating excitement across many industries for a range of use cases.  

Generative AI models can create original content based on user prompts. Generative AI learns from large datasets and then generates new data such as text or images based on patterns the models have learned from existing data.  

In the CCM survey, executives identified improving efficiency, bringing more visibility to clinical decisions, and automating repetitive tasks as the top three ways they expect generative AI to enhance health care.  

While Jones is supportive of the potential for generative AI to complement health care, he also cautioned health system leaders to be aware of possible pitfalls. Chief among them is the need to consistently monitor the models to ensure that they remain accurate.  

“Before adopting generative AI technologies in health care, it’s crucial for executives to clearly define their objectives and establish measurable benchmarks,” he said. “Regular evaluations are essential to adjust strategies as necessary. Generative AI is not a one-time fix, but a dynamic tool that requires attention and calibration.”  

What to read next 

Download a complimentary copy of the CCM’s report, “How Health Systems are Navigating the Complexities of AI.” 

Learn more about Technology Services at UPMC Enterprises and its full suite of technology services to help your business solve problems, create innovative solutions, and deliver outcomes. 

You Might Also Like…

More From the Blog