Mayo's Nilay Shah: Concerns About Predictive Analytics

While in town for the Mayo Clinic Center for Innovation’s Transform conference, Healthcare Analytics News™ had the opportunity to sit down with Nilay Shah, PhD, of the Mayo Kern Center for the Science of Healthcare Delivery. Shah had thoughts on some of the more subtle challenges of predictive analytics implementation and also how to encourage their use.

He pointed first to a risk score stratification project from another prestigious health system, Kaiser Permanente, that opened eyes at Mayo as to a disadvantage.

“Kaiser is nice because they're a health plan and a provider, so they have both the EHR and the claims data and they're able to put it together and see the full picture for a patient,” he said. “For us, we are a provider, and so we see all this data on the EHR to implement the risk model, and what we realized is we don't know how much of the care the patients get out of our system. How complete is our predictive ability?”

The wonder, he said, was whether patients were getting care outside their own system that may lead their primary to assess a lower risk than actually existed. “If you're looking at ambulatory or chronic care long-term predictions, it becomes much harder to just rely on EHR data, just because of the lack of completeness, and not even knowing how complete those data are in the first place,” he said.

Shah pointed again to the need for analytics to be actionable: “I think we can get systems to use it a lot more if they are tied to interventions…better outcomes, lower costs, better reimbursement, or whatever it might be,” he said. Still, some things worry him. The abundance of data means it’s easier for analytics systems and recommendations to be generated, and harder to determine which ones carry weight.

“I am a little concerned broadly about the predictive analytics are being generated: are they using rigorous methods, are they calibrating, validating, and using good methods?” He asked. “Now with the availability of data that's one of the risks, that systems may not be as cautious in testing and using predictive analytics, that we need to be careful about.”