Health providers, look for these 'red flags' in AI pitches

Doctors and hospitals have no shortage of AI vendors vying for their attention—and budgets—these days.

But how can providers sift through all the noise to select tools that will actually be meaningful for their work and their patients? And when should hospital systems opt to build their own tech rather than buying it?

A panel of healthcare professionals considered those questions and more at a New York Tech Week event a few blocks west of one of Manhattan’s hospital rows.

Many big companies and startups have zeroed in on the healthcare industry as a proving ground for AI potential. But at a time when most AI projects fail, picking the right tech partners can be tricky.

What to watch out for

Komal Bajaj, an OB/GYN and geneticist who recently joined the startup Biote as its chief medical officer, shared several telltale red flags from her experience fielding AI pitches during her tenure in New York City’s public health system.

One is when the team making the pitch lacks a basic understanding of publicly available information about the health entity they’re pitching to—data like existing performance metrics, demographics served, and payer mix, Bajaj said.

“Do the best you can to really understand enough about the organization and the people they serve,” she added.

Not having a clinician involved is also a warning sign, as well as not identifying the existing process to be replaced by AI, Bajaj said.

Yet another is a lack of practical implementation and details about short-term goals. She’ll often hear, “It’s this huge problem, and you’ll save lots of huge monies, and there’ll be this huge impact.’ I’m like, ‘What happens day one?’” Bajaj said. “What are you measuring day one to know that you can go to day two?”

Still low-hanging fruit

While there are “a lot of shiny objects [and] a lot of noise” in healthcare right now, Kabir Daya, chief digital officer at mental health provider Thriveworks, said health systems still have many processes where simpler automation could yield outsized value.

“We’re in a moment also where people are rewarded for going AI-first, no matter what…But there are still things that work. There’s a lot of low-hanging fruit in healthcare. A simple change to a process, or do a basic automation, or maybe a [robotic process automation],” Daya said. “There’s a lot of value there, but we should be choiceful.”

Daya said Thriveworks’ approach to the build-versus-buy question has been to focus development efforts on areas where Thriveworks’ data and expertise can give it an edge.

“I don’t want to spend time and resources in a space where it’s not gonna help me build a competitive advantage or any sort of moat. Is this something unique to me or is this commoditized? I don’t want to waste time there,” Daya said. “I’m going to invest those dollars where I think we have unique IP in the long run…And so a lot of that has been on our clinical protocols, on our pathways, on the experience for the clinician that’s unique to us.”

Hands-on experience helps

Hands-on experience with AI tools is also key to giving clinicians a better sense of what works, according to Kathleen Léger, director of women’s health at Extension Health. She said she’s combined her own expertise as an internist, nephrologist, and intensivist with various AI tools. For instance, a pattern recognition protocol can help check her work when stabilizing patients in the intensive care unit.

“That’s very protocolized—you go in, you stabilize them. Once you do that, then what’s the next step? I did many, many, many years of training to actually recognize patterns. AI is able to do that a lot faster,” Léger said. “It doesn’t replace—it enhances the care I can provide, you know, because it catches my blind spots.”