Healthcare Brew covers pharmaceutical developments, health startups, the latest tech, and how it impacts hospitals and providers to keep administrators and providers informed.

Healthcare Brew

illustrated headshot of caroline catherman, healthcare brew reporter

Caroline Catherman

Healthcare Brew delivers the latest news and trends in healthcare.

healthcare-brew.com

The number of FDA-cleared artificial intelligence-enabled medical devices has 

 in 2023 to 1,247 as of Aug. 28, according to the federal agency’s 

And healthcare systems are getting on board. In a 

, about 88% of 233 health system executives said their system uses AI.

Though only a small percentage have been recalled, 60 AI-enabled devices racked up 182 recorded recall events as of November 2024, according to an Aug. 22 JAMA 

Throughout these recalls, there was a consistent thread: Publicly traded companies’ devices (about 92%) were recalled more frequently than private companies’ devices (53%).

“That was really something we were surprised by because we were thinking public companies probably have better risk management,” Tinglong Dai, Bernard T. Ferrari professor at the Johns Hopkins University Carey Business School and one of the paper’s authors, told Healthcare Brew.

The paper theorizes public companies could feel pressure from investors to put out devices as quickly as possible, perhaps not giving them time for more thorough research.

“Investors, they want to see year after year you have new AI devices cleared by the FDA, you have AI devices being rolled out among healthcare providers, so the pressure itself might have contributed to the recalls,” Dai said. 

There were several main reasons for the recalls—a designation that ranges from a quick software update to total market removal—with 109 recalled due to diagnostic or measurement errors, 44 for a functionality delay, 14 for physical hazards, and 13 for biochemical hazards.

“If you have a misdiagnosis, obviously the entire healthcare delivery could be subject to risk because wrong diagnosis means that you could also have the wrong treatment,” Dai said.

The research letter identified another pattern, too. The majority (40% from the private companies, 78% from established public companies, and 97% from smaller public companies) of the recalled devices weren’t clinically validated before they were approved.

 are given the greenlight through the FDA’s 510(k) pathway, which clears a device if it proves it is “substantially equivalent” to a previously cleared device. Clinical data is not normally required to get an OK via a 510(k), as opposed to other clearance processes.

This is something health systems and providers should keep in mind when deciding 

“From the user’s perspective, I think it’s extremely important to understand…the difference between reported performance and real-world performance,” he said.

Even AI programs with strong premarket clinical data “frequently” perform worse in diverse real-world clinical settings, according to a March 2025 

This is one of the reasons the American Medical Association uses the term “augmented intelligence” rather than “artificial intelligence,” according to the organization’s website. AMA 

 says that AI is not supposed to call the shots or operate without oversight by a human healthcare professional.

A Department of Health and Human Services spokesperson told Healthcare Brew the agency “is not able to confirm” the numbers cited in the JAMA study, but “remains committed to ensuring patients have access to safe and effective medical devices, including those enabled with AI.”

In addition to general device software validation and safety requirements, the agency released 

 in January specifically for AI-enabled devices, like what data and documentation is needed to show safety and effectiveness.

Tinglong Dai’s title has been clarified.

Pill capsule with tech system inside, slightly open, and a honeycomb-like web in the background

Over 90% of recalled AI-enabled medical devices were from public companies

Though only a few AI-enabled devices have been recalled, patterns are emerging.

Cassie McGrath

 (EHR), you likely think of Epic, as the healthcare software company has records for 325 million patients and is the largest in the US.

And when you think of the modernization of the EHR, you may think of AI scribes like 

, the note-taking assistants designed to speed up clinical documentation.

Scribes can save physicians 15,000+ hours in work per year, 

 published in the journal NEJM Catalyst Innovations in Care Delivery reported, and between $200 billion to $360 billion in annual healthcare spending, 

But this month, the inevitable happened: 

 it was developing its own AI tools, including Art, an AI scribe that will, like other apps, share medical information with clinicians in real time. It’s part of Epic’s “native AI charting” developed in partnership with Microsoft, and is expected to launch for “limited use” early next year, a spokesperson from the company told Healthcare Brew and other outlets.

So what will happen to the scribe startups when the EHR giant releases its own AI tools?

Experts told Healthcare Brew they expect some market shift, but think their technologies provide strong value to providers.

“I wouldn’t be a very good founder if I didn’t think this was already going to happen,” Tom Kelly, co-founder and CEO at AI scribe developer Heidi Health, said.

You may have noticed, but Epic isn’t only releasing an AI scribe. It’s looking to build a suite of AI tools to help everywhere from the patient room to the back office, according to the company.

This highlights a bit of a trend in the health tech industry: Execs don’t want to just make scribes; they’re instead hoping to automate the entire healthcare workflow.

In fact, Ambience Healthcare—one of the leading AI scribe developers in the US, with at least 40 clients like the Cleveland Clinic and the Houston-based Memorial Hermann Health System—made its own expansion announcement, entering a $243 million Series C fundraising round on July 29.

Then on Aug. 19, Ambience released an AI copilot tool, Chart Chat, which works within Epic to provide real-time health information to clinicians, like lab results and prior treatments.

“We still feel like we’re 5% of the way through everything we possibly want to build,” Nikhil Buduma, Ambience’s co-founder and chief scientist, told us.

AI healthcare company leaders are also confident in their technologies.

The two co-founders said they think Epic will provide an accessible—and possibly cheaper—internal solution, which may be better suited for an outpatient facility. But Buduma said academic medical centers may want something more high tech since they tend to care for more patients and handle more complex cases that require added clinical integration.

It’s also important to differentiate between competitors.

For example, Kelly said Heidi Health is focused on building a tool that can be used in Epic but isn’t necessarily 

 to be. This means it can also complete tasks, like helping with forms outside of the medical record, in addition to its note-taking capabilities, Kelly said.

AI scribe companies also have existing contracts with providers. Ambience and Cleveland Clinic signed an agreement for five years back in February, for instance.

Just because Epic has its own scribe now doesn’t mean these partnerships will disintegrate, the two co-founders said. Buduma said Ambience plans to work with the Cleveland Clinic while further developing its products.

Still, Kelly said he “would never want to let an agreement be the thing that keeps [Heidi’s] customers” adding it’s the health tech company’s responsibility to build an integrated product “that makes it really hard to swap” for another tool.

Where he could see companies running into problems, though, is when their value proposition is their integration with Epic. If Epic is cheaper, doesn’t require additional authentication, and there’s no data-sharing risk, “then it probably will be actually quite easy for [health systems] to swap,” Kelly said.

When contract renewals come around, he expects to see some lower prices because of this. There could also be more 

With this competitive market and increasingly high-tech products, Epic’s announcement "definitely will heat up the market,” Kelly said.

Robot typing text on a typewriter. Artificial intelligence generated text and future of journalism concept. Vector illustration.

How the AI scribe market may—or may not—shift following Epic’s AI tool launch 

AI scribe experts tell us they’re not too worried about Epic launching new AI tools. 

Welcome back to AI 411, a monthly roundup of artificial intelligence (AI) announcements from across the healthcare industry.

 its new suite of AI tools, including integrating an ambient scribe into the electronic health record (EHR) giant’s platform. (We spoke with other AI note-taking platform startups to see how they’re feeling about the news.)

Here’s the rundown of health-related AI updates from around the industry in August.

 Smart hospital company Artisight announced Aug. 28 it can now autonomously document operating room activity into EHRs. The tool could 

 on clinicians and also prompt staff to do things like scrub up when a patient enters the room. Similar tools from the company are already used in patient rooms at health systems like York, Pennsylvania-based WellSpan and Chicago-based Northwestern Medicine.

, or CHAI, a nonprofit dedicated to responsible AI use, and the National Association of Community Health Centers (NACHC), which represents community health centers (CHCs) serving nearly 34 million patients, announced a new partnership Aug. 14. Through policy, education, research, and integration support, the pair said in a release they hope to ensure CHCs and patients “are prioritized and positioned to benefit” from AI in healthcare.

On Aug. 20, EliseAI, which automates healthcare and housing systems like multistep scheduling and prior authorizations, announced a $250 million fundraiser. The Series E round was led by venture capitalist (VC) firm Andreessen Horowitz and had participation from other VCs, including Bessemer Venture Partners, Sapphire Ventures, and Navitas Capital. Funds will be used to hire more staff, according to a release. The company’s valuation has doubled to $2.2 billion since last August, 

Mental health company Grow Therapy announced a new AI feature on Aug. 19. Called “between-session reflections,” it provides a space for clients to write about their thoughts and feelings between appointments. The reflections are organized into themes they can access privately, and they can choose what they'd like to share with their provider.

Infrastructure software company Medallion, which uses AI to automate the back office of healthcare, announced $43 million in new funding on Aug. 18, bringing its total funding to $130 million. The round was led by VC Acrew Capital, and had participation from other firms, such as Washington Harbour Partners, Sequoia Capital, GV, Spark Capital, and NFDG. Among other things, the company is using the funding to launch CredAlliance, a tool designed to speed up and reduce administrative burden around the credentialing process for health insurance companies, according to a release.

On Aug. 13, EHR provider Oracle Health announced a “next-generation” platform, which includes implementing an AI agent to be an “intelligent clinical assistant” in the software, according to the company’s website. The tool automatically drafts clinical documentation, enables voice search, and provides patient summaries to help clinicians prepare their workflows.

assortment of various medical devices around a teal healthcare symbol

AI 411: August 2025

Your monthly roundup of AI announcements is here.

 are rapidly making their way into healthcare, Adrienne Kline is one researcher leading the way in development.

Head of AI and engineering at Illinois-based Northwestern Medicine’s Center for Artificial Intelligence at the Bluhm Cardiovascular Institute, Kline received $1.4 million in research grants in the last year from tech giants Amazon, Microsoft, and Nvidia to develop new AI technologies for both patients and healthcare administrators.

These companies regularly make investments into AI, with Microsoft pledging in January $80 billion to create AI-enabled data centers to train models and deploy AI and cloud-based applications globally.

With this funding, Kline spearheaded the creation of a medical deidentification tool called PixelGuard, which removes identifying data for safe sharing of sensitive health information. The assistant professor at Northwestern also led the creation of a multimodal cardiac platform, Project Corazon, which reads MRI slices to diagnose cardiac abnormalities.

Kline spoke with Healthcare Brew about how she’s used research grants over the past year and what she’s working on now.

While Kline was finishing her postdoctorate in the department of preventive medicine, private tech companies wanted to recruit her. She ultimately decided to stay in academia, though, because she wanted more autonomy to choose what she would work on, she said.

Since Kline was already on these companies’ radars, they contacted her with grant offers and gave her another way to collaborate on ideas she had. With National Institutes of Health funding cuts issued by the Trump administration (funding is 

 at Northwestern), industry financing provided her an opportunity to continue “really important” research, she said.

“Some of these industry partnerships are a bit of a godsend to continue to get your research pushed through and get your research published,” she said, which ultimately helps researchers try “to make a difference in the world.”

De-indeification tools, like the one she built, aren’t “sexy,” Kline said, but they are essential for protecting data sharing, including within hospitals or between industry partners.

PixelGuard does this for images, she said, meaning healthcare providers can safely share MRI slices externally, like, for example, with a health plan that wants to double-check it for billing purposes.

She worked with Amazon and IT company ScaleCapacity to develop the tool, which took about seven months from “pitch to deployment,” she said. 

Working with Nvidia, Amazon Web Services, and tech company Quantiphi, Kline also led the development of a program, Project Corazon that can read cardiac MRI scans and assess medical images “the way a radiologist would.” In other words, the tool uses AI to assist clinicians in making diagnoses.

Project Corazon is not ready to be used in hospitals yet, Kline said, but with the CDC reporting 6.7 million US residents over 20 as having heart failure, she sees the need to ease the diagnostic process.

After sitting in on a faculty meeting and hearing about scheduling challenges, Kline started drawing up plans for a new operating room scheduling tool.

When completed (she’s aiming for November or December), the tool will review items like surgeons, beds, and room availability as well as emergent patient needs to find the most efficient way to conduct procedures.

“We want high utilization, but we also want to ensure that our emergencies are handled without delay, taking into account people going on vacation or [getting] sick,” she said, “because that really ties back to hospital profit margins.”

Headshot of Northwestern's Adrienne Kline on a small blue cloud with darker blue stars nearby

This Northwestern researcher got $1.4m in grants from Amazon, Microsoft, Nvidia. Here’s what she built

Adrienne Kline made de-identification tool PixelGuard and heart diagnoser Project Corazon with private industry research money. 

 of using AI in the clinic, like cybersecurity risks and a 

 published Aug. 12 in the Lancet found another possible issue: clinicians losing their skills.

The research was conducted by hospital and academic researchers affiliated with institutions in Poland, Spain, Italy, Norway, Sweden, Japan, and the UK from Sept. 8, 2021, through March 9, 2022. It looked into how well endoscopists who “regularly used AI” performed colonoscopies when the technology was and was not available to them.

Researchers conducted the observational study of 1,443 patients at four endoscopy centers in Poland that began using AI tools to help detect polyps. They evaluated the quality of colonoscopies three months before and three months after implementing AI.

They found that when using AI tools, clinicians had lower adenoma detection rates (ADR), or a way to measure colonoscopy quality by detecting polyps, compared to when they performed the procedures without AI.

The study also found that the ADR of standard colonoscopy dropped from 28.4% (or 226 of 795 participants) before AI was in place to 22.4% (145 of 648 participants) after AI usage.

“Continuous exposure to AI might reduce the ADR of standard non-AI assisted colonoscopy, suggesting a negative effect on endoscopist behavior,” the researchers wrote.

This research coincides with other studies, including 

 that suggests AI usage weakens critical thinking skills.

Digitized looking pill capsule with small brains inside

Study: AI may weaken clinical skills 

The research aligns with other studies that suggest AI weakens critical thinking. 

Maia Anderson

 using artificial intelligence (AI) as part of their workflows, and while the technology can bring increased efficiency, it can also bring a greater risk of 

 the FDA’s AI tool, Elsa, which was made to accelerate drug and medical device approvals, generated fake research studies in its citations. And in early August, the Verge 

 Google’s healthcare AI model, Med-Gemini, mentioned a nonexistent body part in a 2024 research paper.

 from Mount Sinai health system in New York released on Aug. 2 found AI chatbots are “highly vulnerable” to attacks promoting false medical information.

Given the rising trend of AI use in healthcare, we asked executives from across the industry how leaders should respond when errors occur and what safeguards could be put in place to prevent harm.

Matthew DeCamp, an internal medicine physician and health services researcher at the University of Colorado’s Center for Bioethics and Humanities, told us there’s still “a lot of uncertainty” about who holds responsibility when AI errors occur in healthcare, as it’s not been 

However, the industry can look to existing protocols for sharing responsibility between different stakeholders, such as the AI developer and the end user, he said.

“Even before AI was involved, we found ways to divvy responsibility between, say, the maker of a CT scanner, the health system who purchased it, the radiologists who use it, and the primary care physicians like myself who might read the final report,” DeCamp said. “I think we can be reassured by this, and perhaps take a similar approach.”

Thomas Fuchs, SVP and chief AI officer at pharmaceutical company Eli Lilly, told us the priority for healthcare organizations using AI “should always be patient safety and maintaining trust.”

He echoed DeCamp’s sentiment that it’ll take a group effort to mitigate AI errors, adding that the technology developers should “design systems with rigorous validation, transparency, and continuous monitoring.”

Meanwhile, the organizations that use the technology should set up processes to “detect, assess, and respond” to errors quickly and adhere to guidelines like the National Institute of Standards and Technology’s AI Risk Management Framework, Fuchs said, which was designed by the government agency to help manage risks associated with using AI.

Ethan Berke, chief medical officer and SVP of integrated care at virtual care company Teladoc Health, said that in the same way healthcare organizations have systems in place to address other kinds of medical errors, they should create systems to “track, classify, and investigate” any safety threats that may arise from using AI. Then they can use that data to prevent potential future errors.

Teladoc has more than 60 proprietary AI models, Berke said, and the company has created a “rigorous” process to test and evaluate them for “accuracy, bias, and safety” before they’re deployed.

“We all have a responsibility to ensure that AI solutions are reliable, ethical, and secure,” he said. “By investing in patient safety and quality, we’ve been able to build a program that can analyze potential safety events more quickly, address root causes, and catch errors before they even reach the patient.”

A healthcare cross made up of glowing binary code with a warning sign over the top right corner.

Healthcare execs talk AI errors and how to address them

Everybody makes mistakes; everybody has those days. 

Artificial intelligence (AI) is now commonplace in healthcare. Robust rules for the tech? Not so much, according to a new report.

About 88% of 233 health system executives said their system uses AI in some form, but only 18% have a mature 

 and full AI strategy, according to an Aug. 12 

 from the professional membership organization Healthcare Financial Management Association (HFMA) and Eliciting Insights, a healthcare strategy and market research company.

“I think a lot of organizations need to step back and say, ‘Oh, wait, we have 10 departments doing work with AI,’” Richard Gundling, HFMA’s SVP of professional practice, told Healthcare Brew. “We need an overall governance structure now, so each department is using it the same way, the same oversights are done.”

Responses from 115 of the sampled healthcare system executives suggest the most common use of AI is for 

 and clinical notetaking, with 42% using it for this purpose. Coding is the second most common, at 28%, followed by clinical documentation improvement, denial prediction, and 

The good news: Though most surveyed healthcare orgs don’t have a comprehensive strategy, the report shows nearly three-quarters of CFO respondents said they are at least in the early stages of developing an AI governance structure—a set of rules and regulations around how departments should use AI.

But how should they go about creating one?

, AI could hurt rather than help the organization. There have been instances of AI 

 in the journal npj Digital Medicine suggests large language learning models may recommend “inferior” psychiatric treatments when told a patient’s race.

One important step in creating these guardrails is to make sure all departments are represented in crafting AI strategy, including financial executives, clinical leaders, and human resources professionals, Gundling said.

The most common duty of governance groups is to determine a data policy for AI, according to the report. The groups are also often used to identify and vet AI vendors.

There’s a growing number of resources to help groups create a governance structure.

The American Medical Association released an AI governance toolkit April 29 with an eight-step module highlighting steps such as designating an executive to be accountable for AI’s outcomes and establishing a vendor evaluation policy.

Other groups like the public-private Coalition for Health AI—a collaboration between providers, tech companies, and other health organizations—have 

 a robust AI implementation and assessment process to ensure AI is developed safely and companies 

.“Even if you feel like you’re behind the curve, you can still catch up,” Gundling said.

A medical cross made out of binary code held behind yellow and black safety guardrails.

AI