It’s been more than two months since OpenAI launched ChatGPT Health, and the results are in. They’re…not great. A Feb. 23 study in Nature Medicine found the patient-facing chatbot didn’t tell users to go to the emergency room (ER) in 52% of cases that clinicians felt required emergency attention. The study is described by its authors as the first independent evaluation of the patient-facing chatbot search tool since its Jan. 7 launch. A previous study from the UK with a different design found regular (non-health-focused) large language models triaged users incorrectly the majority of the time, too. For what it’s worth, OpenAI said in a news release ChatGPT Health “isn’t intended for diagnosis or treatment.” But Girish Nadkarni, the study’s senior author and chief AI officer at New York’s Mount Sinai Health System, told Healthcare Brew “the need for these tools is real” amid limited healthcare access. And even though AI isn’t meant to be used this way, it realistically will be, he added. For more on the study, keep reading.—CC |