Google's AI Chatbot Clears US Medical Licensing Exam, But Still Not Ready for Human Patients

(Image credit- According to a peer-reviewed study released on Wednesday, Google's AI health chatbot passed a US medical licensing exam. According to the study, even if the chatbot's performance is commendable, it still lacks the knowledge of actual doctors. In 2022, OpenAI unveiled ChatGPT, an AI language model that ignited rivalry among the world's largest tech companies. The application of AI in healthcare has already shown substantial progress, with algorithms capable of reading particular medical scans with comparable accuracy to human experts, despite the potential and drawbacks of the technology being publicly addressed. [caption id="" align="aligncenter" width="1200"] Image credit- News18[/caption]

Med-PaLM-A  Google AI health chatbot

Google unveiled Med-PaLM, an AI-based tool for answering medical questions, in a preprint paper published in December. Med-PaLM has not been made accessible to the general public like ChatGPT has. The first major language model to pass the US Medical Licensing Examination (USMLE), according to Google, is Med-PaLM. This is a remarkable accomplishment. In the United States, medical students and doctors-in-training who take the USMLE must have a passing score of roughly 60%. An earlier study last year showed that ChatGPT had passing or nearly passing results on the test. Google researchers also discovered that Med-PaLM received a score of 67.6% on multiple-choice questions in the USMLE format. Although the performance of Med-PaLM is impressive, the study noted that it still falls short of that of physicians. To deal with the problem of "hallucinations," which happen when AI models give false information, Google created a new evaluation standard. Also read: Meta poised to launch commercial AI model to rival OpenAI & Google According to a preprint study published in May, a newer version of the model known as Med-PaLM 2 outperformed the older one by about 20%, scoring 86.5 percent on the USMLE exam. [caption id="" align="aligncenter" width="1200"] Image credit- The Economic Times[/caption]

Assistant Not Decision Makers

Experts outside the research emphasized the distinction between answering medical questions and performing actual medicine, which entails diagnosing and treating actual medical illnesses. They issued a warning, saying that AI-powered chatbots should only be used as assistants rather than making final decisions. Google researcher Karan Singhal suggested using Med-PaLM in the future to give alternate possibilities that doctors might not have thought of. Singhal did not specify any specific collaborations, but he did clarify that no testing will include patient risk or direct clinical treatment. The emphasis would be on automating low-risk administrative tasks instead. The renowned Mayo Clinic research center in the US has been evaluating Med-PaLM 2 since April, according to the reports. Singhal, who emphasized that the testing would concentrate on administrative duties that are easily automatable rather than direct patient care, opted not to share particular partnership information.