AI Clears U.S. Medical Licensing Exam

According to two recent articles, two artificial intelligence (AI) algorithms, including ChatGPT, have passed the U.S. Medical Licensing Examination (USMLE). The papers discussed several strategies for employing large language models for taking the USMLE, which consists of the Step 1, Step 2 CK, and Step 3 tests. An artificial intelligence (AI) search engine called ChatGPTopens runs in a new tab or window and mimics long-form writing depending on user cues. It was created by OpenAI, and it gained popularity after multiple social media articles outlined potential applications for the technology in healthcare settings, frequently with unsatisfactory outcomes. The first study examined ChatGPT's performance on the USMLE without any extra training or reinforcement before the examinations, and it was released on medRxivopens in a new tab or window in December. The findings revealed "new and startling evidence" that this AI technology was capable of meeting the challenge, according to Victor Tseng, MD, of Ansible Health in Mountain View, California, and colleagues. ChatGPT was able to perform at >50% accuracy on all of the assessments, and even reached 60% in most of the analyses, according to Tseng and company. Although the USMLE passing score varies from year to year, according to the authors, passing is often around 60%. "ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement," they wrote, noting that the tool was able to demonstrate "a high level of concordance and insight in its explanations." "These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making," they concluded.