ChatGPT also (barely) passed the tough US medical licensing exams

by

ChatGPT has taken the world by storm and aspires to be a more sophisticated and creative version of Google’s search engine (that’s what Google is worried about).

The increasingly popular worldwide – and in Greece – creative artificial intelligence tool ChatGPT of the American company Open AI managed to achieve another milestone: passing the highly demanding exams for a doctor’s license in the USA.

A candidate can pass these tests only if several years of medical training and practice have preceded it.

ChatGPT has taken the world by storm and aspires to be a more sophisticated and creative version of Google’s search engine (which is why Google is worried).

It is an “intelligent” system known as a Large Language Model (LLM), designed to take advantage of information from the Internet and in a few seconds write like a human various texts based on the needs of the user, from scientific articles to poems.

Medical technology company researchers Ansible Healthwho made the relevant publication in the digital health journal PLoS Digital Health, tested whether ChatGPT can answer the approximately 350 questions of the three USMLE (United States Medical Licensing Exam) tests.

These exams assess medical knowledge in a wide range, from biochemistry to diagnostic competence and bioethics.

ChatGPT achieved a score of 52.4% to 70%, when the threshold for a doctor to pass is around 60%.

In fact, in 89% of the questions the “smart” system produced at least one important inspired answer, which was original, non-obvious and clinically valid.

ChatGPT also outperformed PudMedGPT, a similar “smart” model trained exclusively in the biomedical domain, which did not score above 51%.

“Achieving a score above the passing threshold on these notoriously difficult specialized medical tests, without the slightest human assistance, marks a remarkable milestone in the maturation of clinical AI,” the researchers noted. In fact, they made it known that ChatGPT itself helped them significantly in writing their scientific publication.

As they said, “We interacted with ChatGPT as if he were a colleague, asking him to synthesize, simplify, and provide arguments for our draft study. All the co-authors greatly appreciated the contribution of ChatGPT.”

RES-EMP

You May Also Like

Recommended for you

Immediate Peak