Artificial Intelligence as a Medical Assistant

posted  9 Jan 2024
Photo - Artificial Intelligence as a Medical Assistant
A team of medical researchers at Mount Sinai conducted a study on AI chatbots, evaluating their effectiveness as autonomous practitioners of evidence-based medicine.

The experiment featured AI models such as Gemini Pro, ChatGPT 3.5, ChatGPT 4, Mixtral 8x7B, and LLaMA v2. These bots took on the role of a medical professor, adhering to medical protocols and prescribing treatment plans based on analytical results.

Findings indicated that ChatGPT 4 was the most successful, reaching a 74% accuracy rate.
Experiment Workflow Source: arxiv.org

Experiment Workflow Source: arxiv.org

However, the limitations of large language models still hinder their comprehensive utilization, due to persistent issues with "hallucinations" and the potential for providing inaccurate or misleading information.