Artificial intelligence may be transforming how people access medical information, but a new study suggests its drug recommendations could come with a hefty price tag. Researchers have found that popular AI chatbots tend to favour certain medications over equally suitable alternatives, potentially adding millions of dollars to healthcare costs. The findings raise concerns about whether AI systems are giving patients a balanced picture of available treatments or simply recommending drugs with the strongest online presence.

The study, led by Professor Daiju Ueda of Osaka Metropolitan University’s Graduate School of Medicine in collaboration with researchers at the University of Texas MD Anderson Cancer Center, examined how five large language models (LLMs) selected prescription medications. The researchers tested GPT-5.4, Claude Sonnet 4.6, Gemini 3.1 Flash, Llama 4 and MedGemma, asking each model to recommend up to three FDA-approved drug products across a series of simulated medical scenarios.

The results revealed a striking lack of diversity in the recommendations. Although an average of 5.5 FDA-approved products were available for each specified active ingredient, just one product dominated the responses, accounting for an average of 97.5% of first-choice recommendations across the scenarios and models tested. This suggested that AI systems were repeatedly steering users towards a narrow selection of medications despite the availability of alternatives.

“The recommendations were mostly for brand-name drugs, those highly visible in Google searches, and treatments with dedicated Wikipedia pages,” Professor Ueda explained. The models were also less likely to recommend drugs approved more recently by the FDA or products commonly prescribed by physicians in everyday clinical practice. In some cases, the AI systems even suggested discontinued medications or provided recommendations that conflicted with regulatory information, highlighting potential problems beyond simple brand preference.

To understand the financial implications, the researchers examined what might happen if American physicians followed the AI-generated recommendations when prescribing medications. Using US data, they estimated that annual drug acquisition costs could rise by approximately US$55.6 million under such a scenario. “This represents a 1,019% increase in the cost of purchasing these prescriptions,” Professor Ueda said. The figures illustrate how seemingly minor biases in automated recommendations could translate into substantial healthcare spending if adopted widely.

The researchers also investigated whether clearer instructions could improve the models’ performance. When specifically asked to prioritise lower-cost medications or generic alternatives, the AI systems adjusted their recommendations, demonstrating that their choices could be influenced by how questions were phrased. Nevertheless, the responses continued to favour a relatively small group of products, suggesting that prompting alone may not fully resolve the underlying selection bias.

The findings underscore the importance of caution as AI becomes an increasingly familiar source of medical advice. While chatbots can make treatment information more accessible, their recommendations may not reflect the full range of available, affordable or currently approved options. “Combining AI systems with trusted drug information sources and human oversight may help support more appropriate medication choices,” Professor Ueda said, emphasising the need for stronger safeguards before AI-generated recommendations play a greater role in prescribing decisions.

More information: Junya Sato et al, Large language models show product name convergence in drug recommendations, Nature Health. DOI: 10.1038/s44360-026-00205-y

Journal information: Nature Health Provided by Osaka Metropolitan University