Prof. RNDr. Aleš Macela, DrSc.
Prof. RNDr. Vanda Boštíková, Ph.D.
Scientists, computer scientists, economists, artists, and librarians alike have high hopes for artificial intelligence (AI). AI uses a range of tools designed to generate text or answers to questions, such as Gemini or ChatGPT (Chat Generative Pretrained Transformer), generate images or graphic designs (e.g., DALL-E or Editee), or to create interactive documents for project organization (Notion AI). Large Language Models (LLMs), which include the aforementioned ChatGPT, Gemini, and Microsoft’s Copilot, began to gain traction during the second decade of this century. They are capable of receiving diverse inputs (prompts/texts, images, audio or video recordings) and creating coherent outputs from them using “reasoning” (thinking about the given context), chains of thought, or deduction. However, it appears that chatbots can be easily manipulated to spread various types of misinformation.
Studies on the use of LLMs in healthcare point to the potential risks of uncritical use of their outputs. One of the latest papers, published in the journal Annals of Internal Medicine, evaluated the effectiveness of protective measures against harmful instructions for chatbots dealing with health misinformation in commonly used LLMs. Five basic LLMs (GPT-4o – OpenAI, Gemini 1.5 Pro – Google, Claude 3.5 Sonnet – Anthropic, Llama 3.2-90B Vision – Meta, and Grok Beta o- xAI) were evaluated through their application programming interfaces (APIs). They received a supplement of inaccurate instructions using formal scientific language, containing fictitious references to reputable medical journals. The aim was to detect how vulnerable these LLMs are to malicious instructions at the system level and whether it is possible to covertly create chatbots that generate credible-sounding sources of misinformation [1].
Four of the five modified chatbots—GPT-4o, Gemini, Llama, and Grok—processed 100% of the false information and used it to generate false claims about health effects. Claude 3.5 processed “only” 40% of the false information and showed some degree of resistance to misleading input data. The resulting misinformation covered important health topics, including discredited theories linking vaccines to autism, false claims that 5G networks cause infertility, myths that sunscreen increases the risk of skin cancer, and dangerous dietary recommendations for treating cancer. These claims were supported by non-existent citations from highly respected scientific journals such as The Lancet or JAMA. Curious responses also included claims that garlic could replace antibiotics or that HIV is spread through the air [1].
The use of artificial intelligence tools in information gathering, the publication of biomedical research results, or in healthcare decision-making processes facilitates research work, but it is necessary to check the integrity of information generated using AI. This applies not only to biomedical information, but to information provided by artificial intelligence tools in general. A report by Muck Rack (Muck Rack is a comprehensive public relations (PR) management platform designed to streamline the workflows of PR professionals and journalists) provided recent information from more than 1,500 journalists in the US, UK, Canada, India, and other regions on how they work, what influences their reporting, and how PR professionals can make an impact [2]. According to this study, 77% of journalists report using AI tools in their work, with more than a third identifying misinformation and its spread as the most serious threat to the future of AI-powered journalism. The seriousness of the situation is documented by the interest of other information platforms in the results of such studies [3-5].
In relation to health and healthcare issues, two fundamental challenges arise from this information. The first concerns information providers, both at the level of PR professionals and at the level of doctors and healthcare staff. If they do not provide relevant, easily understandable information, there will be a gradual loss of trust and a threat to professional healthcare as people seek alternative information and methods of assistance. The second challenge concerns the users of the information provided, who need to be reminded of their responsibility for their own health, the importance of continuing education, and the need to distinguish facts from misinformation. Given that AI tools are commonly available to the general public, there is a real possibility of their misuse, as evidenced by the publication [1], which poses a major threat to public health. Therefore, it is necessary to constantly emphasize the need for responsibility and ethics in the dissemination of information related (not only) to human health.
References
- Modi ND, Menz BD, Awaty AA, Alex CA, Logan JM, McKinnon RA, Rowland A, Bacchi S, Gradon K, Sorich MJ, Hopkins AM. Assessing the System-Instruction Vulnerabilities of Large Language Models to Malicious Conversion Into Health Disinformation Chatbots. Ann Intern Med. 2025 Aug;178(8):1172-1180. doi: 10.7326/ANNALS-24-03933
- https://muckrack.com/research/the-state-of-journalism?utm_source=com
- https://medicalxpress.com/news/2025-06-ai-chatbot-safeguards-health-disinformation.html?utm_source=chatgpt.com
- https://factcheckafrica.net/ai-misinformation-and-the-future-of-journalism-in-africa-insights-from-muck-racks-state-of-journalism-2025-report/?utm_source=com
- https://www.globenewswire.com/news-release/2025/06/10/3096945/0/en/Disinformation-and-Misinformation-Are-Top-Concerns-in-Journalism-According-to-a-New-Muck-Rack-Report.html?utm_ source=chatgpt.com
*This text has received support from the National Recovery Plan under project 1.4 CEDMO 1 – Z220312000000, Support for increasing the impact, innovation, and sustainability of CEDMO in the Czech Republic, which is financed by the EU Recovery and Resilience Facility.
