Artificial Intelligence-Generated Scientific Literature May Contain Fabricated Sources and Incorrect Information Study Finds
November 14, 2023
Contact:
Candace Archie, Communications & Public Relations Manager
carchie@aaaai.org
(414) 272-6071
MILWAUKEE - Artificial intelligence has the potential to create scientific medical literature and reviews in the future, but current AI technology has been shown to produce inaccurate information with fabricated references according to new research from The Journal of Allergy and Clinical Immunology: In Practice (JACI: In Practice), an official journal of the American Academy of Allergy, Asthma & Immunology (AAAAI).
Researchers at Penn State College of Medicine tasked a chatbot, generative pre-trained transformer (GPT-4), with a series of complex prompts to create two 1,000-word scientific mini-review articles on the topics of hereditary angioedema (HAE) and eosinophilic esophagitis (EoE), with the intention of evaluating the factual accuracy of output, appropriateness of language, and contents included in the AI-created review articles. Upon examination, the mini-reviews were found to include some cases of potential plagiarism, inaccurate information and fabricated references, suggesting that the AI-generated content lacked depth and did not appear to be the result of an analytical process.
Researchers critically appraised the chatbot min-review papers using a modified Joanna Briggs Institute (JBI) assessment tool and found that the language and structure of the reviews appeared clearly presented and articulated, however, the AI chatbot failed to fully utilize the 1,000-word prompt limit, creating reviews with only 653 words and 805 words. Similarly, despite being provided instruction to utilize scientific references, more than 40% of cited sources were fabricated. Forty-six percent of the articles cited for the HAE mini-review and 47% of those cited for the EoE mini-review did not exist. Of the remaining references, 31% were real but did not include the information that was cited by the AI chatbot. Of the legitimate references, 85% were freely available and only two required a subscription, suggesting bias in the completed reviews.
According to the study, “The basis of a strong scientific review article is the quality of sources used to aggregate data, and therefore this is a glaring fault and shortcoming of the AI which makes it an unreliable source for review article generation.”
Plagiarism was another area of concern for researchers. Though the two mini-reviews in the study passed grammar checks, plagiarism checks identified 16% plagiarism in the HAE review article, and 24% in the EoE review article. Upon careful examination of the plagiarized sentences, the plagiarism was found primarily in reference to well-established information and therefore, may not represent an actual case of plagiarism. Additional research is needed to determine AI capabilities surrounding the creation of original articles.
“The ability of artificial intelligence to synthesize and summarize medical literature holds the potential for changing the landscape of scientific writing. However, at its current nascent stage, it carries a risk of distributing fabricated information and can very well overlook critical information necessary for the readers of medical journals especially when addressing specialized topics such as allergic and immunologic disorders,” says Taha Al-Shaikhly, MBChB, FAAAAI, corresponding author for the study.
While there is limited research on this topic, there is concern that AI could be used to generate false research in a convincing manner with nonexistent data. This research is a necessary step in better understanding new tools in scientific research and can hopefully inform future efforts to distinguish between artificially generated and human literature. Additional, expanded research will be valuable to understand and utilize AI in scientific literature according to the study.
Read the full study.
The American Academy of Allergy, Asthma & Immunology (AAAAI) is the leading membership organization of more than 7,100 allergists, asthma specialists, clinical immunologists, allied health professionals and other professionals with a special interest in the research and treatment of allergic and immunologic diseases. Established in 1943, the AAAAI has more than 7,100 members in the United States, Canada and 72 other countries and is the go-to resource for patients living with allergies, asthma and immune deficiency disorders.