Artificial intelligence unable to predict true penicillin allergy
Published: January 16, 2024
The penicillin allergy label is the most commonly reported drug allergy label in the US; however, less than 5% of these patients with a penicillin allergy label are truly allergic to penicillin after evaluation. Prior international studies empirically determined that having had a recent allergic reaction (<5 years) and the presence of anaphylaxis were factors that predict penicillin allergy. However, these prediction models were developed using non-US data.
In a recent study in The Journal of Allergy and Clinical Immunology: In Practice, Gonzalez-Estrada et al. developed a machine learning penicillin allergy prediction model from retrospective multi-site data between 2013 and 2020. Data were grouped into 4 datasets: enriched training (1:3 case-control matched cohort), enriched testing, nonenriched internal testing, and nonenriched external testing. Machine learning algorithms were used for model development in the enriched training dataset and then tested in the other datasets. The authors determined area under the curve and applied the Shapley Additive exPlanations framework to interpret risk drivers.
A total of 4,777 patients with a penicillin allergy label were included in the study. The gradient-boosted model, one out of 2,500 distinct machine learning and artificial intelligence models tested, was the strongest model to predict penicillin allergy with an area under the curve of 0.67 (95% confidence interval: 0.57-0.77). Top drivers for positive penicillin skin testing included penicillin allergy labels that had occurred within <1 year of evaluation and those reactions requiring medical attention (defined as calling a health care provider, visiting an urgent care or emergency department, or requiring hospitalization), as well as female sex and reaction of hives/urticaria. Negative drivers included reactions that occur more than 30 years ago and a delayed reaction onset (a reaction occurring >1 hour after initial exposure). Anaphylaxis was not an important predictor in this study, but was uncommon overall, reported by only 2% of patients. The authors concluded that the machine learning model was unable to accurately predict penicillin allergy. Despite this, they were able to identify new drivers for positive penicillin allergy skin testing.
The Journal of Allergy and Clinical Immunology: In Practice is an official journal of the AAAAI, focusing on practical information for the practicing clinician.
Full Article