Detection of suicidality from medical text using privacy-preserving large language models: feature
BackgroundAttempts to use artificial intelligence (AI) in psychiatric disorders show moderate success, highlighting the potential of incorporating information from clinical assessments to improve the models. This study focuses on using large language models (LLMs) to detect suicide risk from medical...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article (Journal) |
| Language: | English |
| Published: |
05 November 2024
|
| In: |
The British journal of psychiatry
Year: 2024, Volume: 225, Issue: 6, Pages: 532-537 |
| ISSN: | 1472-1465 |
| DOI: | 10.1192/bjp.2024.134 |
| Online Access: | Verlag, kostenfrei, Volltext: https://doi.org/10.1192/bjp.2024.134 Verlag, kostenfrei, Volltext: http://www.cambridge.org/core/journals/the-british-journal-of-psychiatry/article/detection-of-suicidality-from-medical-text-using-privacypreserving-large-language-models/75E6B08AECDF68443C2594F421805FD9# 
|
| Author Notes: | Isabella Catharina Wiest, Falk Gerrik Verhees, Dyke Ferber, Jiefu Zhu, Michael Bauer, Ute Lewitzka, Andrea Pfennig, Pavol Mikolas and Jakob Nikolas Kather |
| Summary: | BackgroundAttempts to use artificial intelligence (AI) in psychiatric disorders show moderate success, highlighting the potential of incorporating information from clinical assessments to improve the models. This study focuses on using large language models (LLMs) to detect suicide risk from medical text in psychiatric care.AimsTo extract information about suicidality status from the admission notes in electronic health records (EHRs) using privacy-sensitive, locally hosted LLMs, specifically evaluating the efficacy of Llama-2 models.MethodWe compared the performance of several variants of the open source LLM Llama-2 in extracting suicidality status from 100 psychiatric reports against a ground truth defined by human experts, assessing accuracy, sensitivity, specificity and F1 score across different prompting strategies.ResultsA German fine-tuned Llama-2 model showed the highest accuracy (87.5%), sensitivity (83.0%) and specificity (91.8%) in identifying suicidality, with significant improvements in sensitivity and specificity across various prompt designs.ConclusionsThe study demonstrates the capability of LLMs, particularly Llama-2, in accurately extracting information on suicidality from psychiatric records while preserving data privacy. This suggests their application in surveillance systems for psychiatric emergencies and improving the clinical management of suicidality by improving systematic quality control and research. |
|---|---|
| Item Description: | Gesehen am 29.04.2025 |
| Physical Description: | Online Resource |
| ISSN: | 1472-1465 |
| DOI: | 10.1192/bjp.2024.134 |