Validating large language models against manual information extraction from case reports of drug-induced parkinsonism in patients with schizophrenia spectrum and mood disorders: a proof of concept study
In this proof of concept study, we demonstrated how Large Language Models (LLMs) can automate the conversion of unstructured case reports into clinical ratings. By leveraging instructions from a standardized clinical rating scale and evaluating the LLM’s confidence in its outputs, we aimed to refine...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article (Journal) |
| Language: | English |
| Published: |
20 March 2025
|
| In: |
Schizophrenia
Year: 2025, Volume: 11, Issue: 1, Pages: 1-4 |
| ISSN: | 2754-6993 |
| DOI: | 10.1038/s41537-025-00601-5 |
| Online Access: | Verlag, lizenzpflichtig, Volltext: https://doi.org/10.1038/s41537-025-00601-5 Verlag, lizenzpflichtig, Volltext: https://www.nature.com/articles/s41537-025-00601-5 
|
| Author Notes: | Sebastian Volkmer, Alina Glück, Andreas Meyer-Lindenberg, Emanuel Schwarz and Dusan Hirjak |
MARC
| LEADER | 00000naa a2200000 c 4500 | ||
|---|---|---|---|
| 001 | 192829474X | ||
| 003 | DE-627 | ||
| 005 | 20250616152839.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 250616s2025 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1038/s41537-025-00601-5 |2 doi | |
| 035 | |a (DE-627)192829474X | ||
| 035 | |a (DE-599)KXP192829474X | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 11 |2 sdnb | ||
| 100 | 1 | |a Volkmer, Sebastian |e VerfasserIn |0 (DE-588)1317351657 |0 (DE-627)187935246X |4 aut | |
| 245 | 1 | 0 | |a Validating large language models against manual information extraction from case reports of drug-induced parkinsonism in patients with schizophrenia spectrum and mood disorders |b a proof of concept study |c Sebastian Volkmer, Alina Glück, Andreas Meyer-Lindenberg, Emanuel Schwarz and Dusan Hirjak |
| 264 | 1 | |c 20 March 2025 | |
| 300 | |b Illustrationen | ||
| 300 | |a 4 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Gesehen am 16.06.2025 | ||
| 520 | |a In this proof of concept study, we demonstrated how Large Language Models (LLMs) can automate the conversion of unstructured case reports into clinical ratings. By leveraging instructions from a standardized clinical rating scale and evaluating the LLM’s confidence in its outputs, we aimed to refine prompting strategies and enhance reproducibility. Using this strategy and case reports of drug-induced Parkinsonism, we showed that LLM-extracted data closely align with clinical rater manual extraction, achieving an accuracy of 90%. | ||
| 650 | 4 | |a Biomarkers | |
| 650 | 4 | |a Neuroscience | |
| 700 | 1 | |a Glück, Alina |d 1998- |e VerfasserIn |0 (DE-588)1368825184 |0 (DE-627)1928295266 |4 aut | |
| 700 | 1 | |a Meyer-Lindenberg, Andreas |d 1965- |e VerfasserIn |0 (DE-588)1029137390 |0 (DE-627)732483069 |0 (DE-576)376589876 |4 aut | |
| 700 | 1 | |a Schwarz, Emanuel |e VerfasserIn |0 (DE-588)1055051260 |0 (DE-627)792581040 |0 (DE-576)411121596 |4 aut | |
| 700 | 1 | |a Hirjak, Dusan |d 1985- |e VerfasserIn |0 (DE-588)1024129306 |0 (DE-627)719194717 |0 (DE-576)367982463 |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Schizophrenia |d [London] : Nature Publishing Group, 2022 |g 11(2025), 1, Artikel-ID 47, Seite 1-4 |h Online-Ressource |w (DE-627)1814996451 |w (DE-600)3133210-9 |x 2754-6993 |7 nnas |a Validating large language models against manual information extraction from case reports of drug-induced parkinsonism in patients with schizophrenia spectrum and mood disorders a proof of concept study |
| 773 | 1 | 8 | |g volume:11 |g year:2025 |g number:1 |g elocationid:47 |g pages:1-4 |g extent:4 |a Validating large language models against manual information extraction from case reports of drug-induced parkinsonism in patients with schizophrenia spectrum and mood disorders a proof of concept study |
| 856 | 4 | 0 | |u https://doi.org/10.1038/s41537-025-00601-5 |x Verlag |x Resolving-System |z lizenzpflichtig |3 Volltext |
| 856 | 4 | 0 | |u https://www.nature.com/articles/s41537-025-00601-5 |x Verlag |z lizenzpflichtig |3 Volltext |
| 951 | |a AR | ||
| 992 | |a 20250616 | ||
| 993 | |a Article | ||
| 994 | |a 2025 | ||
| 998 | |g 1024129306 |a Hirjak, Dusan |m 1024129306:Hirjak, Dusan |d 60000 |e 60000PH1024129306 |k 0/60000/ |p 5 |y j | ||
| 998 | |g 1055051260 |a Schwarz, Emanuel |m 1055051260:Schwarz, Emanuel |d 60000 |e 60000PS1055051260 |k 0/60000/ |p 4 | ||
| 998 | |g 1029137390 |a Meyer-Lindenberg, Andreas |m 1029137390:Meyer-Lindenberg, Andreas |d 60000 |e 60000PM1029137390 |k 0/60000/ |p 3 | ||
| 998 | |g 1368825184 |a Glück, Alina |m 1368825184:Glück, Alina |d 60000 |e 60000PG1368825184 |k 0/60000/ |p 2 | ||
| 999 | |a KXP-PPN192829474X |e 4734646155 | ||
| BIB | |a Y | ||
| SER | |a journal | ||
| JSO | |a {"id":{"eki":["192829474X"],"doi":["10.1038/s41537-025-00601-5"]},"note":["Gesehen am 16.06.2025"],"name":{"displayForm":["Sebastian Volkmer, Alina Glück, Andreas Meyer-Lindenberg, Emanuel Schwarz and Dusan Hirjak"]},"person":[{"family":"Volkmer","role":"aut","display":"Volkmer, Sebastian","given":"Sebastian"},{"role":"aut","family":"Glück","display":"Glück, Alina","given":"Alina"},{"given":"Andreas","display":"Meyer-Lindenberg, Andreas","family":"Meyer-Lindenberg","role":"aut"},{"family":"Schwarz","role":"aut","given":"Emanuel","display":"Schwarz, Emanuel"},{"given":"Dusan","display":"Hirjak, Dusan","role":"aut","family":"Hirjak"}],"title":[{"subtitle":"a proof of concept study","title_sort":"Validating large language models against manual information extraction from case reports of drug-induced parkinsonism in patients with schizophrenia spectrum and mood disorders","title":"Validating large language models against manual information extraction from case reports of drug-induced parkinsonism in patients with schizophrenia spectrum and mood disorders"}],"language":["eng"],"origin":[{"dateIssuedKey":"2025","dateIssuedDisp":"20 March 2025"}],"relHost":[{"language":["eng"],"name":{"displayForm":["published in partnership with Schizophrenia International Research Society"]},"disp":"Validating large language models against manual information extraction from case reports of drug-induced parkinsonism in patients with schizophrenia spectrum and mood disorders a proof of concept studySchizophrenia","part":{"text":"11(2025), 1, Artikel-ID 47, Seite 1-4","issue":"1","extent":"4","volume":"11","pages":"1-4","year":"2025"},"note":["Gesehen am 26.08.2022"],"title":[{"title_sort":"Schizophrenia","title":"Schizophrenia"}],"origin":[{"publisherPlace":"[London]","dateIssuedDisp":"[2022]-","publisher":"Nature Publishing Group"}],"id":{"eki":["1814996451"],"issn":["2754-6993"],"zdb":["3133210-9"]},"physDesc":[{"extent":"Online-Ressource"}],"type":{"bibl":"periodical","media":"Online-Ressource"},"recId":"1814996451","pubHistory":["8, article number 1 (2022)-"]}],"recId":"192829474X","type":{"media":"Online-Ressource","bibl":"article-journal"},"physDesc":[{"noteIll":"Illustrationen","extent":"4 S."}]} | ||
| SRT | |a VOLKMERSEBVALIDATING2020 | ||