Evaluating the impact of retinal vessel segmentation metrics on retest reliability in a clinical setting: a comparative analysis using AutoMorph
Current research on artificial intelligence-based fundus photography biomarkers has demonstrated inconsistent results. Consequently, we aimed to evaluate and predict the test-retest reliability of retinal parameters extracted from fundus photography. Two groups of patients were recruited for the...
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Dokumenttyp: | Article (Journal) |
| Sprache: | Englisch |
| Veröffentlicht: |
November 2024
|
| In: |
Investigative ophthalmology & visual science
Year: 2024, Jahrgang: 65, Heft: 13, Pages: 1-14 |
| ISSN: | 1552-5783 |
| DOI: | 10.1167/iovs.65.13.24 |
| Online-Zugang: | Verlag, kostenfrei, Volltext: https://doi.org/10.1167/iovs.65.13.24
|
| Verfasserangaben: | Samuel D. Giesser, Ferhat Turgut, Amr Saad, Jay R. Zoellin, Chiara Sommer, Yukun Zhou, Siegfried K. Wagner, Pearse A. Keane, Matthias Becker, Delia Cabrera DeBuc, Gábor Márk Somfai |
MARC
| LEADER | 00000naa a2200000 c 4500 | ||
|---|---|---|---|
| 001 | 1948706512 | ||
| 003 | DE-627 | ||
| 005 | 20260114102910.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 260114s2024 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1167/iovs.65.13.24 |2 doi | |
| 035 | |a (DE-627)1948706512 | ||
| 035 | |a (DE-599)KXP1948706512 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 33 |2 sdnb | ||
| 100 | 1 | |a Giesser, Samuel David |e VerfasserIn |0 (DE-588)1373716681 |0 (DE-627)1933090294 |4 aut | |
| 245 | 1 | 0 | |a Evaluating the impact of retinal vessel segmentation metrics on retest reliability in a clinical setting |b a comparative analysis using AutoMorph |c Samuel D. Giesser, Ferhat Turgut, Amr Saad, Jay R. Zoellin, Chiara Sommer, Yukun Zhou, Siegfried K. Wagner, Pearse A. Keane, Matthias Becker, Delia Cabrera DeBuc, Gábor Márk Somfai |
| 264 | 1 | |c November 2024 | |
| 300 | |a 14 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Gesehen am 14.01.2026 | ||
| 520 | |a Current research on artificial intelligence-based fundus photography biomarkers has demonstrated inconsistent results. Consequently, we aimed to evaluate and predict the test-retest reliability of retinal parameters extracted from fundus photography. Two groups of patients were recruited for the study: an intervisit group (n = 28) to assess retest reliability over a period of 1 to 14 days and an intravisit group (n = 44) to evaluate retest reliability within a single session. Using AutoMorph, we generated test and retest vessel segmentation maps; measured segmentation map agreement via accuracy, sensitivity, F1 score and Jaccard index; and calculated 76 metrics from each fundus image. The retest reliability of each metric was analyzed in terms of the Spearman correlation coefficient, intraclass correlation coefficient (ICC), and relative percentage change. A linear model with the input variables contrast-to-noise-ratio and fractal dimension, chosen by a P-value-based backward selection process, was developed to predict the median percentage difference on retest per image based on image-quality metrics. This model was trained on the intravisit dataset and validated using the intervisit dataset. In the intervisit group, retest reliability varied between Spearman correlation coefficients of 0.34 and 0.99, ICC values of 0.31 to 0.99, and mean absolute percentage differences of 0.96% to 223.67%. Similarly, in the intravisit group, the retest reliability ranged from Spearman correlation coefficients of 0.55 and 0.96, ICC values of 0.40 to 0.97, and mean percentage differences of 0.49% to 371.23%. Segmentation map accuracy between test and retest never dropped below 97%; the mean F1 scores were 0.85 for the intravisit dataset and 0.82 for the intervisit dataset. The best retest was achieved with disc-width regarding the Spearman correlation coefficient in both datasets. In terms of the Spearman correlation coefficient, the worst retests of the intervisit and intravisit groups were tortuosity density and artery tortuosity density, respectively. The intravisit group exhibited better retest reliability than the intervisit group (P < 0.001). Our linear model, with the two independent variables contrast-to-noise ratio and fractal dimension predicted the median retest reliability per image on its validation dataset, the intervisit group, with an R2 of 0.53 (P < 0.001). Our findings highlight a considerable volatility in the reliability of some retinal biomarkers. Improving retest could allow disease progression modeling in smaller datasets or an individualized treatment approach. Image quality is moderately predictive of retest reliability, and further work is warranted to understand the reasons behind our observations better and thus ensure consistent retest results. | ||
| 700 | 1 | |a Turgut, Ferhat |e VerfasserIn |0 (DE-588)1309942323 |0 (DE-627)1870349725 |4 aut | |
| 700 | 1 | |a Saad, Amr |e VerfasserIn |0 (DE-588)1302315374 |0 (DE-627)1859333648 |4 aut | |
| 700 | 1 | |a Zoellin, Jay Rodney Toby |e VerfasserIn |0 (DE-588)137040879X |0 (DE-627)1929526202 |4 aut | |
| 700 | 1 | |a Sommer, Chiara |e VerfasserIn |4 aut | |
| 700 | 1 | |a Zhou, Yukun |e VerfasserIn |4 aut | |
| 700 | 1 | |a Wagner, Siegfried K. |e VerfasserIn |4 aut | |
| 700 | 1 | |a Keane, Pearse A. |e VerfasserIn |4 aut | |
| 700 | 1 | |a Becker, Matthias D. |e VerfasserIn |0 (DE-588)135571855 |0 (DE-627)56707255X |0 (DE-576)300521871 |4 aut | |
| 700 | 1 | |a DeBuc, Delia Cabrera |e VerfasserIn |4 aut | |
| 700 | 1 | |a Somfai, Gábor Márk |e VerfasserIn |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Investigative ophthalmology & visual science |d Rockville, Md. : ARVO, 1977 |g 65(2024), 13, Artikel-ID 24, Seite 1-14 |h Online-Ressource |w (DE-627)32048159X |w (DE-600)2009858-3 |w (DE-576)091138515 |x 1552-5783 |7 nnas |a Evaluating the impact of retinal vessel segmentation metrics on retest reliability in a clinical setting a comparative analysis using AutoMorph |
| 773 | 1 | 8 | |g volume:65 |g year:2024 |g number:13 |g elocationid:24 |g pages:1-14 |g extent:14 |a Evaluating the impact of retinal vessel segmentation metrics on retest reliability in a clinical setting a comparative analysis using AutoMorph |
| 856 | 4 | 0 | |u https://doi.org/10.1167/iovs.65.13.24 |x Verlag |x Resolving-System |z kostenfrei |3 Volltext |7 0 |
| 951 | |a AR | ||
| 992 | |a 20260114 | ||
| 993 | |a Article | ||
| 994 | |a 2024 | ||
| 998 | |g 135571855 |a Becker, Matthias D. |m 135571855:Becker, Matthias D. |d 50000 |e 50000PB135571855 |k 0/50000/ |p 9 | ||
| 999 | |a KXP-PPN1948706512 |e 4847733517 | ||
| BIB | |a Y | ||
| SER | |a journal | ||
| JSO | |a {"name":{"displayForm":["Samuel D. Giesser, Ferhat Turgut, Amr Saad, Jay R. Zoellin, Chiara Sommer, Yukun Zhou, Siegfried K. Wagner, Pearse A. Keane, Matthias Becker, Delia Cabrera DeBuc, Gábor Márk Somfai"]},"recId":"1948706512","id":{"doi":["10.1167/iovs.65.13.24"],"eki":["1948706512"]},"origin":[{"dateIssuedKey":"2024","dateIssuedDisp":"November 2024"}],"type":{"bibl":"article-journal","media":"Online-Ressource"},"language":["eng"],"physDesc":[{"extent":"14 S."}],"person":[{"role":"aut","given":"Samuel David","family":"Giesser","display":"Giesser, Samuel David"},{"display":"Turgut, Ferhat","family":"Turgut","role":"aut","given":"Ferhat"},{"family":"Saad","role":"aut","given":"Amr","display":"Saad, Amr"},{"display":"Zoellin, Jay Rodney Toby","family":"Zoellin","given":"Jay Rodney Toby","role":"aut"},{"display":"Sommer, Chiara","family":"Sommer","role":"aut","given":"Chiara"},{"family":"Zhou","given":"Yukun","role":"aut","display":"Zhou, Yukun"},{"role":"aut","given":"Siegfried K.","family":"Wagner","display":"Wagner, Siegfried K."},{"given":"Pearse A.","role":"aut","family":"Keane","display":"Keane, Pearse A."},{"display":"Becker, Matthias D.","family":"Becker","given":"Matthias D.","role":"aut"},{"given":"Delia Cabrera","role":"aut","family":"DeBuc","display":"DeBuc, Delia Cabrera"},{"display":"Somfai, Gábor Márk","family":"Somfai","role":"aut","given":"Gábor Márk"}],"title":[{"title_sort":"Evaluating the impact of retinal vessel segmentation metrics on retest reliability in a clinical setting","title":"Evaluating the impact of retinal vessel segmentation metrics on retest reliability in a clinical setting","subtitle":"a comparative analysis using AutoMorph"}],"relHost":[{"part":{"text":"65(2024), 13, Artikel-ID 24, Seite 1-14","extent":"14","volume":"65","pages":"1-14","issue":"13","year":"2024"},"pubHistory":["16.1977 -"],"titleAlt":[{"title":"IOVS"}],"note":["Gesehen am 01.08.2016"],"disp":"Evaluating the impact of retinal vessel segmentation metrics on retest reliability in a clinical setting a comparative analysis using AutoMorphInvestigative ophthalmology & visual science","recId":"32048159X","origin":[{"dateIssuedDisp":"1977-","dateIssuedKey":"1977","publisher":"ARVO ; [Verlag nicht ermittelbar]","publisherPlace":"Rockville, Md. ; St. Louis, Mo."}],"id":{"zdb":["2009858-3"],"issn":["1552-5783"],"eki":["32048159X"]},"physDesc":[{"extent":"Online-Ressource"}],"language":["eng"],"corporate":[{"role":"isb","display":"Association for Research in Vision and Ophthalmology"}],"type":{"media":"Online-Ressource","bibl":"periodical"},"title":[{"title":"Investigative ophthalmology & visual science","subtitle":"IOVS ; official journal of the Association for Research in Vision and Ophthalmology","title_sort":"Investigative ophthalmology & visual science"}]}],"note":["Gesehen am 14.01.2026"]} | ||
| SRT | |a GIESSERSAMEVALUATING2024 | ||