Deep learning-based quality control using subcellular RNA spatial distribution patterns for cell segmentation in spatial transcriptomics data
Sequencing-based spatial transcriptomics (sST) techniques with high resolution enable transcriptome-wide RNA capture at subcellular resolution. Although new cell segmentation methods for sST data are continually being developed, accurately assigning RNA spots to corresponding cells still presents si...
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Dokumenttyp: | Article (Journal) |
| Sprache: | Englisch |
| Veröffentlicht: |
10 January 2026
|
| In: |
Small Methods
Year: 2026, Jahrgang: 10, Heft: 1, Pages: 1-15 |
| ISSN: | 2366-9608 |
| DOI: | 10.1002/smtd.202500885 |
| Online-Zugang: | Verlag, lizenzpflichtig, Volltext: https://doi.org/10.1002/smtd.202500885 Verlag, lizenzpflichtig, Volltext: https://onlinelibrary.wiley.com/doi/abs/10.1002/smtd.202500885 
|
| Verfasserangaben: | Renpeng Ding, Kerem Celikay, Ming Ni, Yong Hou, Yan Zhou, and Karl Rohr |
MARC
| LEADER | 00000naa a2200000 c 4500 | ||
|---|---|---|---|
| 001 | 1950289818 | ||
| 003 | DE-627 | ||
| 005 | 20260127092552.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 260127s2026 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1002/smtd.202500885 |2 doi | |
| 035 | |a (DE-627)1950289818 | ||
| 035 | |a (DE-599)KXP1950289818 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 32 |2 sdnb | ||
| 100 | 1 | |a Ding, Renpeng |e VerfasserIn |0 (DE-588)138806569X |0 (DE-627)1950290212 |4 aut | |
| 245 | 1 | 0 | |a Deep learning-based quality control using subcellular RNA spatial distribution patterns for cell segmentation in spatial transcriptomics data |c Renpeng Ding, Kerem Celikay, Ming Ni, Yong Hou, Yan Zhou, and Karl Rohr |
| 264 | 1 | |c 10 January 2026 | |
| 300 | |b Illustrationen | ||
| 300 | |a 15 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Zuerst veröffentlicht: 27. November 2025 | ||
| 500 | |a Gesehen am 27.01.2026 | ||
| 520 | |a Sequencing-based spatial transcriptomics (sST) techniques with high resolution enable transcriptome-wide RNA capture at subcellular resolution. Although new cell segmentation methods for sST data are continually being developed, accurately assigning RNA spots to corresponding cells still presents significant challenges and there is a lack of quality control methods. This work introduces a deep learning method for quality control of cell segmentation and improvement of the segmentation result. The proposed method exploits the subcellular spatial distribution patterns of different types of RNA by a deep neural network to assess the quality of segmented cells. The method identifies partially segmented cells typically due to low RNA capture or strong RNA diffusion as well as merged cells due to high cell density. In addition, the quality control method is combined with a Transformer-based cell segmentation method and it is shown that the cell segmentation performance improves by automatically removing low-quality segmented cells from the training dataset. The method is applied to both synthetic data and real Stereo-seq data, demonstrating its potential for quality control and enhancement of cell segmentation in sST data. | ||
| 650 | 4 | |a cell segmentation | |
| 650 | 4 | |a quality control | |
| 650 | 4 | |a spatial transcriptomics | |
| 650 | 4 | |a subcellular RNA spatial distribution | |
| 700 | 1 | |a Celikay, Kerem |e VerfasserIn |0 (DE-588)1333563477 |0 (DE-627)1891933280 |4 aut | |
| 700 | 1 | |a Ni, Ming |e VerfasserIn |4 aut | |
| 700 | 1 | |a Hou, Yong |e VerfasserIn |4 aut | |
| 700 | 1 | |a Zhou, Yan |e VerfasserIn |4 aut | |
| 700 | 1 | |a Rohr, Karl |e VerfasserIn |0 (DE-588)137474466 |0 (DE-627)695829440 |0 (DE-576)303788593 |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Small Methods |d Weinheim : WILEY-VCH Verlag GmbH & Co. KGaA, 2017 |g 10(2026), 1 vom: Jan., Artikel-ID e00885, Seite 1-15 |h Online-Ressource |w (DE-627)880235276 |w (DE-600)2884448-8 |w (DE-576)483498378 |x 2366-9608 |7 nnas |a Deep learning-based quality control using subcellular RNA spatial distribution patterns for cell segmentation in spatial transcriptomics data |
| 773 | 1 | 8 | |g volume:10 |g year:2026 |g number:1 |g month:01 |g elocationid:e00885 |g pages:1-15 |g extent:15 |a Deep learning-based quality control using subcellular RNA spatial distribution patterns for cell segmentation in spatial transcriptomics data |
| 856 | 4 | 0 | |u https://doi.org/10.1002/smtd.202500885 |x Verlag |x Resolving-System |z lizenzpflichtig |3 Volltext |7 1 |
| 856 | 4 | 0 | |u https://onlinelibrary.wiley.com/doi/abs/10.1002/smtd.202500885 |x Verlag |z lizenzpflichtig |3 Volltext |7 1 |
| 951 | |a AR | ||
| 992 | |a 20260127 | ||
| 993 | |a Article | ||
| 994 | |a 2026 | ||
| 998 | |g 137474466 |a Rohr, Karl |m 137474466:Rohr, Karl |d 160000 |d 160100 |d 160000 |e 160000PR137474466 |e 160100PR137474466 |e 160000PR137474466 |k 0/160000/ |k 1/160000/160100/ |k 0/160000/ |p 3 | ||
| 998 | |g 1333563477 |a Celikay, Kerem |m 1333563477:Celikay, Kerem |d 700000 |d 716000 |e 700000PC1333563477 |e 716000PC1333563477 |k 0/700000/ |k 1/700000/716000/ |p 2 | ||
| 998 | |g 138806569X |a Ding, Renpeng |m 138806569X:Ding, Renpeng |d 700000 |d 716000 |e 700000PD138806569X |e 716000PD138806569X |k 0/700000/ |k 1/700000/716000/ |p 1 |x j | ||
| 999 | |a KXP-PPN1950289818 |e 4861196051 | ||
| BIB | |a Y | ||
| SER | |a journal | ||
| JSO | |a {"note":["Zuerst veröffentlicht: 27. November 2025","Gesehen am 27.01.2026"],"name":{"displayForm":["Renpeng Ding, Kerem Celikay, Ming Ni, Yong Hou, Yan Zhou, and Karl Rohr"]},"recId":"1950289818","origin":[{"dateIssuedDisp":"10 January 2026","dateIssuedKey":"2026"}],"person":[{"given":"Renpeng","family":"Ding","role":"aut","display":"Ding, Renpeng"},{"display":"Celikay, Kerem","role":"aut","family":"Celikay","given":"Kerem"},{"given":"Ming","family":"Ni","role":"aut","display":"Ni, Ming"},{"family":"Hou","given":"Yong","display":"Hou, Yong","role":"aut"},{"display":"Zhou, Yan","role":"aut","family":"Zhou","given":"Yan"},{"family":"Rohr","given":"Karl","display":"Rohr, Karl","role":"aut"}],"language":["eng"],"title":[{"title":"Deep learning-based quality control using subcellular RNA spatial distribution patterns for cell segmentation in spatial transcriptomics data","title_sort":"Deep learning-based quality control using subcellular RNA spatial distribution patterns for cell segmentation in spatial transcriptomics data"}],"relHost":[{"note":["Gesehen am 09.05.22"],"disp":"Deep learning-based quality control using subcellular RNA spatial distribution patterns for cell segmentation in spatial transcriptomics dataSmall Methods","language":["eng"],"title":[{"title":"Small Methods","title_sort":"Small Methods"}],"physDesc":[{"extent":"Online-Ressource"}],"recId":"880235276","origin":[{"publisherPlace":"Weinheim","dateIssuedDisp":"[2017]-","publisher":"WILEY-VCH Verlag GmbH & Co. KGaA"}],"pubHistory":["Volume 1, issue 1/2 (February 13, 2017)-"],"part":{"text":"10(2026), 1 vom: Jan., Artikel-ID e00885, Seite 1-15","issue":"1","pages":"1-15","extent":"15","year":"2026","volume":"10"},"type":{"bibl":"periodical","media":"Online-Ressource"},"id":{"eki":["880235276"],"doi":["10.1002/(ISSN)2366-9608"],"issn":["2366-9608"],"zdb":["2884448-8"]}}],"physDesc":[{"extent":"15 S.","noteIll":"Illustrationen"}],"id":{"doi":["10.1002/smtd.202500885"],"eki":["1950289818"]},"type":{"bibl":"article-journal","media":"Online-Ressource"}} | ||
| SRT | |a DINGRENPENDEEPLEARNI1020 | ||