Large-scale interactive retrieval in art collections using multi-style feature aggregation
Finding objects and motifs across artworks is of great importance for art history as it helps to understand individual works and analyze relations between them. The advent of digitization has produced extensive digital art collections with many research opportunities. However, manual approaches are...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article (Journal) |
| Language: | English |
| Published: |
November 24, 2021
|
| In: |
PLOS ONE
Year: 2021, Volume: 16, Issue: 11, Pages: 1-38 |
| ISSN: | 1932-6203 |
| DOI: | 10.1371/journal.pone.0259718 |
| Online Access: | Verlag, lizenzpflichtig, Volltext: https://doi.org/10.1371/journal.pone.0259718 Verlag, lizenzpflichtig, Volltext: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0259718 
|
| Author Notes: | Nikolai Ufer, Max Simon, Sabine Lang, Björn Ommer |
MARC
| LEADER | 00000caa a2200000 c 4500 | ||
|---|---|---|---|
| 001 | 1793436223 | ||
| 003 | DE-627 | ||
| 005 | 20220820133259.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 220221s2021 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1371/journal.pone.0259718 |2 doi | |
| 035 | |a (DE-627)1793436223 | ||
| 035 | |a (DE-599)KXP1793436223 | ||
| 035 | |a (OCoLC)1341441781 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 28 |2 sdnb | ||
| 100 | 1 | |a Ufer, Nikolai |d 1984- |e VerfasserIn |0 (DE-588)1152282514 |0 (DE-627)101376398X |0 (DE-576)499785347 |4 aut | |
| 245 | 1 | 0 | |a Large-scale interactive retrieval in art collections using multi-style feature aggregation |c Nikolai Ufer, Max Simon, Sabine Lang, Björn Ommer |
| 264 | 1 | |c November 24, 2021 | |
| 300 | |a 38 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Gesehen am 21.02.2022 | ||
| 520 | |a Finding objects and motifs across artworks is of great importance for art history as it helps to understand individual works and analyze relations between them. The advent of digitization has produced extensive digital art collections with many research opportunities. However, manual approaches are inadequate to handle this amount of data, and it requires appropriate computer-based methods to analyze them. This article presents a visual search algorithm and user interface to support art historians to find objects and motifs in extensive datasets. Artistic image collections are subject to significant domain shifts induced by large variations in styles, artistic media, and materials. This poses new challenges to most computer vision models which are trained on photographs. To alleviate this problem, we introduce a multi-style feature aggregation that projects images into the same distribution, leading to more accurate and style-invariant search results. Our retrieval system is based on a voting procedure combined with fast nearest-neighbor search and enables finding and localizing motifs within an extensive image collection in seconds. The presented approach significantly improves the state-of-the-art in terms of accuracy and search time on various datasets and applies to large and inhomogeneous collections. In addition to the search algorithm, we introduce a user interface that allows art historians to apply our algorithm in practice. The interface enables users to search for single regions, multiple regions regarding different connection types and holds an interactive feedback system to improve retrieval results further. With our methodological contribution and easy-to-use user interface, this work manifests further progress towards a computer-based analysis of visual art. | ||
| 650 | 4 | |a Algorithms | |
| 650 | 4 | |a Computer vision | |
| 650 | 4 | |a Historians | |
| 650 | 4 | |a Horses | |
| 650 | 4 | |a Information retrieval | |
| 650 | 4 | |a Lions | |
| 650 | 4 | |a Metadata | |
| 650 | 4 | |a User interfaces | |
| 700 | 1 | |a Simon, Max |e VerfasserIn |0 (DE-588)1237423945 |0 (DE-627)1764128222 |4 aut | |
| 700 | 1 | |a Lang, Sabine |e VerfasserIn |0 (DE-588)1100083812 |0 (DE-627)858973847 |0 (DE-576)469478187 |4 aut | |
| 700 | 1 | |a Ommer, Björn |d 1981- |e VerfasserIn |0 (DE-588)1034893106 |0 (DE-627)746457510 |0 (DE-576)382507916 |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t PLOS ONE |d San Francisco, California, US : PLOS, 2006 |g 16(2021), 11, Artikel-ID e0259718, Seite 1-38 |h Online-Ressource |w (DE-627)523574592 |w (DE-600)2267670-3 |w (DE-576)281331979 |x 1932-6203 |7 nnas |a Large-scale interactive retrieval in art collections using multi-style feature aggregation |
| 773 | 1 | 8 | |g volume:16 |g year:2021 |g number:11 |g elocationid:e0259718 |g pages:1-38 |g extent:38 |a Large-scale interactive retrieval in art collections using multi-style feature aggregation |
| 856 | 4 | 0 | |u https://doi.org/10.1371/journal.pone.0259718 |x Verlag |x Resolving-System |z lizenzpflichtig |3 Volltext |
| 856 | 4 | 0 | |u https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0259718 |x Verlag |z lizenzpflichtig |3 Volltext |
| 951 | |a AR | ||
| 992 | |a 20220221 | ||
| 993 | |a Article | ||
| 994 | |a 2021 | ||
| 998 | |g 1034893106 |a Ommer, Björn |m 1034893106:Ommer, Björn |p 4 |y j | ||
| 998 | |g 1100083812 |a Lang, Sabine |m 1100083812:Lang, Sabine |p 3 | ||
| 998 | |g 1237423945 |a Simon, Max |m 1237423945:Simon, Max |p 2 | ||
| 998 | |g 1152282514 |a Ufer, Nikolai |m 1152282514:Ufer, Nikolai |d 60000 |e 60000PU1152282514 |k 0/60000/ |p 1 |x j | ||
| 999 | |a KXP-PPN1793436223 |e 4070847634 | ||
| BIB | |a Y | ||
| SER | |a journal | ||
| JSO | |a {"physDesc":[{"extent":"38 S."}],"relHost":[{"disp":"Large-scale interactive retrieval in art collections using multi-style feature aggregationPLOS ONE","note":["Schreibweise des Titels bis 2012: PLoS ONE","Gesehen am 20.03.19"],"type":{"bibl":"periodical","media":"Online-Ressource"},"language":["eng"],"corporate":[{"roleDisplay":"Herausgebendes Organ","display":"Public Library of Science","role":"isb"}],"recId":"523574592","pubHistory":["1.2006 -"],"part":{"year":"2021","issue":"11","pages":"1-38","text":"16(2021), 11, Artikel-ID e0259718, Seite 1-38","volume":"16","extent":"38"},"title":[{"title_sort":"PLOS ONE","title":"PLOS ONE"}],"physDesc":[{"extent":"Online-Ressource"}],"name":{"displayForm":["Public Library of Science"]},"origin":[{"dateIssuedDisp":"2006-","dateIssuedKey":"2006","publisher":"PLOS ; PLoS","publisherPlace":"San Francisco, California, US ; Lawrence, Kan."}],"id":{"issn":["1932-6203"],"eki":["523574592"],"zdb":["2267670-3"]}}],"origin":[{"dateIssuedDisp":"November 24, 2021","dateIssuedKey":"2021"}],"id":{"doi":["10.1371/journal.pone.0259718"],"eki":["1793436223"]},"name":{"displayForm":["Nikolai Ufer, Max Simon, Sabine Lang, Björn Ommer"]},"type":{"bibl":"article-journal","media":"Online-Ressource"},"note":["Gesehen am 21.02.2022"],"recId":"1793436223","language":["eng"],"title":[{"title_sort":"Large-scale interactive retrieval in art collections using multi-style feature aggregation","title":"Large-scale interactive retrieval in art collections using multi-style feature aggregation"}],"person":[{"family":"Ufer","given":"Nikolai","display":"Ufer, Nikolai","roleDisplay":"VerfasserIn","role":"aut"},{"display":"Simon, Max","roleDisplay":"VerfasserIn","role":"aut","family":"Simon","given":"Max"},{"given":"Sabine","family":"Lang","role":"aut","display":"Lang, Sabine","roleDisplay":"VerfasserIn"},{"role":"aut","roleDisplay":"VerfasserIn","display":"Ommer, Björn","given":"Björn","family":"Ommer"}]} | ||
| SRT | |a UFERNIKOLALARGESCALE2420 | ||