Towards brain-activity-controlled information retrieval: decoding image relevance from MEG signals
We hypothesize that brain activity can be used to control future information retrieval systems. To this end, we conducted a feasibility study on predicting the relevance of visual objects from brain activity. We analyze both magnetoencephalographic (MEG) and gaze signals from nine subjects who were...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article (Journal) |
| Language: | English |
| Published: |
13 January 2015
|
| In: |
NeuroImage
Year: 2015, Volume: 112, Pages: 288-298 |
| ISSN: | 1095-9572 |
| DOI: | 10.1016/j.neuroimage.2014.12.079 |
| Online Access: | Verlag, lizenzpflichtig, Volltext: https://doi.org/10.1016/j.neuroimage.2014.12.079 Verlag, lizenzpflichtig, Volltext: http://www.sciencedirect.com/science/article/pii/S1053811915000026 
|
| Author Notes: | Jukka-Pekka Kauppi, Melih Kandemir, Veli-Matti Saarinen, Lotta Hirvenkari, Lauri Parkkonen, Arto Klami, Riitta Hari, Samuel Kaski |
| Summary: | We hypothesize that brain activity can be used to control future information retrieval systems. To this end, we conducted a feasibility study on predicting the relevance of visual objects from brain activity. We analyze both magnetoencephalographic (MEG) and gaze signals from nine subjects who were viewing image collages, a subset of which was relevant to a predetermined task. We report three findings: i) the relevance of an image a subject looks at can be decoded from MEG signals with performance significantly better than chance, ii) fusion of gaze-based and MEG-based classifiers significantly improves the prediction performance compared to using either signal alone, and iii) non-linear classification of the MEG signals using Gaussian process classifiers outperforms linear classification. These findings break new ground for building brain-activity-based interactive image retrieval systems, as well as for systems utilizing feedback both from brain activity and eye movements. |
|---|---|
| Item Description: | Gesehen am 07.07.2020 |
| Physical Description: | Online Resource |
| ISSN: | 1095-9572 |
| DOI: | 10.1016/j.neuroimage.2014.12.079 |