Beyond one-hot encoding: lower dimensional target embedding
Target encoding plays a central role when learning Convolutional Neural Networks. In this realm, one-hot encoding is the most prevalent strategy due to its simplicity. However, this so widespread encoding schema assumes a flat label space, thus ignoring rich relationships existing among labels that...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article (Journal) |
| Language: | English |
| Published: |
11 May 2018
|
| In: |
Image and vision computing
Year: 2018, Volume: 75, Pages: 21-31 |
| DOI: | 10.1016/j.imavis.2018.04.004 |
| Online Access: | Verlag, lizenzpflichtig, Volltext: https://doi.org/10.1016/j.imavis.2018.04.004 Verlag, lizenzpflichtig, Volltext: http://www.sciencedirect.com/science/article/pii/S0262885618300623 
|
| Author Notes: | Pau Rodríguez, Miguel A. Bautista, Jordi Gonzàlez, Sergio Escalera |
| Summary: | Target encoding plays a central role when learning Convolutional Neural Networks. In this realm, one-hot encoding is the most prevalent strategy due to its simplicity. However, this so widespread encoding schema assumes a flat label space, thus ignoring rich relationships existing among labels that can be exploited during training. In large-scale datasets, data does not span the full label space, but instead lies in a low-dimensional output manifold. Following this observation, we embed the targets into a low-dimensional space, drastically improving convergence speed while preserving accuracy. Our contribution is two fold: (i) We show that random projections of the label space are a valid tool to find such lower dimensional embeddings, boosting dramatically convergence rates at zero computational cost; and (ii) we propose a normalized eigenrepresentation of the class manifold that encodes the targets with minimal information loss, improving the accuracy of random projections encoding while enjoying the same convergence rates. Experiments on CIFAR-100, CUB200-2011, Imagenet, and MIT Places demonstrate that the proposed approach drastically improves convergence speed while reaching very competitive accuracy rates. |
|---|---|
| Item Description: | Gesehen am 21.04.2020 |
| Physical Description: | Online Resource |
| DOI: | 10.1016/j.imavis.2018.04.004 |