Discourse-aware semantic self-attention for narrative reading comprehension
In this work, we propose to use linguistic annotations as a basis for a Discourse-Aware Semantic Self-Attention encoder that we employ for reading comprehension on narrative texts. We extract relations between discourse units, events, and their arguments as well as coreferring mentions, using availa...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article (Journal) Chapter/Article |
| Language: | English |
| Published: |
November 2019
|
| In: |
2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing - proceedings of the conference
Year: 2019, Pages: 2541-2552 |
| DOI: | 10.18653/v1/D19-1257 |
| Online Access: | Verlag, kostenfrei, Volltext: https://doi.org/10.18653/v1/D19-1257 Verlag, kostenfrei, Volltext: https://aclanthology.org/D19-1257 
|
| Author Notes: | Todor Mihaylov, Anette Frank |
| Summary: | In this work, we propose to use linguistic annotations as a basis for a Discourse-Aware Semantic Self-Attention encoder that we employ for reading comprehension on narrative texts. We extract relations between discourse units, events, and their arguments as well as coreferring mentions, using available annotation tools. Our empirical evaluation shows that the investigated structures improve the overall performance (up to +3.4 Rouge-L), especially intra-sentential and cross-sentential discourse relations, sentence-internal semantic role relations, and long-distance coreference relations. We show that dedicating self-attention heads to intra-sentential relations and relations connecting neighboring sentences is beneficial for finding answers to questions in longer contexts. Our findings encourage the use of discourse-semantic annotations to enhance the generalization capacity of self-attention models for reading comprehension. |
|---|---|
| Item Description: | Gesehen am 15.07.2024 |
| Physical Description: | Online Resource |
| ISBN: | 9781950737901 |
| DOI: | 10.18653/v1/D19-1257 |