Tang, M., Dogariu, E., & Yu, J. RePEAtO: Relational Paragraph-level Embeddings for Article Outlining.
|
 |
|
Abstract
|
It is found that humans far outperform the authors' best models, and thus lay the groundwork for further study in this task, and by extension, in hierarchical tasks that require embeddings of longer forms of text. We formulate a novel task: article outlining, the reconstruction of the hierarchical structure of headings and sub-headings within an article from just a list of paragraphs. We propose and analyze the adJacent Paragraph Least-common-ancestor-distance (JPL) score evaluation metric, compose a full-stack web application to collect human performance data on the article outlining task, and then explore various models under a composable encoder-decoder general architecture. We find that out of various embedding techniques based on past work on word, sentence, and document embeddings, SimCSE [2] performs the best and re-sults in our strongest model using a recursive MLP decoder and paragraph embeddings composed of averaged SimCSE sentence embeddings. We find evidence that recursive models best capture the hierarchical information needed for this task, and perform ablations and further analysis of the relevant paragraph embedding spaces. Nevertheless, we find that humans far outperform our best models, and thus lay the groundwork for further study in this task, and by extension, in hierarchical tasks that require embeddings of longer forms of text.
|
| Notes |
[Online; accessed 1. Jun. 2024]
|
