Conley, A., & Kalita, J. K. Enhancing Language Models with Knowledge Graph Embeddings.
|
 |
|
Abstract
|
By incorporating knowledge directly into the word embedding, this work aims to improve the task of nat-ural language inference, similar to those achieved by applying knowledge bases to machine reading (Yang and Mitchell 2019). Most NLP tasks use word embeddings to improve performance. Breakthroughs like ELMo (Peters et al. 2018) and BERT (Devlin et al. 2018) have shown that state of the art results can be achieved in many NLP tasks through good language models, even without a task-specific architecture. Word vectors have been a simple, popular, and effective language model for years. Methods for generating these word vectors typically use unsupervised learning based on the context in which each word is used within the greater corpus. We propose new method of generating these word vectors. We use knowledge embeddings extracted from knowledge bases like Freebase and WordNet (Bordes et al. 2013) as a starting point, and introduce syntactic information captured from existing language models. By incorporating knowledge directly into the word embedding we aim to improve the task of nat-ural language inference, similar to those achieved by applying knowledge bases to machine reading (Yang and Mitchell 2019). The new embeddings are judged primarily on their perfermance on the natural language inference model HBMP (Talman, Yli-Jyr¨a, and Tiedemann 2019). This performance is compared to that of Glove (Pennington, Socher, and Man-ning 2014), with the same architecture. No improvement was found to accuracy, however further steps to achieve the desired improvement are well defined.
|
| Notes |
[Online; accessed 31. May 2024]
|
