Biswas, D., Linzbach, S., Dimitrov, D., Jabeen, H., & Dietze, S. Broadening BERT vocabulary for Knowledge Graph Construction using Wikipedia2Vec.
|
 |
|
Abstract
|
The proposed approach in Track 1 focuses on expanding BERT’s vocabulary with a task-specific one (i.e., Wikipedia2Vec) and facilitating its usage through prompt tuning with OPTIPROMPT. Recent advancements in natural language processing (NLP) have been driven by the utilization of large language models like BERT. These models, pre-trained on extensive textual data, capture linguistic and relational knowledge. Therefore, cloze-style prompts, which involve filling in missing words in a sentence, can be used to solve knowledge-intensive NLP tasks with the help of a language model. The "Knowledge Base Construction from Pre-trained Language Models (LM-KBC 2023)" challenge aims to harness language models’ potential for knowledge graph construction through prompts. In particular, contestants are challenged to infer the correct Wikidata ID of objects, given a prompt used to link subject, relation, and object. Automatically inferring the correct objects would help in reducing the need for an expensive manual graph population. Our proposed approach in Track 1 focuses on expanding BERT’s vocabulary with a task-specific one (i.e., Wikipedia2Vec) and facilitating its usage through prompt tuning with OPTIPROMPT.
|
