Niu, J. ML-DPR: A Meta-Learning-Based Model for Domain-Adaptive Dense Passage Retrieval.
|
 |
|
Abstract
|
A meta-learning model is proposed which acquires knowledge from each domain-specific dataset (meta-task) and can generalize across domains, including finance, education, and others, and achieves the state-of-the-art performance. Dense Passage Retrieval (DPR) models have demonstrated substantial advances in retrieving relevant passages from large document collections. These models employ encoders to generate vector representations of documents and queries. However, DPR models tend to produce low-quality sentence representations when applied across domains. Collecting domain-specific data and training specialized models for each domain is expensive and time-consuming. In this paper, we propose a meta-learning model which acquires knowledge from each domain-specific dataset (meta-task) and can generalize across domains, including finance, education, and others. We further apply knowledge distillation to transfer the knowledge from an interaction-based BERT model to a representation-based BERT model, improving performance. Experiments on five domains from the DPR dataset demonstrate that our method achieves the state-of-the-art performance.
|
