Dong, H., Lin, J., Leng, Y., Chen, J., & Xie, Y. Retriever and Ranker Framework with Probabilistic Hard Negative Sampling for Code Search.
|
 |
|
Abstract
|
A cross-encoder architecture for code search that jointly encodes the semantic matching of query and code is introduced, and a Retriever-Ranker (RR) framework that cascades the dual-Encoder and cross- Encoder to promote the efficiency of evaluation and online serving is introduced. Pretrained Language Models (PLMs) have emerged as the state-of-the-art paradigm for code search tasks. The paradigm involves pretraining the model on search-irrelevant tasks such as masked language modeling, followed by the finetuning stage, which focuses on the search-relevant task. The typical finetuning method is to employ a dual-encoder architecture to encode semantic embeddings of query and code separately, and then calculate their similarity based on the embeddings. However, the typical dual-encoder architecture falls short in modeling token-level interactions between query and code, which limits the model's capabilities. In this paper, we propose a novel approach to address this limitation, introducing a cross-encoder architecture for code search that jointly encodes the semantic matching of query and code. We further introduce a Retriever-Ranker (RR) framework that cascades the dual-encoder and cross-encoder to promote the efficiency of evaluation and online serving. Moreover, we present a probabilistic hard negative sampling method to improve the cross-encoder's ability to distinguish hard negative codes, which further enhances the cascade RR framework. Experiments on four datasets using three code PLMs demonstrate the superiority of our proposed method.
|
