Cs224n, S., Project, D., Ayyagari, R., & Hodges, A. We do it BERTer: Comparison of Finetuning Methods to Improve Sentence Embeddings.
|
 |
|
Abstract
|
This paper implements an ensemble method combining multi-task finetuning, cosine similarity, and the AdamW optimizer that achieves an overall accuracy of 0.622 on the test leaderboard, which is an improvement on the traditional BERT model’s overall accuracy. We aim to contribute to the growing body of work that finetunes Bidirectional Encoder Representations for Transformers (BERT) (3) for natural language understanding (NLU) tasks by creating a more efficient task-specific model for (1) sentiment analysis, (2) paraphrase detection, and (3) semantic textual similarity. In this paper, we implement a model that achieves an overall accuracy of 0.622 on the test leaderboard, which is an improvement on the traditional BERT model’s overall accuracy. In doing so, we demonstrate that an ensemble method combining multi-task finetuning, cosine similarity, the AdamW optimizer
|
| Notes |
[Online; accessed 1. Jun. 2024]
|
