Alert button

Deep Continuous Prompt for Contrastive Learning of Sentence Embeddings

Mar 14, 2022
Yuxin Jiang, Wei Wang

Figure 1 for Deep Continuous Prompt for Contrastive Learning of Sentence Embeddings
Figure 2 for Deep Continuous Prompt for Contrastive Learning of Sentence Embeddings
Figure 3 for Deep Continuous Prompt for Contrastive Learning of Sentence Embeddings
Figure 4 for Deep Continuous Prompt for Contrastive Learning of Sentence Embeddings

Share this with someone who'll enjoy it:

The performance of sentence representation has been remarkably improved by the framework of contrastive learning. However, recent works still require full fine-tuning, which is quite inefficient for large-scaled pre-trained language models. To this end, we present a novel method which freezes the whole language model and only optimizes the prefix deep continuous prompts. It not only tunes around 0.1% parameters of the original language model, but avoids the cumbersome computation of searching handcrafted prompts. Experimental results show that our proposed DCPCSE outperforms the state-of-the-art method SimCSE by a large margin. We raise the performance of unsupervised BERT$_{base}$ and supervised RoBERTa$_{large}$ by 2.24 and 1.00 points, respectively. Our code is publicly avaliable at https://github.com/YJiangcm/DCPCSE

View paper onarxiv icon

Share this with someone who'll enjoy it: