Alert button

Label-Aware Automatic Verbalizer for Few-Shot Text Classification

Oct 19, 2023
Thanakorn Thaminkaew, Piyawat Lertvittayakumjorn, Peerapon Vateekul

Figure 1 for Label-Aware Automatic Verbalizer for Few-Shot Text Classification
Figure 2 for Label-Aware Automatic Verbalizer for Few-Shot Text Classification
Figure 3 for Label-Aware Automatic Verbalizer for Few-Shot Text Classification
Figure 4 for Label-Aware Automatic Verbalizer for Few-Shot Text Classification

Share this with someone who'll enjoy it:

Prompt-based learning has shown its effectiveness in few-shot text classification. One important factor in its success is a verbalizer, which translates output from a language model into a predicted class. Notably, the simplest and widely acknowledged verbalizer employs manual labels to represent the classes. However, manual selection does not guarantee the optimality of the selected words when conditioned on the chosen language model. Therefore, we propose Label-Aware Automatic Verbalizer (LAAV), effectively augmenting the manual labels to achieve better few-shot classification results. Specifically, we use the manual labels along with the conjunction "and" to induce the model to generate more effective words for the verbalizer. The experimental results on five datasets across five languages demonstrate that LAAV significantly outperforms existing verbalizers. Furthermore, our analysis reveals that LAAV suggests more relevant words compared to similar approaches, especially in mid-to-low resource languages.

View paper onarxiv icon

Share this with someone who'll enjoy it: