Alert button

Correlation between Alignment-Uniformity and Performance of Dense Contrastive Representations

Oct 17, 2022
Jong Hak Moon, Wonjae Kim, Edward Choi

Figure 1 for Correlation between Alignment-Uniformity and Performance of Dense Contrastive Representations
Figure 2 for Correlation between Alignment-Uniformity and Performance of Dense Contrastive Representations
Figure 3 for Correlation between Alignment-Uniformity and Performance of Dense Contrastive Representations
Figure 4 for Correlation between Alignment-Uniformity and Performance of Dense Contrastive Representations

Share this with someone who'll enjoy it:

Recently, dense contrastive learning has shown superior performance on dense prediction tasks compared to instance-level contrastive learning. Despite its supremacy, the properties of dense contrastive representations have not yet been carefully studied. Therefore, we analyze the theoretical ideas of dense contrastive learning using a standard CNN and straightforward feature matching scheme rather than propose a new complex method. Inspired by the analysis of the properties of instance-level contrastive representations through the lens of alignment and uniformity on the hypersphere, we employ and extend the same lens for the dense contrastive representations to analyze their underexplored properties. We discover the core principle in constructing a positive pair of dense features and empirically proved its validity. Also, we introduces a new scalar metric that summarizes the correlation between alignment-and-uniformity and downstream performance. Using this metric, we study various facets of densely learned contrastive representations such as how the correlation changes over single- and multi-object datasets or linear evaluation and dense prediction tasks. The source code is publicly available at: https://github.com/SuperSupermoon/DenseCL-analysis

* BMVC22 accepted  
View paper onarxiv icon

Share this with someone who'll enjoy it: