Alert button

CLUSTSEG: Clustering for Universal Segmentation

May 03, 2023
James Liang, Tianfei Zhou, Dongfang Liu, Wenguan Wang

Figure 1 for CLUSTSEG: Clustering for Universal Segmentation
Figure 2 for CLUSTSEG: Clustering for Universal Segmentation
Figure 3 for CLUSTSEG: Clustering for Universal Segmentation
Figure 4 for CLUSTSEG: Clustering for Universal Segmentation

Share this with someone who'll enjoy it:

We present CLUSTSEG, a general, transformer-based framework that tackles different image segmentation tasks (i.e., superpixel, semantic, instance, and panoptic) through a unified neural clustering scheme. Regarding queries as cluster centers, CLUSTSEG is innovative in two aspects:1) cluster centers are initialized in heterogeneous ways so as to pointedly address task-specific demands (e.g., instance- or category-level distinctiveness), yet without modifying the architecture; and 2) pixel-cluster assignment, formalized in a cross-attention fashion, is alternated with cluster center update, yet without learning additional parameters. These innovations closely link CLUSTSEG to EM clustering and make it a transparent and powerful framework that yields superior results across the above segmentation tasks.

* Accepted to ICML 2023; Code: https://github.com/JamesLiang819/ClustSeg  
View paper onarxiv icon

Share this with someone who'll enjoy it: