Alert button
Picture for Jie Wen

Jie Wen

Alert button

Information Recovery-Driven Deep Incomplete Multi-view Clustering Network

Apr 02, 2023
Chengliang Liu, Jie Wen, Zhihao Wu, Xiaoling Luo, Chao Huang, Yong Xu

Figure 1 for Information Recovery-Driven Deep Incomplete Multi-view Clustering Network
Figure 2 for Information Recovery-Driven Deep Incomplete Multi-view Clustering Network
Figure 3 for Information Recovery-Driven Deep Incomplete Multi-view Clustering Network
Figure 4 for Information Recovery-Driven Deep Incomplete Multi-view Clustering Network

Incomplete multi-view clustering is a hot and emerging topic. It is well known that unavoidable data incompleteness greatly weakens the effective information of multi-view data. To date, existing incomplete multi-view clustering methods usually bypass unavailable views according to prior missing information, which is considered as a second-best scheme based on evasion. Other methods that attempt to recover missing information are mostly applicable to specific two-view datasets. To handle these problems, in this paper, we propose an information recovery-driven deep incomplete multi-view clustering network, termed as RecFormer. Concretely, a two-stage autoencoder network with the self-attention structure is built to synchronously extract high-level semantic representations of multiple views and recover the missing data. Besides, we develop a recurrent graph reconstruction mechanism that cleverly leverages the restored views to promote the representation learning and the further data reconstruction. Visualization of recovery results are given and sufficient experimental results confirm that our RecFormer has obvious advantages over other top methods.

* Please contact me if you have any questions: liucl1996@163.com. The code will be opened upon acceptance 
Viaarxiv icon

Learning Reliable Representations for Incomplete Multi-View Partial Multi-Label Classification

Mar 30, 2023
Chengliang Liu, Jie Wen, Yong Xu, Liqiang Nie, Min Zhang

Figure 1 for Learning Reliable Representations for Incomplete Multi-View Partial Multi-Label Classification
Figure 2 for Learning Reliable Representations for Incomplete Multi-View Partial Multi-Label Classification
Figure 3 for Learning Reliable Representations for Incomplete Multi-View Partial Multi-Label Classification
Figure 4 for Learning Reliable Representations for Incomplete Multi-View Partial Multi-Label Classification

As a cross-topic of multi-view learning and multi-label classification, multi-view multi-label classification has gradually gained traction in recent years. The application of multi-view contrastive learning has further facilitated this process, however, the existing multi-view contrastive learning methods crudely separate the so-called negative pair, which largely results in the separation of samples belonging to the same category or similar ones. Besides, plenty of multi-view multi-label learning methods ignore the possible absence of views and labels. To address these issues, in this paper, we propose an incomplete multi-view partial multi-label classification network named RANK. In this network, a label-driven multi-view contrastive learning strategy is proposed to leverage supervised information to preserve the structure within view and perform consistent alignment across views. Furthermore, we break through the view-level weights inherent in existing methods and propose a quality-aware sub-network to dynamically assign quality scores to each view of each sample. The label correlation information is fully utilized in the final multi-label cross-entropy classification loss, effectively improving the discriminative power. Last but not least, our model is not only able to handle complete multi-view multi-label datasets, but also works on datasets with missing instances and labels. Extensive experiments confirm that our RANK outperforms existing state-of-the-art methods.

* Please contact me if you have any questions: liucl1996@163.com 
Viaarxiv icon

DICNet: Deep Instance-Level Contrastive Network for Double Incomplete Multi-View Multi-Label Classification

Mar 23, 2023
Chengliang Liu, Jie Wen, Xiaoling Luo, Chao Huang, Zhihao Wu, Yong Xu

Figure 1 for DICNet: Deep Instance-Level Contrastive Network for Double Incomplete Multi-View Multi-Label Classification
Figure 2 for DICNet: Deep Instance-Level Contrastive Network for Double Incomplete Multi-View Multi-Label Classification
Figure 3 for DICNet: Deep Instance-Level Contrastive Network for Double Incomplete Multi-View Multi-Label Classification
Figure 4 for DICNet: Deep Instance-Level Contrastive Network for Double Incomplete Multi-View Multi-Label Classification

In recent years, multi-view multi-label learning has aroused extensive research enthusiasm. However, multi-view multi-label data in the real world is commonly incomplete due to the uncertain factors of data collection and manual annotation, which means that not only multi-view features are often missing, and label completeness is also difficult to be satisfied. To deal with the double incomplete multi-view multi-label classification problem, we propose a deep instance-level contrastive network, namely DICNet. Different from conventional methods, our DICNet focuses on leveraging deep neural network to exploit the high-level semantic representations of samples rather than shallow-level features. First, we utilize the stacked autoencoders to build an end-to-end multi-view feature extraction framework to learn the view-specific representations of samples. Furthermore, in order to improve the consensus representation ability, we introduce an incomplete instance-level contrastive learning scheme to guide the encoders to better extract the consensus information of multiple views and use a multi-view weighted fusion module to enhance the discrimination of semantic features. Overall, our DICNet is adept in capturing consistent discriminative representations of multi-view multi-label data and avoiding the negative effects of missing views and missing labels. Extensive experiments performed on five datasets validate that our method outperforms other state-of-the-art methods.

* Accepted to AAAI-2023, code is available at https://github.com/justsmart/DICNet 
Viaarxiv icon

Incomplete Multi-View Multi-Label Learning via Label-Guided Masked View- and Category-Aware Transformers

Mar 13, 2023
Chengliang Liu, Jie Wen, Xiaoling Luo, Yong Xu

Figure 1 for Incomplete Multi-View Multi-Label Learning via Label-Guided Masked View- and Category-Aware Transformers
Figure 2 for Incomplete Multi-View Multi-Label Learning via Label-Guided Masked View- and Category-Aware Transformers
Figure 3 for Incomplete Multi-View Multi-Label Learning via Label-Guided Masked View- and Category-Aware Transformers
Figure 4 for Incomplete Multi-View Multi-Label Learning via Label-Guided Masked View- and Category-Aware Transformers

As we all know, multi-view data is more expressive than single-view data and multi-label annotation enjoys richer supervision information than single-label, which makes multi-view multi-label learning widely applicable for various pattern recognition tasks. In this complex representation learning problem, three main challenges can be characterized as follows: i) How to learn consistent representations of samples across all views? ii) How to exploit and utilize category correlations of multi-label to guide inference? iii) How to avoid the negative impact resulting from the incompleteness of views or labels? To cope with these problems, we propose a general multi-view multi-label learning framework named label-guided masked view- and category-aware transformers in this paper. First, we design two transformer-style based modules for cross-view features aggregation and multi-label classification, respectively. The former aggregates information from different views in the process of extracting view-specific features, and the latter learns subcategory embedding to improve classification performance. Second, considering the imbalance of expressive power among views, an adaptively weighted view fusion module is proposed to obtain view-consistent embedding features. Third, we impose a label manifold constraint in sample-level representation learning to maximize the utilization of supervised information. Last but not least, all the modules are designed under the premise of incomplete views and labels, which makes our method adaptable to arbitrary multi-view and multi-label data. Extensive experiments on five datasets confirm that our method has clear advantages over other state-of-the-art methods.

* Accepted to AAAI-23 
Viaarxiv icon

Cross-view Graph Contrastive Representation Learning on Partially Aligned Multi-view Data

Nov 08, 2022
Yiming Wang, Dongxia Chang, Zhiqiang Fu, Jie Wen, Yao Zhao

Figure 1 for Cross-view Graph Contrastive Representation Learning on Partially Aligned Multi-view Data
Figure 2 for Cross-view Graph Contrastive Representation Learning on Partially Aligned Multi-view Data
Figure 3 for Cross-view Graph Contrastive Representation Learning on Partially Aligned Multi-view Data
Figure 4 for Cross-view Graph Contrastive Representation Learning on Partially Aligned Multi-view Data

Multi-view representation learning has developed rapidly over the past decades and has been applied in many fields. However, most previous works assumed that each view is complete and aligned. This leads to an inevitable deterioration in their performance when encountering practical problems such as missing or unaligned views. To address the challenge of representation learning on partially aligned multi-view data, we propose a new cross-view graph contrastive learning framework, which integrates multi-view information to align data and learn latent representations. Compared with current approaches, the proposed method has the following merits: (1) our model is an end-to-end framework that simultaneously performs view-specific representation learning via view-specific autoencoders and cluster-level data aligning by combining multi-view information with the cross-view graph contrastive learning; (2) it is easy to apply our model to explore information from three or more modalities/sources as the cross-view graph contrastive learning is devised. Extensive experiments conducted on several real datasets demonstrate the effectiveness of the proposed method on the clustering and classification tasks.

Viaarxiv icon

A Survey on Incomplete Multi-view Clustering

Aug 17, 2022
Jie Wen, Zheng Zhang, Lunke Fei, Bob Zhang, Yong Xu, Zhao Zhang, Jinxing Li

Figure 1 for A Survey on Incomplete Multi-view Clustering
Figure 2 for A Survey on Incomplete Multi-view Clustering
Figure 3 for A Survey on Incomplete Multi-view Clustering
Figure 4 for A Survey on Incomplete Multi-view Clustering

Conventional multi-view clustering seeks to partition data into respective groups based on the assumption that all views are fully observed. However, in practical applications, such as disease diagnosis, multimedia analysis, and recommendation system, it is common to observe that not all views of samples are available in many cases, which leads to the failure of the conventional multi-view clustering methods. Clustering on such incomplete multi-view data is referred to as incomplete multi-view clustering. In view of the promising application prospects, the research of incomplete multi-view clustering has noticeable advances in recent years. However, there is no survey to summarize the current progresses and point out the future research directions. To this end, we review the recent studies of incomplete multi-view clustering. Importantly, we provide some frameworks to unify the corresponding incomplete multi-view clustering methods, and make an in-depth comparative analysis for some representative methods from theoretical and experimental perspectives. Finally, some open problems in the incomplete multi-view clustering field are offered for researchers.

* Accepted by IEEE Transactions on Systems, Man, and Cybernetics: Systems (2022) 
Viaarxiv icon

Localized Sparse Incomplete Multi-view Clustering

Aug 05, 2022
Chengliang Liu, Zhihao Wu, Jie Wen, Chao Huang, Yong Xu

Figure 1 for Localized Sparse Incomplete Multi-view Clustering
Figure 2 for Localized Sparse Incomplete Multi-view Clustering
Figure 3 for Localized Sparse Incomplete Multi-view Clustering
Figure 4 for Localized Sparse Incomplete Multi-view Clustering

Incomplete multi-view clustering, which aims to solve the clustering problem on the incomplete multi-view data with partial view missing, has received more and more attention in recent years. Although numerous methods have been developed, most of the methods either cannot flexibly handle the incomplete multi-view data with arbitrary missing views or do not consider the negative factor of information imbalance among views. Moreover, some methods do not fully explore the local structure of all incomplete views. To tackle these problems, this paper proposes a simple but effective method, named localized sparse incomplete multi-view clustering (LSIMVC). Different from the existing methods, LSIMVC intends to learn a sparse and structured consensus latent representation from the incomplete multi-view data by optimizing a sparse regularized and novel graph embedded multi-view matrix factorization model. Specifically, in such a novel model based on the matrix factorization, a l1 norm based sparse constraint is introduced to obtain the sparse low-dimensional individual representations and the sparse consensus representation. Moreover, a novel local graph embedding term is introduced to learn the structured consensus representation. Different from the existing works, our local graph embedding term aggregates the graph embedding task and consensus representation learning task into a concise term. Furthermore, to reduce the imbalance factor of incomplete multi-view learning, an adaptive weighted learning scheme is introduced to LSIMVC. Finally, an efficient optimization strategy is given to solve the optimization problem of our proposed model. Comprehensive experimental results performed on six incomplete multi-view databases verify that the performance of our LSIMVC is superior to the state-of-the-art IMC approaches. The code is available in https://github.com/justsmart/LSIMVC.

* Published in IEEE Transactions on Multimedia (TMM). The code is available in Github https://github.com/justsmart/LSIMVC 
Viaarxiv icon

Global-Supervised Contrastive Loss and View-Aware-Based Post-Processing for Vehicle Re-Identification

Apr 17, 2022
Zhijun Hu, Yong Xu, Jie Wen, Xianjing Cheng, Zaijun Zhang, Lilei Sun, Yaowei Wang

Figure 1 for Global-Supervised Contrastive Loss and View-Aware-Based Post-Processing for Vehicle Re-Identification
Figure 2 for Global-Supervised Contrastive Loss and View-Aware-Based Post-Processing for Vehicle Re-Identification
Figure 3 for Global-Supervised Contrastive Loss and View-Aware-Based Post-Processing for Vehicle Re-Identification
Figure 4 for Global-Supervised Contrastive Loss and View-Aware-Based Post-Processing for Vehicle Re-Identification

In this paper, we propose a Global-Supervised Contrastive loss and a view-aware-based post-processing (VABPP) method for the field of vehicle re-identification. The traditional supervised contrastive loss calculates the distances of features within the batch, so it has the local attribute. While the proposed Global-Supervised Contrastive loss has new properties and has good global attributes, the positive and negative features of each anchor in the training process come from the entire training set. The proposed VABPP method is the first time that the view-aware-based method is used as a post-processing method in the field of vehicle re-identification. The advantages of VABPP are that, first, it is only used during testing and does not affect the training process. Second, as a post-processing method, it can be easily integrated into other trained re-id models. We directly apply the view-pair distance scaling coefficient matrix calculated by the model trained in this paper to another trained re-id model, and the VABPP method greatly improves its performance, which verifies the feasibility of the VABPP method.

Viaarxiv icon

ACTIVE:Augmentation-Free Graph Contrastive Learning for Partial Multi-View Clustering

Mar 01, 2022
Yiming Wang, Dongxia Chang, Zhiqiang Fu, Jie Wen, Yao Zhao

Figure 1 for ACTIVE:Augmentation-Free Graph Contrastive Learning for Partial Multi-View Clustering
Figure 2 for ACTIVE:Augmentation-Free Graph Contrastive Learning for Partial Multi-View Clustering
Figure 3 for ACTIVE:Augmentation-Free Graph Contrastive Learning for Partial Multi-View Clustering
Figure 4 for ACTIVE:Augmentation-Free Graph Contrastive Learning for Partial Multi-View Clustering

In this paper, we propose an augmentation-free graph contrastive learning framework, namely ACTIVE, to solve the problem of partial multi-view clustering. Notably, we suppose that the representations of similar samples (i.e., belonging to the same cluster) and their multiply views features should be similar. This is distinct from the general unsupervised contrastive learning that assumes an image and its augmentations share a similar representation. Specifically, relation graphs are constructed using the nearest neighbours to identify existing similar samples, then the constructed inter-instance relation graphs are transferred to the missing views to build graphs on the corresponding missing data. Subsequently, two main components, within-view graph contrastive learning (WGC) and cross-view graph consistency learning (CGC), are devised to maximize the mutual information of different views within a cluster. The proposed approach elevates instance-level contrastive learning and missing data inference to the cluster-level, effectively mitigating the impact of individual missing data on clustering. Experiments on several challenging datasets demonstrate the superiority of our proposed methods.

Viaarxiv icon