Understanding Survey Paper Taxonomy about Large Language Models via Graph Representation Learning

As new research on Large Language Models (LLMs) continues, it is difficult to keep up with new research and models. To help researchers synthesize the new research many have written survey papers, but even those have become numerous. In this paper, we develop a method to automatically assign survey papers to a taxonomy. We collect the metadata of 144 LLM survey papers and explore three paradigms to classify papers within the taxonomy. Our work indicates that leveraging graph structure information on co-category graphs can significantly outperform the language models in two paradigms; pre-trained language models' fine-tuning and zero-shot/few-shot classifications using LLMs. We find that our model surpasses an average human recognition level and that fine-tuning LLMs using weak labels generated by a smaller model, such as the GCN in this study, can be more effective than using ground-truth labels, revealing the potential of weak-to-strong generalization in the taxonomy classification task.

Robust Node Representation Learning via Graph Variational Diffusion Networks

Node representation learning by using Graph Neural Networks (GNNs) has been widely explored. However, in recent years, compelling evidence has revealed that GNN-based node representation learning can be substantially deteriorated by delicately-crafted perturbations in a graph structure. To learn robust node representation in the presence of perturbations, various works have been proposed to safeguard GNNs. Within these existing works, Bayesian label transition has been proven to be more effective, but this method is extensively reliant on a well-built prior distribution. The variational inference could address this limitation by sampling the latent node embedding from a Gaussian prior distribution. Besides, leveraging the Gaussian distribution (noise) in hidden layers is an appealing strategy to strengthen the robustness of GNNs. However, our experiments indicate that such a strategy can cause over-smoothing issues during node aggregation. In this work, we propose the Graph Variational Diffusion Network (GVDN), a new node encoder that effectively manipulates Gaussian noise to safeguard robustness on perturbed graphs while alleviating over-smoothing issues through two mechanisms: Gaussian diffusion and node embedding propagation. Thanks to these two mechanisms, our model can generate robust node embeddings for recovery. Specifically, we design a retraining mechanism using the generated node embedding to recover the performance of node classifications in the presence of perturbations. The experiments verify the effectiveness of our proposed model across six public datasets.

Robust Node Classification on Graphs: Jointly from Bayesian Label Transition and Topology-based Label Propagation

Node classification using Graph Neural Networks (GNNs) has been widely applied in various real-world scenarios. However, in recent years, compelling evidence emerges that the performance of GNN-based node classification may deteriorate substantially by topological perturbation, such as random connections or adversarial attacks. Various solutions, such as topological denoising methods and mechanism design methods, have been proposed to develop robust GNN-based node classifiers but none of these works can fully address the problems related to topological perturbations. Recently, the Bayesian label transition model is proposed to tackle this issue but its slow convergence may lead to inferior performance. In this work, we propose a new label inference model, namely LInDT, which integrates both Bayesian label transition and topology-based label propagation for improving the robustness of GNNs against topological perturbations. LInDT is superior to existing label transition methods as it improves the label prediction of uncertain nodes by utilizing neighborhood-based label propagation leading to better convergence of label inference. Besides, LIndT adopts asymmetric Dirichlet distribution as a prior, which also helps it to improve label inference. Extensive experiments on five graph datasets demonstrate the superiority of LInDT for GNN-based node classification under three scenarios of topological perturbations.

Defending Graph Convolutional Networks against Dynamic Graph Perturbations via Bayesian Self-supervision

In recent years, plentiful evidence illustrates that Graph Convolutional Networks (GCNs) achieve extraordinary accomplishments on the node classification task. However, GCNs may be vulnerable to adversarial attacks on label-scarce dynamic graphs. Many existing works aim to strengthen the robustness of GCNs; for instance, adversarial training is used to shield GCNs against malicious perturbations. However, these works fail on dynamic graphs for which label scarcity is a pressing issue. To overcome label scarcity, self-training attempts to iteratively assign pseudo-labels to highly confident unlabeled nodes but such attempts may suffer serious degradation under dynamic graph perturbations. In this paper, we generalize noisy supervision as a kind of self-supervised learning method and then propose a novel Bayesian self-supervision model, namely GraphSS, to address the issue. Extensive experiments demonstrate that GraphSS can not only affirmatively alert the perturbations on dynamic graphs but also effectively recover the prediction of a node classifier when the graph is under such perturbations. These two advantages prove to be generalized over three classic GCNs across five public graph datasets.

Non-Exhaustive Learning Using Gaussian Mixture Generative Adversarial Networks

Supervised learning, while deployed in real-life scenarios, often encounters instances of unknown classes. Conventional algorithms for training a supervised learning model do not provide an option to detect such instances, so they miss-classify such instances with 100% probability. Open Set Recognition (OSR) and Non-Exhaustive Learning (NEL) are potential solutions to overcome this problem. Most existing methods of OSR first classify members of existing classes and then identify instances of new classes. However, many of the existing methods of OSR only makes a binary decision, i.e., they only identify the existence of the unknown class. Hence, such methods cannot distinguish test instances belonging to incremental unseen classes. On the other hand, the majority of NEL methods often make a parametric assumption over the data distribution, which either fail to return good results, due to the reason that real-life complex datasets may not follow a well-known data distribution. In this paper, we propose a new online non-exhaustive learning model, namely, Non-Exhaustive Gaussian Mixture Generative Adversarial Networks (NE-GM-GAN) to address these issues. Our proposed model synthesizes Gaussian mixture based latent representation over a deep generative model, such as GAN, for incremental detection of instances of emerging classes in the test data. Extensive experimental results on several benchmark datasets show that NE-GM-GAN significantly outperforms the state-of-the-art methods in detecting instances of novel classes in streaming data.

Anti-perturbation of Online Social Networks by Graph Label Transition

Numerous popular online social networks (OSN) would classify users into different categories and recommend users to each other with similar interests. A small number of users, so-called perturbators, may perform some types of behaviors, which significantly disturb such an OSN classifier. Manual annotation by OSN administrators is one kind of potential solutions. However, the manual annotation unavoidably brings into noise. Besides, such perturbators are not Sybil users, and therefore their accounts cannot be frozen. To improve the robustness of such an OSN classifier, we generalize this issue as the defense of Graph Convolutional Networks (GCNs) on the node classification task. Most existing defenses on this task can be divided into the adversarial-based method and the detection-based method. The adversarial-based method improves the robustness of GCNs by training with adversarial samples. However, in our case, the perturbators are hard to be distinguished by OSN administrators and thus we cannot use adversarial samples in the training phase. By contrast, the detection-based method aims at detecting the attacker nodes or edges and alleviates the negative impact by removing them. In our scenario, nevertheless, the perturbators are not the attacker and thus cannot be eliminated. Both methods could not solve the aforementioned problems. To address these issues, we propose a novel graph label transition model, named GraphLT, to improve the robustness of the OSN classifier by transiting the node latent representation based on dynamic conditional label transition. Extensive experiments demonstrate that GraphLT can not only considerably enhance the performance of the node classifier in a clean environment but also successfully remedy the classifier with superior performance over competing methods on seven benchmark datasets after graph perturbation.

Geometrically Matched Multi-source Microscopic Image Synthesis Using Bidirectional Adversarial Networks

Microscopic images from different modality can provide more complete experimental information. In practice, biological and physical limitations may prohibit the acquisition of enough microscopic images at a given observation period. Image synthesis is one promising solution. However, most existing data synthesis methods only translate the image from a source domain to a target domain without strong geometric correlations. To address this issue, we propose a novel model to synthesize diversified microscopic images from multi-sources with different geometric features. The application of our model to a 3D live time-lapse embryonic images of C. elegans presents favorable results. To the best of our knowledge, it is the first effort to synthesize microscopic images with strong underlie geometric correlations from multi-source domains that of entirely separated spatial features.