Alert button
Picture for Guanyu Li

Guanyu Li

Alert button

An Unsupervised Domain Adaptation Model based on Dual-module Adversarial Training

Dec 31, 2021
Yiju Yang, Tianxiao Zhang, Guanyu Li, Taejoon Kim, Guanghui Wang

Figure 1 for An Unsupervised Domain Adaptation Model based on Dual-module Adversarial Training
Figure 2 for An Unsupervised Domain Adaptation Model based on Dual-module Adversarial Training
Figure 3 for An Unsupervised Domain Adaptation Model based on Dual-module Adversarial Training
Figure 4 for An Unsupervised Domain Adaptation Model based on Dual-module Adversarial Training

In this paper, we propose a dual-module network architecture that employs a domain discriminative feature module to encourage the domain invariant feature module to learn more domain invariant features. The proposed architecture can be applied to any model that utilizes domain invariant features for unsupervised domain adaptation to improve its ability to extract domain invariant features. We conduct experiments with the Domain-Adversarial Training of Neural Networks (DANN) model as a representative algorithm. In the training process, we supply the same input to the two modules and then extract their feature distribution and prediction results respectively. We propose a discrepancy loss to find the discrepancy of the prediction results and the feature distribution between the two modules. Through the adversarial training by maximizing the loss of their feature distribution and minimizing the discrepancy of their prediction results, the two modules are encouraged to learn more domain discriminative and domain invariant features respectively. Extensive comparative evaluations are conducted and the proposed approach outperforms the state-of-the-art in most unsupervised domain adaptation tasks.

* arXiv admin note: text overlap with arXiv:2108.00610 
Viaarxiv icon

An MAP Estimation for Between-Class Variance

Nov 24, 2021
Jiao Han, Yunqi Cai, Lantian Li, Guanyu Li, Dong Wang

Figure 1 for An MAP Estimation for Between-Class Variance
Figure 2 for An MAP Estimation for Between-Class Variance
Figure 3 for An MAP Estimation for Between-Class Variance
Figure 4 for An MAP Estimation for Between-Class Variance

Probabilistic linear discriminant analysis (PLDA) has been widely used in open-set verification tasks, such as speaker verification. A potential issue of this model is that the training set often contains limited number of classes, which makes the estimation for the between-class variance unreliable. This unreliable estimation often leads to degraded generalization. In this paper, we present an MAP estimation for the between-class variance, by employing an Inverse-Wishart prior. A key problem is that with hierarchical models such as PLDA, the prior is placed on the variance of class means while the likelihood is based on class members, which makes the posterior inference intractable. We derive a simple MAP estimation for such a model, and test it in both PLDA scoring and length normalization. In both cases, the MAP-based estimation delivers interesting performance improvement.

Viaarxiv icon

Attention Boosted Sequential Inference Model

Dec 06, 2018
Guanyu Li, Pengfei Zhang, Caiyan Jia

Figure 1 for Attention Boosted Sequential Inference Model
Figure 2 for Attention Boosted Sequential Inference Model
Figure 3 for Attention Boosted Sequential Inference Model
Figure 4 for Attention Boosted Sequential Inference Model

Attention mechanism has been proven effective on natural language processing. This paper proposes an attention boosted natural language inference model named aESIM by adding word attention and adaptive direction-oriented attention mechanisms to the traditional Bi-LSTM layer of natural language inference models, e.g. ESIM. This makes the inference model aESIM has the ability to effectively learn the representation of words and model the local subsentential inference between pairs of premise and hypothesis. The empirical studies on the SNLI, MultiNLI and Quora benchmarks manifest that aESIM is superior to the original ESIM model.

Viaarxiv icon