Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox
Picture for Paul Pu Liang

Paul Pu Liang

MultiBench: Multiscale Benchmarks for Multimodal Representation Learning


Jul 15, 2021
Paul Pu Liang, Yiwei Lyu, Xiang Fan, Zetian Wu, Yun Cheng, Jason Wu, Leslie Chen, Peter Wu, Michelle A. Lee, Yuke Zhu, Ruslan Salakhutdinov, Louis-Philippe Morency

Figure 1 for MultiBench: Multiscale Benchmarks for Multimodal Representation Learning
Figure 2 for MultiBench: Multiscale Benchmarks for Multimodal Representation Learning
* Code: https://github.com/pliang279/MultiBench and Website: https://cmu-multicomp-lab.github.io/multibench/ 
  Access Paper or Ask Questions

Towards Understanding and Mitigating Social Biases in Language Models


Jun 24, 2021
Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, Ruslan Salakhutdinov

Figure 1 for Towards Understanding and Mitigating Social Biases in Language Models
Figure 2 for Towards Understanding and Mitigating Social Biases in Language Models
Figure 3 for Towards Understanding and Mitigating Social Biases in Language Models
Figure 4 for Towards Understanding and Mitigating Social Biases in Language Models
* ICML 2021, code available at https://github.com/pliang279/LM_bias 
  Access Paper or Ask Questions

Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data


Jun 24, 2021
Paul Pu Liang, Terrance Liu, Anna Cai, Michal Muszynski, Ryo Ishii, Nicholas Allen, Randy Auerbach, David Brent, Ruslan Salakhutdinov, Louis-Philippe Morency

Figure 1 for Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data
Figure 2 for Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data
Figure 3 for Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data
Figure 4 for Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data
* ACL 2021. arXiv admin note: substantial text overlap with arXiv:2012.02359 
  Access Paper or Ask Questions

Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning


Jun 10, 2021
Liangqiong Qu, Yuyin Zhou, Paul Pu Liang, Yingda Xia, Feifei Wang, Li Fei-Fei, Ehsan Adeli, Daniel Rubin

Figure 1 for Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning
Figure 2 for Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning
Figure 3 for Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning
Figure 4 for Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning

  Access Paper or Ask Questions

Ask & Explore: Grounded Question Answering for Curiosity-Driven Exploration


Apr 24, 2021
Jivat Neet Kaur, Yiding Jiang, Paul Pu Liang

Figure 1 for Ask & Explore: Grounded Question Answering for Curiosity-Driven Exploration
Figure 2 for Ask & Explore: Grounded Question Answering for Curiosity-Driven Exploration
Figure 3 for Ask & Explore: Grounded Question Answering for Curiosity-Driven Exploration
Figure 4 for Ask & Explore: Grounded Question Answering for Curiosity-Driven Exploration
* Accepted at ICLR 2021 Workshop on Embodied Multimodal Learning 
  Access Paper or Ask Questions

StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer


Apr 12, 2021
Yiwei Lyu, Paul Pu Liang, Hai Pham, Eduard Hovy, Barnabás Póczos, Ruslan Salakhutdinov, Louis-Philippe Morency

Figure 1 for StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer
Figure 2 for StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer
Figure 3 for StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer
Figure 4 for StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer
* NAACL 2021, code available at https://github.com/lvyiwei1/StylePTB/ 
  Access Paper or Ask Questions

Understanding the Tradeoffs in Client-Side Privacy for Speech Recognition


Jan 22, 2021
Peter Wu, Paul Pu Liang, Ruslan Salakhutdinov, Louis-Philippe Morency

Figure 1 for Understanding the Tradeoffs in Client-Side Privacy for Speech Recognition
Figure 2 for Understanding the Tradeoffs in Client-Side Privacy for Speech Recognition
Figure 3 for Understanding the Tradeoffs in Client-Side Privacy for Speech Recognition
Figure 4 for Understanding the Tradeoffs in Client-Side Privacy for Speech Recognition

  Access Paper or Ask Questions

Cross-Modal Generalization: Learning in Low Resource Modalities via Meta-Alignment


Dec 04, 2020
Paul Pu Liang, Peter Wu, Liu Ziyin, Louis-Philippe Morency, Ruslan Salakhutdinov

Figure 1 for Cross-Modal Generalization: Learning in Low Resource Modalities via Meta-Alignment
Figure 2 for Cross-Modal Generalization: Learning in Low Resource Modalities via Meta-Alignment
Figure 3 for Cross-Modal Generalization: Learning in Low Resource Modalities via Meta-Alignment
Figure 4 for Cross-Modal Generalization: Learning in Low Resource Modalities via Meta-Alignment

  Access Paper or Ask Questions

Multimodal Privacy-preserving Mood Prediction from Mobile Data: A Preliminary Study


Dec 04, 2020
Terrance Liu, Paul Pu Liang, Michal Muszynski, Ryo Ishii, David Brent, Randy Auerbach, Nicholas Allen, Louis-Philippe Morency

Figure 1 for Multimodal Privacy-preserving Mood Prediction from Mobile Data: A Preliminary Study
Figure 2 for Multimodal Privacy-preserving Mood Prediction from Mobile Data: A Preliminary Study
Figure 3 for Multimodal Privacy-preserving Mood Prediction from Mobile Data: A Preliminary Study
Figure 4 for Multimodal Privacy-preserving Mood Prediction from Mobile Data: A Preliminary Study

  Access Paper or Ask Questions

An Investigation of how Label Smoothing Affects Generalization


Oct 23, 2020
Blair Chen, Liu Ziyin, Zihao Wang, Paul Pu Liang

Figure 1 for An Investigation of how Label Smoothing Affects Generalization
Figure 2 for An Investigation of how Label Smoothing Affects Generalization
Figure 3 for An Investigation of how Label Smoothing Affects Generalization
Figure 4 for An Investigation of how Label Smoothing Affects Generalization

  Access Paper or Ask Questions

Towards Debiasing Sentence Representations


Jul 16, 2020
Paul Pu Liang, Irene Mengze Li, Emily Zheng, Yao Chong Lim, Ruslan Salakhutdinov, Louis-Philippe Morency

Figure 1 for Towards Debiasing Sentence Representations
Figure 2 for Towards Debiasing Sentence Representations
Figure 3 for Towards Debiasing Sentence Representations
Figure 4 for Towards Debiasing Sentence Representations
* ACL 2020, code available at https://github.com/pliang279/sent_debias 
  Access Paper or Ask Questions

Diverse and Admissible Trajectory Forecasting through Multimodal Context Understanding


Apr 03, 2020
Seong Hyeon Park, Gyubok Lee, Manoj Bhat, Jimin Seo, Minseok Kang, Jonathan Francis, Ashwin R. Jadhav, Paul Pu Liang, Louis-Philippe Morency

Figure 1 for Diverse and Admissible Trajectory Forecasting through Multimodal Context Understanding
Figure 2 for Diverse and Admissible Trajectory Forecasting through Multimodal Context Understanding
Figure 3 for Diverse and Admissible Trajectory Forecasting through Multimodal Context Understanding
Figure 4 for Diverse and Admissible Trajectory Forecasting through Multimodal Context Understanding

  Access Paper or Ask Questions

Anchor & Transform: Learning Sparse Representations of Discrete Objects


Mar 18, 2020
Paul Pu Liang, Manzil Zaheer, Yuan Wang, Amr Ahmed

Figure 1 for Anchor & Transform: Learning Sparse Representations of Discrete Objects
Figure 2 for Anchor & Transform: Learning Sparse Representations of Discrete Objects
Figure 3 for Anchor & Transform: Learning Sparse Representations of Discrete Objects
Figure 4 for Anchor & Transform: Learning Sparse Representations of Discrete Objects

  Access Paper or Ask Questions

On Emergent Communication in Competitive Multi-Agent Teams


Mar 04, 2020
Paul Pu Liang, Jeffrey Chen, Ruslan Salakhutdinov, Louis-Philippe Morency, Satwik Kottur

Figure 1 for On Emergent Communication in Competitive Multi-Agent Teams
Figure 2 for On Emergent Communication in Competitive Multi-Agent Teams
Figure 3 for On Emergent Communication in Competitive Multi-Agent Teams
Figure 4 for On Emergent Communication in Competitive Multi-Agent Teams
* AAMAS 2020 
  Access Paper or Ask Questions

Learning Not to Learn in the Presence of Noisy Labels


Feb 16, 2020
Liu Ziyin, Blair Chen, Ru Wang, Paul Pu Liang, Ruslan Salakhutdinov, Louis-Philippe Morency, Masahito Ueda

Figure 1 for Learning Not to Learn in the Presence of Noisy Labels
Figure 2 for Learning Not to Learn in the Presence of Noisy Labels
Figure 3 for Learning Not to Learn in the Presence of Noisy Labels
Figure 4 for Learning Not to Learn in the Presence of Noisy Labels

  Access Paper or Ask Questions

Think Locally, Act Globally: Federated Learning with Local and Global Representations


Jan 06, 2020
Paul Pu Liang, Terrance Liu, Liu Ziyin, Ruslan Salakhutdinov, Louis-Philippe Morency

Figure 1 for Think Locally, Act Globally: Federated Learning with Local and Global Representations
Figure 2 for Think Locally, Act Globally: Federated Learning with Local and Global Representations
Figure 3 for Think Locally, Act Globally: Federated Learning with Local and Global Representations
Figure 4 for Think Locally, Act Globally: Federated Learning with Local and Global Representations
* Workshop on Federated Learning for Data Privacy and Confidentiality, NeurIPS 2019, Vancouver, Canada 
  Access Paper or Ask Questions

Factorized Multimodal Transformer for Multimodal Sequential Learning


Nov 22, 2019
Amir Zadeh, Chengfeng Mao, Kelly Shi, Yiwei Zhang, Paul Pu Liang, Soujanya Poria, Louis-Philippe Morency

Figure 1 for Factorized Multimodal Transformer for Multimodal Sequential Learning
Figure 2 for Factorized Multimodal Transformer for Multimodal Sequential Learning
Figure 3 for Factorized Multimodal Transformer for Multimodal Sequential Learning
Figure 4 for Factorized Multimodal Transformer for Multimodal Sequential Learning

  Access Paper or Ask Questions

Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization


Jul 01, 2019
Paul Pu Liang, Zhun Liu, Yao-Hung Hubert Tsai, Qibin Zhao, Ruslan Salakhutdinov, Louis-Philippe Morency

Figure 1 for Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization
Figure 2 for Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization
Figure 3 for Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization

  Access Paper or Ask Questions

Deep Gamblers: Learning to Abstain with Portfolio Theory


Jun 29, 2019
Liu Ziyin, Zhikang Wang, Paul Pu Liang, Ruslan Salakhutdinov, Louis-Philippe Morency, Masahito Ueda

Figure 1 for Deep Gamblers: Learning to Abstain with Portfolio Theory
Figure 2 for Deep Gamblers: Learning to Abstain with Portfolio Theory
Figure 3 for Deep Gamblers: Learning to Abstain with Portfolio Theory
Figure 4 for Deep Gamblers: Learning to Abstain with Portfolio Theory

  Access Paper or Ask Questions

Multimodal Transformer for Unaligned Multimodal Language Sequences


Jun 01, 2019
Yao-Hung Hubert Tsai, Shaojie Bai, Paul Pu Liang, J. Zico Kolter, Louis-Philippe Morency, Ruslan Salakhutdinov

Figure 1 for Multimodal Transformer for Unaligned Multimodal Language Sequences
Figure 2 for Multimodal Transformer for Unaligned Multimodal Language Sequences
Figure 3 for Multimodal Transformer for Unaligned Multimodal Language Sequences
Figure 4 for Multimodal Transformer for Unaligned Multimodal Language Sequences

  Access Paper or Ask Questions

Strong and Simple Baselines for Multimodal Utterance Embeddings


May 14, 2019
Paul Pu Liang, Yao Chong Lim, Yao-Hung Hubert Tsai, Ruslan Salakhutdinov, Louis-Philippe Morency

Figure 1 for Strong and Simple Baselines for Multimodal Utterance Embeddings
Figure 2 for Strong and Simple Baselines for Multimodal Utterance Embeddings
Figure 3 for Strong and Simple Baselines for Multimodal Utterance Embeddings
Figure 4 for Strong and Simple Baselines for Multimodal Utterance Embeddings
* NAACL 2019 
  Access Paper or Ask Questions

Variational Auto-Decoder


Apr 03, 2019
Amir Zadeh, Yao-Chong Lim, Paul Pu Liang, Louis-Philippe Morency

Figure 1 for Variational Auto-Decoder
Figure 2 for Variational Auto-Decoder
Figure 3 for Variational Auto-Decoder
Figure 4 for Variational Auto-Decoder
* Link to code and data available from https://github.com/A2Zadeh/Variational-Autodecoder 
  Access Paper or Ask Questions

An Empirical Evaluation of Sketched SVD and its Application to Leverage Score Ordering


Dec 19, 2018
Hui Han Chin, Paul Pu Liang

Figure 1 for An Empirical Evaluation of Sketched SVD and its Application to Leverage Score Ordering
Figure 2 for An Empirical Evaluation of Sketched SVD and its Application to Leverage Score Ordering
Figure 3 for An Empirical Evaluation of Sketched SVD and its Application to Leverage Score Ordering
Figure 4 for An Empirical Evaluation of Sketched SVD and its Application to Leverage Score Ordering
* ACML 2018 
  Access Paper or Ask Questions

Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities


Dec 19, 2018
Hai Pham, Paul Pu Liang, Thomas Manzini, Louis-Philippe Morency, Barnabas Poczos

Figure 1 for Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities
Figure 2 for Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities
Figure 3 for Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities
Figure 4 for Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities
* AAAI 2019 
  Access Paper or Ask Questions

Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors


Nov 26, 2018
Yansen Wang, Ying Shen, Zhun Liu, Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency

Figure 1 for Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors
Figure 2 for Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors
Figure 3 for Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors
Figure 4 for Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors
* Accepted by AAAI2019 
  Access Paper or Ask Questions

Multimodal Language Analysis with Recurrent Multistage Fusion


Aug 12, 2018
Paul Pu Liang, Ziyin Liu, Amir Zadeh, Louis-Philippe Morency

Figure 1 for Multimodal Language Analysis with Recurrent Multistage Fusion
Figure 2 for Multimodal Language Analysis with Recurrent Multistage Fusion
Figure 3 for Multimodal Language Analysis with Recurrent Multistage Fusion
Figure 4 for Multimodal Language Analysis with Recurrent Multistage Fusion
* EMNLP 2018 
  Access Paper or Ask Questions

Multimodal Local-Global Ranking Fusion for Emotion Recognition


Aug 12, 2018
Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency

Figure 1 for Multimodal Local-Global Ranking Fusion for Emotion Recognition
Figure 2 for Multimodal Local-Global Ranking Fusion for Emotion Recognition
Figure 3 for Multimodal Local-Global Ranking Fusion for Emotion Recognition
Figure 4 for Multimodal Local-Global Ranking Fusion for Emotion Recognition
* ACM International Conference on Multimodal Interaction (ICMI 2018) 
  Access Paper or Ask Questions

Seq2Seq2Sentiment: Multimodal Sequence to Sequence Models for Sentiment Analysis


Aug 06, 2018
Hai Pham, Thomas Manzini, Paul Pu Liang, Barnabas Poczos

Figure 1 for Seq2Seq2Sentiment: Multimodal Sequence to Sequence Models for Sentiment Analysis
Figure 2 for Seq2Seq2Sentiment: Multimodal Sequence to Sequence Models for Sentiment Analysis
Figure 3 for Seq2Seq2Sentiment: Multimodal Sequence to Sequence Models for Sentiment Analysis
Figure 4 for Seq2Seq2Sentiment: Multimodal Sequence to Sequence Models for Sentiment Analysis
* 8 pages of content, 11 pages total, 2 figures. Published as a workshop paper at ACL 2018, Proceedings of Grand Challenge and Workshop on Human Multimodal Language (Challenge-HML). 2018 
  Access Paper or Ask Questions

Learning Factorized Multimodal Representations


Jun 16, 2018
Yao-Hung Hubert Tsai, Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency, Ruslan Salakhutdinov

Figure 1 for Learning Factorized Multimodal Representations
Figure 2 for Learning Factorized Multimodal Representations
Figure 3 for Learning Factorized Multimodal Representations
Figure 4 for Learning Factorized Multimodal Representations

  Access Paper or Ask Questions