Zhe Gan

According to our database, Zhe Gan authored at least 156 papers between 2009 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of three.

2024

Multimodal Foundation Models: From Specialists to General-Purpose Assistants.

Chunyuan Li, Zhe Gan, Zhengyuan Yang, Jianwei Yang, Linjie Li, Lijuan Wang, Jianfeng Gao

Found. Trends Comput. Graph. Vis., 2024

Ferret-UI 2: Mastering Universal User Interface Understanding Across Platforms.

Zhangheng Li, Keen You, Haotian Zhang, Di Feng, Harsh Agrawal, Xiujun Li, Mohana Prasad Sathya Moorthy, Jeff Nichols, Yinfei Yang, Zhe Gan

CoRR, 2024

Improve Vision Language Model Chain-of-thought Reasoning.

Ruohong Zhang, Bowen Zhang, Yanghao Li, Haotian Zhang, Zhiqing Sun, Zhe Gan, Yinfei Yang, Ruoming Pang, Yiming Yang

CoRR, 2024

MM-Ego: Towards Building Egocentric Multimodal LLMs.

Hanrong Ye, Haotian Zhang, Erik A. Daxberger, Lin Chen, Zongyu Lin, Yanghao Li, Bowen Zhang, Haoxuan You, Dan Xu, Zhe Gan, Jiasen Lu, Yinfei Yang

CoRR, 2024

Contrastive Localized Language-Image Pre-Training.

Hong-You Chen, Zhengfeng Lai, Haotian Zhang, Xinze Wang, Marcin Eichner, Keen You, Meng Cao, Bowen Zhang, Yinfei Yang, Zhe Gan

CoRR, 2024

Revisit Large-Scale Image-Caption Data in Pre-training Multimodal Foundation Models.

Zhengfeng Lai, Vasileios Saveris, Chen Chen, Hong-You Chen, Haotian Zhang, Bowen Zhang, Juan Lao Tebar, Wenze Hu, Zhe Gan, Peter Grasch, Meng Cao, Yinfei Yang

CoRR, 2024

MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuning.

Haotian Zhang, Mingfei Gao, Zhe Gan, Philipp Dufter, Nina Wenzel, Forrest Huang, Dhruti Shah, Xianzhi Du, Bowen Zhang, Yanghao Li, Sam Dodge, Keen You, Zhen Yang, Aleksei Timofeev, Mingze Xu, Hong-You Chen, Jean-Philippe Fauconnier, Zhengfeng Lai, Haoxuan You, Zirui Wang, Afshin Dehghan, Peter Grasch, Yinfei Yang

CoRR, 2024

SlowFast-LLaVA: A Strong Training-Free Baseline for Video Large Language Models.

Mingze Xu, Mingfei Gao, Zhe Gan, Hong-You Chen, Zhengfeng Lai, Haiming Gang, Kai Kang, Afshin Dehghan

CoRR, 2024

Understanding Alignment in Multimodal LLMs: A Comprehensive Study.

Elmira Amirloo, Jean-Philippe Fauconnier, Christoph Roesmann, Christian Kerl, Rinu Boney, Yusu Qian, Zirui Wang, Afshin Dehghan, Yinfei Yang, Zhe Gan, Peter Grasch

CoRR, 2024

MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMs.

Yusu Qian, Hanrong Ye, Jean-Philippe Fauconnier, Peter Grasch, Yinfei Yang, Zhe Gan

CoRR, 2024

Ferret-v2: An Improved Baseline for Referring and Grounding with Large Language Models.

Haotian Zhang, Haoxuan You, Philipp Dufter, Bowen Zhang, Chen Chen, Hong-You Chen, Tsu-Jui Fu, William Yang Wang, Shih-Fu Chang, Zhe Gan, Yinfei Yang

CoRR, 2024

MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training.

Brandon McKinzie, Zhe Gan, Jean-Philippe Fauconnier, Sam Dodge, Bowen Zhang, Philipp Dufter, Dhruti Shah, Xianzhi Du, Futang Peng, Floris Weers, Anton Belyi, Haotian Zhang, Karanjeet Singh, Doug Kang, Ankur Jain, Hongyu Hè, Max Schwarzer, Tom Gunter, Xiang Kong, Aonan Zhang, Jianyu Wang, Chong Wang, Nan Du, Tao Lei, Sam Wiseman, Guoli Yin, Mark Lee, Zirui Wang, Ruoming Pang, Peter Grasch, Alexander Toshev, Yinfei Yang

CoRR, 2024

How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Prompts.

Yusu Qian, Haotian Zhang, Yinfei Yang, Zhe Gan

CoRR, 2024

Ferret: Refer and Ground Anything Anywhere at Any Granularity.

Haoxuan You, Haotian Zhang, Zhe Gan, Xianzhi Du, Bowen Zhang, Zirui Wang, Liangliang Cao, Shih-Fu Chang, Yinfei Yang

Proceedings of the Twelfth International Conference on Learning Representations, 2024

Compressing LLMs: The Truth is Rarely Pure and Never Simple.

Ajay Kumar Jaiswal, Zhe Gan, Xianzhi Du, Bowen Zhang, Zhangyang Wang, Yinfei Yang

Proceedings of the Twelfth International Conference on Learning Representations, 2024

Guiding Instruction-based Image Editing via Multimodal Large Language Models.

Tsu-Jui Fu, Wenze Hu, Xianzhi Du, William Yang Wang, Yinfei Yang, Zhe Gan

Proceedings of the Twelfth International Conference on Learning Representations, 2024

Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs.

Keen You, Haotian Zhang, Eldon Schoop, Floris Weers, Amanda Swearngin, Jeffrey Nichols, Yinfei Yang, Zhe Gan

Proceedings of the Computer Vision - ECCV 2024, 2024

GRiT: A Generative Region-to-Text Transformer for Object Understanding.

Jialian Wu, Jianfeng Wang, Zhengyuan Yang, Zhe Gan, Zicheng Liu, Junsong Yuan, Lijuan Wang

Proceedings of the Computer Vision - ECCV 2024, 2024

MM1: Methods, Analysis and Insights from Multimodal LLM Pre-training.

Brandon McKinzie, Zhe Gan, Jean-Philippe Fauconnier, Sam Dodge, Bowen Zhang, Philipp Dufter, Dhruti Shah, Xianzhi Du, Futang Peng, Anton Belyi, Haotian Zhang, Karanjeet Singh, Doug Kang, Hongyu Hè, Max Schwarzer, Tom Gunter, Xiang Kong, Aonan Zhang, Jianyu Wang, Chong Wang, Nan Du, Tao Lei, Sam Wiseman, Mark Lee, Zirui Wang, Ruoming Pang, Peter Grasch, Alexander Toshev, Yinfei Yang

Proceedings of the Computer Vision - ECCV 2024, 2024

VeCLIP: Improving CLIP Training via Visual-Enriched Captions.

Zhengfeng Lai, Haotian Zhang, Bowen Zhang, Wentao Wu, Haoping Bai, Aleksei Timofeev, Xianzhi Du, Zhe Gan, Jiulong Shan, Chen-Nee Chuah, Yinfei Yang, Meng Cao

Proceedings of the Computer Vision - ECCV 2024, 2024

Diagnostic Benchmark and Iterative Inpainting for Layout-Guided Image Generation.

Jaemin Cho, Linjie Li, Zhengyuan Yang, Zhe Gan, Lijuan Wang, Mohit Bansal

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

2023

InfoVisDial: An Informative Visual Dialogue Dataset by Bridging Large Multimodal and Language Models.

Bingbing Wen, Zhengyuan Yang, Jianfeng Wang, Zhe Gan, Bill Howe, Lijuan Wang

CoRR, 2023

From Scarcity to Efficiency: Improving CLIP Training via Visual-enriched Captions.

Zhengfeng Lai, Haotian Zhang, Wentao Wu, Haoping Bai, Aleksei Timofeev, Xianzhi Du, Zhe Gan, Jiulong Shan, Chen-Nee Chuah, Yinfei Yang, Meng Cao

CoRR, 2023

MOFI: Learning Image Representations from Noisy Entity Annotated Images.

Wentao Wu, Aleksei Timofeev, Chen Chen, Bowen Zhang, Kun Duan, Shuangning Liu, Yantao Zheng, Jonathon Shlens, Xianzhi Du, Zhe Gan, Yinfei Yang

CoRR, 2023

Prompting GPT-3 To Be Reliable.

Chenglei Si, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan L. Boyd-Graber, Lijuan Wang

Proceedings of the Eleventh International Conference on Learning Representations, 2023

Pre-trained Language Models Do Not Help Auto-regressive Text-to-Image Generation.

Yuhui Zhang, Brandon McKinzie, Zhe Gan, Vaishaal Shankar, Alexander Toshev

Proceedings of the Proceedings on "I Can't Believe It's Not Better: Failure Modes in the Age of Foundation Models" at NeurIPS 2023 Workshops, 2023

An Empirical Study of Multimodal Model Merging.

Yi-Lin Sung, Linjie Li, Kevin Lin, Zhe Gan, Mohit Bansal, Lijuan Wang

Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2023, 2023

Generalized Decoding for Pixel, Image, and Language.

Xueyan Zou, Zi-Yi Dou, Jianwei Yang, Zhe Gan, Linjie Li, Chunyuan Li, Xiyang Dai, Harkirat Behl, Jianfeng Wang, Lu Yuan, Nanyun Peng, Lijuan Wang, Yong Jae Lee, Jianfeng Gao

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023

Non-Contrastive Learning Meets Language-Image Pre-Training.

Jinghao Zhou, Li Dong, Zhe Gan, Lijuan Wang, Furu Wei

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023

ReCo: Region-Controlled Text-to-Image Generation.

Zhengyuan Yang, Jianfeng Wang, Zhe Gan, Linjie Li, Kevin Lin, Chenfei Wu, Nan Duan, Zicheng Liu, Ce Liu, Michael Zeng, Lijuan Wang

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023

LAVENDER: Unifying Video-Language Understanding as Masked Language Modeling.

Linjie Li, Zhe Gan, Kevin Lin, Chung-Ching Lin, Zicheng Liu, Ce Liu, Lijuan Wang

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023

An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modeling.

Tsu-Jui Fu, Linjie Li, Zhe Gan, Kevin Lin, William Yang Wang, Lijuan Wang, Zicheng Liu

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023

2022

GIT: A Generative Image-to-text Transformer for Vision and Language.

Jianfeng Wang, Zhengyuan Yang, Xiaowei Hu, Linjie Li, Kevin Lin, Zhe Gan, Zicheng Liu, Ce Liu, Lijuan Wang

Trans. Mach. Learn. Res., 2022

Adversarial Feature Augmentation and Normalization for Visual Recognition.

Tianlong Chen, Yu Cheng, Zhe Gan, Jianfeng Wang, Lijuan Wang, Jingjing Liu, Zhangyang Wang

Trans. Mach. Learn. Res., 2022

Vision-Language Pre-Training: Basics, Recent Advances, and Future Trends.

Zhe Gan, Linjie Li, Chunyuan Li, Lijuan Wang, Zicheng Liu, Jianfeng Gao

Found. Trends Comput. Graph. Vis., 2022

Exploring Discrete Diffusion Models for Image Captioning.

Zixin Zhu, Yixuan Wei, Jianfeng Wang, Zhe Gan, Zheng Zhang, Le Wang, Gang Hua, Lijuan Wang, Zicheng Liu, Han Hu

CoRR, 2022

K-LITE: Learning Transferable Visual Models with External Knowledge.

Sheng Shen, Chunyuan Li, Xiaowei Hu, Yujia Xie, Jianwei Yang, Pengchuan Zhang, Zhe Gan, Lijuan Wang, Lu Yuan, Ce Liu, Kurt Keutzer, Trevor Darrell, Anna Rohrbach, Jianfeng Gao

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

NUWA-Infinity: Autoregressive over Autoregressive Generation for Infinite Visual Synthesis.

Jian Liang, Chenfei Wu, Xiaowei Hu, Zhe Gan, Jianfeng Wang, Lijuan Wang, Zicheng Liu, Yuejian Fang, Nan Duan

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone.

Zi-Yi Dou, Aishwarya Kamath, Zhe Gan, Pengchuan Zhang, Jianfeng Wang, Linjie Li, Zicheng Liu, Ce Liu, Yann LeCun, Nanyun Peng, Jianfeng Gao, Lijuan Wang

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

UniTAB: Unifying Text and Box Outputs for Grounded Vision-Language Modeling.

Zhengyuan Yang, Zhe Gan, Jianfeng Wang, Xiaowei Hu, Faisal Ahmed, Zicheng Liu, Yumao Lu, Lijuan Wang

Proceedings of the Computer Vision - ECCV 2022, 2022

SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning.

Kevin Lin, Linjie Li, Chung-Ching Lin, Faisal Ahmed, Zhe Gan, Zicheng Liu, Yumao Lu, Lijuan Wang

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022

Injecting Semantic Concepts into End-to-End Image Captioning.

Zhiyuan Fang, Jianfeng Wang, Xiaowei Hu, Lin Liang, Zhe Gan, Lijuan Wang, Yezhou Yang, Zicheng Liu

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022

An Empirical Study of Training End-to-End Vision-and-Language Transformers.

Zi-Yi Dou, Yichong Xu, Zhe Gan, Jianfeng Wang, Shuohang Wang, Lijuan Wang, Chenguang Zhu, Pengchuan Zhang, Lu Yuan, Nanyun Peng, Zicheng Liu, Michael Zeng

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022

Scaling Up Vision-Language Pretraining for Image Captioning.

Xiaowei Hu, Zhe Gan, Jianfeng Wang, Zhengyuan Yang, Zicheng Liu, Yumao Lu, Lijuan Wang

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022

An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA.

Zhengyuan Yang, Zhe Gan, Jianfeng Wang, Xiaowei Hu, Yumao Lu, Zicheng Liu, Lijuan Wang

Proceedings of the Thirty-Sixth AAAI Conference on Artificial Intelligence, 2022

Playing Lottery Tickets with Vision and Language.

Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu

Proceedings of the Thirty-Sixth AAAI Conference on Artificial Intelligence, 2022

Efficient Robust Training via Backward Smoothing.

Jinghui Chen, Yu Cheng, Zhe Gan, Quanquan Gu, Jingjing Liu

Proceedings of the Thirty-Sixth AAAI Conference on Artificial Intelligence, 2022

2021

MLP Architectures for Vision-and-Language Modeling: An Empirical Study.

Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang

CoRR, 2021

VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling.

Tsu-Jui Fu, Linjie Li, Zhe Gan, Kevin Lin, William Yang Wang, Lijuan Wang, Zicheng Liu

CoRR, 2021

Scaling Up Vision-Language Pre-training for Image Captioning.

Xiaowei Hu, Zhe Gan, Jianfeng Wang, Zhengyuan Yang, Zicheng Liu, Yumao Lu, Lijuan Wang

CoRR, 2021

Crossing the Format Boundary of Text and Boxes: Towards Unified Vision-Language Modeling.

Zhengyuan Yang, Zhe Gan, Jianfeng Wang, Xiaowei Hu, Faisal Ahmed, Zicheng Liu, Yumao Lu, Lijuan Wang

CoRR, 2021

UFO: A UniFied TransfOrmer for Vision-Language Representation Learning.

Jianfeng Wang, Xiaowei Hu, Zhe Gan, Zhengyuan Yang, Xiyang Dai, Zicheng Liu, Yumao Lu, Lijuan Wang

CoRR, 2021

Simpler, Faster, Stronger: Breaking The log-K Curse On Contrastive Learners With FlatNCE.

Junya Chen, Zhe Gan, Xuan Li, Qing Guo, Liqun Chen, Shuyang Gao, Tagyoung Chung, Yi Xu, Belinda Zeng, Wenlian Lu, Fan Li, Lawrence Carin, Chenyang Tao

CoRR, 2021

Playing Lottery Tickets with Vision and Language.

Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu

CoRR, 2021

CUPID: Adaptive Curation of Pre-training Data for Video-and-Language Representation Learning.

Luowei Zhou, Jingjing Liu, Yu Cheng, Zhe Gan, Lei Zhang

CoRR, 2021

Ultra-Data-Efficient GAN Training: Drawing A Lottery Ticket First, Then Training It Toughly.

Tianlong Chen, Yu Cheng, Zhe Gan, Jingjing Liu, Zhangyang Wang

CoRR, 2021

Meta Module Network for Compositional Visual Reasoning.

Wenhu Chen, Zhe Gan, Linjie Li, Yu Cheng, William Yang Wang, Jingjing Liu

Proceedings of the IEEE Winter Conference on Applications of Computer Vision, 2021

MaxVA: Fast Adaptation of Step Sizes by Maximizing Observed Variance of Gradients.

Chen Zhu, Yu Cheng, Zhe Gan, Furong Huang, Jingjing Liu, Tom Goldstein

Proceedings of the Machine Learning and Knowledge Discovery in Databases. Research Track, 2021

Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models.

Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Hassan Awadallah, Bo Li

Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, 2021

VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation.

Linjie Li, Jie Lei, Zhe Gan, Licheng Yu, Yen-Chun Chen, Rohit Pillai, Yu Cheng, Luowei Zhou, Xin Wang, William Yang Wang, Tamara L. Berg, Mohit Bansal, Jingjing Liu, Lijuan Wang, Zicheng Liu

Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, 2021

The Elastic Lottery Ticket Hypothesis.

Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Jingjing Liu, Zhangyang Wang

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Chasing Sparsity in Vision Transformers: An End-to-End Exploration.

Tianlong Chen, Yu Cheng, Zhe Gan, Lu Yuan, Lei Zhang, Zhangyang Wang

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspective.

Tianlong Chen, Yu Cheng, Zhe Gan, Jingjing Liu, Zhangyang Wang

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

APo-VAE: Text Generation in Hyperbolic Space.

Shuyang Dai, Zhe Gan, Yu Cheng, Chenyang Tao, Lawrence Carin, Jingjing Liu

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021

Improving Zero-Shot Voice Style Transfer via Disentangled Representation Learning.

Siyang Yuan, Pengyu Cheng, Ruiyi Zhang, Weituo Hao, Zhe Gan, Lawrence Carin

Proceedings of the 9th International Conference on Learning Representations, 2021

InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective.

Boxin Wang, Shuohang Wang, Yu Cheng, Zhe Gan, Ruoxi Jia, Bo Li, Jingjing Liu

Proceedings of the 9th International Conference on Learning Representations, 2021

Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models.

Linjie Li, Jie Lei, Zhe Gan, Jingjing Liu

Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision, 2021

Less Is More: ClipBERT for Video-and-Language Learning via Sparse Sampling.

Jie Lei, Linjie Li, Luowei Zhou, Zhe Gan, Tamara L. Berg, Mohit Bansal, Jingjing Liu

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021

Wasserstein Contrastive Representation Distillation.

Liqun Chen, Dong Wang, Zhe Gan, Jingjing Liu, Ricardo Henao, Lawrence Carin

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021

Cluster-Former: Clustering-based Sparse Transformer for Question Answering.

Shuohang Wang, Luowei Zhou, Zhe Gan, Yen-Chun Chen, Yuwei Fang, Siqi Sun, Yu Cheng, Jingjing Liu

Proceedings of the Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, 2021

EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets.

Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Zhangyang Wang, Jingjing Liu

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 2021

FILTER: An Enhanced Fusion Method for Cross-lingual Language Understanding.

Yuwei Fang, Shuohang Wang, Zhe Gan, Siqi Sun, Jingjing Liu

Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, 2021

2020

A Closer Look at the Robustness of Vision-and-Language Pre-trained Models.

Linjie Li, Zhe Gan, Jingjing Liu

CoRR, 2020

Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding.

Shuohang Wang, Luowei Zhou, Zhe Gan, Yen-Chun Chen, Yuwei Fang, Siqi Sun, Yu Cheng, Jingjing Liu

CoRR, 2020

Accelerating Real-Time Question Answering via Question Generation.

Yuwei Fang, Shuohang Wang, Zhe Gan, Siqi Sun, Jingjing Liu

CoRR, 2020

Adaptive Learning Rates with Maximum Variation Averaging.

Chen Zhu, Yu Cheng, Zhe Gan, Furong Huang, Jingjing Liu, Tom Goldstein

CoRR, 2020

POINTER: Constrained Text Generation via Insertion-based Generative Pre-training.

Yizhe Zhang, Guoyin Wang, Chunyuan Li, Zhe Gan, Chris Brockett, Bill Dolan

CoRR, 2020

Large-Scale Adversarial Training for Vision-and-Language Representation Learning.

Zhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu, Yu Cheng, Jingjing Liu

Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

Sequential Attention GAN for Interactive Image Editing.

Yu Cheng, Zhe Gan, Yitong Li, Jingjing Liu, Jianfeng Gao

Proceedings of the MM '20: The 28th ACM International Conference on Multimedia, 2020

CLUB: A Contrastive Log-ratio Upper Bound of Mutual Information.

Pengyu Cheng, Weituo Hao, Shuyang Dai, Jiachang Liu, Zhe Gan, Lawrence Carin

Proceedings of the 37th International Conference on Machine Learning, 2020

Graph Optimal Transport for Cross-Domain Alignment.

Liqun Chen, Zhe Gan, Yu Cheng, Linjie Li, Lawrence Carin, Jingjing Liu

Proceedings of the 37th International Conference on Machine Learning, 2020

FreeLB: Enhanced Adversarial Training for Natural Language Understanding.

Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, Jingjing Liu

Proceedings of the 8th International Conference on Learning Representations, 2020

POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training.

Yizhe Zhang, Guoyin Wang, Chunyuan Li, Zhe Gan, Chris Brockett, Bill Dolan

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 2020

Cross-Thought for Sentence Encoder Pre-training.

Shuohang Wang, Yuwei Fang, Siqi Sun, Zhe Gan, Yu Cheng, Jingjing Liu, Jing Jiang

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 2020

Contrastive Distillation on Intermediate Representations for Language Model Compression.

Siqi Sun, Zhe Gan, Yuwei Fang, Yu Cheng, Shuohang Wang, Jingjing Liu

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 2020

HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training.

Linjie Li, Yen-Chun Chen, Yu Cheng, Zhe Gan, Licheng Yu, Jingjing Liu

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 2020

Hierarchical Graph Network for Multi-hop Question Answering.

Yuwei Fang, Siqi Sun, Zhe Gan, Rohit Pillai, Shuohang Wang, Jingjing Liu

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 2020

Multi-Fact Correction in Abstractive Text Summarization.

Yue Dong, Shuohang Wang, Zhe Gan, Yu Cheng, Jackie Chi Kit Cheung, Jingjing Liu

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 2020

Contextual Text Style Transfer.

Yu Cheng, Zhe Gan, Yizhe Zhang, Oussama Elachqar, Dianqi Li, Jingjing Liu

Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2020, 2020

UNITER: UNiversal Image-TExt Representation Learning.

Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, Jingjing Liu

Proceedings of the Computer Vision - ECCV 2020, 2020

Behind the Scene: Revealing the Secrets of Pre-trained Vision-and-Language Models.

Jize Cao, Zhe Gan, Yu Cheng, Licheng Yu, Yen-Chun Chen, Jingjing Liu

Proceedings of the Computer Vision - ECCV 2020, 2020

Violin: A Large-Scale Dataset for Video-and-Language Inference.

Jingzhou Liu, Wenhu Chen, Yu Cheng, Zhe Gan, Licheng Yu, Yiming Yang, Jingjing Liu

Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020

BachGAN: High-Resolution Image Synthesis From Salient Object Layout.

Yandong Li, Yu Cheng, Zhe Gan, Licheng Yu, Liqiang Wang, Jingjing Liu

Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020

Nested-Wasserstein Self-Imitation Learning for Sequence Generation.

Ruiyi Zhang, Changyou Chen, Zhe Gan, Zheng Wen, Wenlin Wang, Lawrence Carin

Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics, 2020

Improving Adversarial Text Generation by Modeling the Distant Future.

Ruiyi Zhang, Changyou Chen, Zhe Gan, Wenlin Wang, Dinghan Shen, Guoyin Wang, Zheng Wen, Lawrence Carin

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020

Discourse-Aware Neural Extractive Text Summarization.

Jiacheng Xu, Zhe Gan, Yu Cheng, Jingjing Liu

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020

Distilling Knowledge Learned in BERT for Text Generation.

Yen-Chun Chen, Zhe Gan, Yu Cheng, Jingzhou Liu, Jingjing Liu

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020

MagGAN: High-Resolution Face Attribute Editing with Mask-Guided Generative Adversarial Network.

Yi Wei, Zhe Gan, Wenbo Li, Siwei Lyu, Ming-Ching Chang, Lei Zhang, Jianfeng Gao, Pengchuan Zhang

Proceedings of the Computer Vision - ACCV 2020 - 15th Asian Conference on Computer Vision, Kyoto, Japan, November 30, 2020

Contrastively Smoothed Class Alignment for Unsupervised Domain Adaptation.

Shuyang Dai, Yu Cheng, Yizhe Zhang, Zhe Gan, Jingjing Liu, Lawrence Carin

Proceedings of the Computer Vision - ACCV 2020 - 15th Asian Conference on Computer Vision, Kyoto, Japan, November 30, 2020

Graph-Driven Generative Models for Heterogeneous Multi-Task Learning.

Wenlin Wang, Hongteng Xu, Zhe Gan, Bai Li, Guoyin Wang, Liqun Chen, Qian Yang, Wenqi Wang, Lawrence Carin

Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020

What Makes A Good Story? Designing Composite Rewards for Visual Storytelling.

Junjie Hu, Yu Cheng, Zhe Gan, Jingjing Liu, Jianfeng Gao, Graham Neubig

Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020

2019

Distilling the Knowledge of BERT for Text Generation.

Yen-Chun Chen, Zhe Gan, Yu Cheng, Jingzhou Liu, Jingjing Liu

CoRR, 2019

Discourse-Aware Neural Extractive Model for Text Summarization.

Jiacheng Xu, Zhe Gan, Yu Cheng, Jingjing Liu

CoRR, 2019

FreeLB: Enhanced Adversarial Training for Language Understanding.

Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, Jingjing Liu

CoRR, 2019

UNITER: Learning UNiversal Image-TExt Representations.

Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, Jingjing Liu

CoRR, 2019

Topic-Guided Variational Autoencoders for Text Generation.

Wenlin Wang, Zhe Gan, Hongteng Xu, Ruiyi Zhang, Guoyin Wang, Dinghan Shen, Changyou Chen, Lawrence Carin

CoRR, 2019

Improving Textual Network Learning with Variational Homophilic Embeddings.

Wenlin Wang, Chenyang Tao, Zhe Gan, Guoyin Wang, Liqun Chen, Xinyuan Zhang, Ruiyi Zhang, Qian Yang, Ricardo Henao, Lawrence Carin

Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, 2019

Topic-Guided Variational Auto-Encoder for Text Generation.

Wenlin Wang, Zhe Gan, Hongteng Xu, Ruiyi Zhang, Guoyin Wang, Dinghan Shen, Changyou Chen, Lawrence Carin

Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019

Improving Sequence-to-Sequence Learning via Optimal Transport.

Liqun Chen, Yizhe Zhang, Ruiyi Zhang, Chenyang Tao, Zhe Gan, Haichao Zhang, Bai Li, Dinghan Shen, Changyou Chen, Lawrence Carin

Proceedings of the 7th International Conference on Learning Representations, 2019

Relation-Aware Graph Attention Network for Visual Question Answering.

Linjie Li, Zhe Gan, Yu Cheng, Jingjing Liu

Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, 2019

Adversarial Domain Adaptation for Machine Reading Comprehension.

Huazheng Wang, Zhe Gan, Xiaodong Liu, Jingjing Liu, Jianfeng Gao, Hongning Wang

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019

Patient Knowledge Distillation for BERT Model Compression.

Siqi Sun, Yu Cheng, Zhe Gan, Jingjing Liu

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019

Domain Adaptive Text Style Transfer.

Dianqi Li, Yizhe Zhang, Zhe Gan, Yu Cheng, Chris Brockett, Bill Dolan, Ming-Ting Sun

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019

TIGEr: Text-to-Image Grounding for Image Caption Evaluation.

Ming Jiang, Qiuyuan Huang, Lei Zhang, Xin Wang, Pengchuan Zhang, Zhe Gan, Jana Diesner, Jianfeng Gao

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019

StoryGAN: A Sequential Conditional GAN for Story Visualization.

Yitong Li, Zhe Gan, Yelong Shen, Jingjing Liu, Yu Cheng, Yuexin Wu, Lawrence Carin, David E. Carlson, Jianfeng Gao

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019

Tactical Rewind: Self-Correction via Backtracking in Vision-And-Language Navigation.

Liyiming Ke, Xiujun Li, Yonatan Bisk, Ari Holtzman, Zhe Gan, Jingjing Liu, Jianfeng Gao, Yejin Choi, Siddhartha S. Srinivasa

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019

Multi-step Reasoning via Recurrent Dual Attention for Visual Dialog.

Zhe Gan, Yu Cheng, Ahmed El Kholy, Linjie Li, Jingjing Liu, Jianfeng Gao

Proceedings of the 57th Conference of the Association for Computational Linguistics, 2019

Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation.

Qiuyuan Huang, Zhe Gan, Asli Celikyilmaz, Dapeng Oliver Wu, Jianfeng Wang, Xiaodong He

Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, 2019

2018

Deep Generative Models for Vision and Language Intelligence.

Zhe Gan

PhD thesis, 2018

Sequential Attention GAN for Interactive Image Editing via Dialogue.

Yu Cheng, Zhe Gan, Yitong Li, Jingjing Liu, Jianfeng Gao

CoRR, 2018

Sequence Generation with Guider Network.

Ruiyi Zhang, Changyou Chen, Zhe Gan, Wenlin Wang, Liqun Chen, Dinghan Shen, Guoyin Wang, Lawrence Carin

CoRR, 2018

Adversarial Text Generation via Feature-Mover's Distance.

Liqun Chen, Shuyang Dai, Chenyang Tao, Dinghan Shen, Zhe Gan, Haichao Zhang, Yizhe Zhang, Lawrence Carin

CoRR, 2018

Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization.

Yizhe Zhang, Michel Galley, Jianfeng Gao, Zhe Gan, Xiujun Li, Chris Brockett, Bill Dolan

Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, 2018

Adversarial Text Generation via Feature-Mover's Distance.

Liqun Chen, Shuyang Dai, Chenyang Tao, Haichao Zhang, Zhe Gan, Dinghan Shen, Yizhe Zhang, Guoyin Wang, Ruiyi Zhang, Lawrence Carin

Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, 2018

Multi-Label Learning from Medical Plain Text with Convolutional Residual Models.

Xinyuan Zhang, Ricardo Henao, Zhe Gan, Yitong Li, Lawrence Carin

Proceedings of the Machine Learning for Healthcare Conference, 2018

JointGAN: Multi-Domain Joint Distribution Learning with Generative Adversarial Nets.

Yunchen Pu, Shuyang Dai, Zhe Gan, Weiyao Wang, Guoyin Wang, Yizhe Zhang, Ricardo Henao, Lawrence Carin

Proceedings of the 35th International Conference on Machine Learning, 2018

AttnGAN: Fine-Grained Text to Image Generation With Attentional Generative Adversarial Networks.

Tao Xu, Pengchuan Zhang, Qiuyuan Huang, Han Zhang, Zhe Gan, Xiaolei Huang, Xiaodong He

Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, 2018

Topic Compositional Neural Language Model.

Wenlin Wang, Zhe Gan, Wenqi Wang, Dinghan Shen, Jiaji Huang, Wei Ping, Sanjeev Satheesh, Lawrence Carin

Proceedings of the International Conference on Artificial Intelligence and Statistics, 2018

Adaptive Feature Abstraction for Translating Video to Text.

Yunchen Pu, Martin Renqiang Min, Zhe Gan, Lawrence Carin

Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, 2018

2017

Stein Variational Autoencoder.

Yunchen Pu, Zhe Gan, Ricardo Henao, Chunyuan Li, Shaobo Han, Lawrence Carin

CoRR, 2017

Deconvolutional Paragraph Representation Learning.

Yizhe Zhang, Dinghan Shen, Guoyin Wang, Zhe Gan, Ricardo Henao, Lawrence Carin

Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 2017

Adversarial Symmetric Variational Autoencoder.

Yunchen Pu, Weiyao Wang, Ricardo Henao, Liqun Chen, Zhe Gan, Chunyuan Li, Lawrence Carin

Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 2017

VAE Learning via Stein Variational Gradient Descent.

Yunchen Pu, Zhe Gan, Ricardo Henao, Chunyuan Li, Shaobo Han, Lawrence Carin

Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 2017

Triangle Generative Adversarial Networks.

Zhe Gan, Liqun Chen, Weiyao Wang, Yunchen Pu, Yizhe Zhang, Hao Liu, Chunyuan Li, Lawrence Carin

Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 2017

Adversarial Feature Matching for Text Generation.

Yizhe Zhang, Zhe Gan, Kai Fan, Zhi Chen, Ricardo Henao, Dinghan Shen, Lawrence Carin

Proceedings of the 34th International Conference on Machine Learning, 2017

Stochastic Gradient Monomial Gamma Sampler.

Yizhe Zhang, Changyou Chen, Zhe Gan, Ricardo Henao, Lawrence Carin

Proceedings of the 34th International Conference on Machine Learning, 2017

Adaptive Feature Abstraction for Translating Video to Language.

Yunchen Pu, Martin Renqiang Min, Zhe Gan, Lawrence Carin

Proceedings of the 5th International Conference on Learning Representations, 2017

Adaptive DCTNet for audio signal classification.

Yin Xian, Yunchen Pu, Zhe Gan, Liang Lu, Andrew Thompson

Proceedings of the 2017 IEEE International Conference on Acoustics, 2017

Character-level deep conflation for business data analytics.

Zhe Gan, P. D. Singh, Ameet Joshi, Xiaodong He, Jianshu Chen, Jianfeng Gao, Li Deng

Proceedings of the 2017 IEEE International Conference on Acoustics, 2017

Learning Generic Sentence Representations Using Convolutional Neural Networks.

Zhe Gan, Yunchen Pu, Ricardo Henao, Chunyuan Li, Xiaodong He, Lawrence Carin

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017

Semantic Compositional Networks for Visual Captioning.

Zhe Gan, Chuang Gan, Xiaodong He, Yunchen Pu, Kenneth Tran, Jianfeng Gao, Lawrence Carin, Li Deng

Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017

StyleNet: Generating Attractive Visual Captions with Styles.

Chuang Gan, Zhe Gan, Xiaodong He, Jianfeng Gao, Li Deng

Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017

Scalable Bayesian Learning of Recurrent Neural Networks for Language Modeling.

Zhe Gan, Chunyuan Li, Changyou Chen, Yunchen Pu, Qinliang Su, Lawrence Carin

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017

Unsupervised Learning with Truncated Gaussian Graphical Models.

Qinliang Su, Xuejun Liao, Chunyuan Li, Zhe Gan, Lawrence Carin

Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, 2017

2016

Unsupervised Learning of Sentence Representations using Convolutional Neural Networks.

Zhe Gan, Yunchen Pu, Ricardo Henao, Chunyuan Li, Xiaodong He, Lawrence Carin

CoRR, 2016

Variational Autoencoder for Deep Learning of Images, Labels and Captions.

Yunchen Pu, Zhe Gan, Ricardo Henao, Xin Yuan, Chunyuan Li, Andrew Stevens, Lawrence Carin

Proceedings of the Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, 2016

Factored Temporal Sigmoid Belief Networks for Sequence Learning.

Jiaming Song, Zhe Gan, Lawrence Carin

Proceedings of the 33nd International Conference on Machine Learning, 2016

Learning Weight Uncertainty with Stochastic Gradient MCMC for Shape Classification.

Chunyuan Li, Andrew Stevens, Changyou Chen, Yunchen Pu, Zhe Gan, Lawrence Carin

Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016

Bridging the Gap between Stochastic Gradient MCMC and Stochastic Optimization.

Changyou Chen, David E. Carlson, Zhe Gan, Chunyuan Li, Lawrence Carin

Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, 2016

Inference of gene networks associated with the host response to infectious disease.

Zhe Gan, Xin Yuan, Ricardo Henao, Ephraim L. Tsalik, Lawrence Carin

Proceedings of the Big Data over Networks, 2016

2015

Deep Poisson Factor Modeling.

Ricardo Henao, Zhe Gan, James Lu, Lawrence Carin

Proceedings of the Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, 2015

Deep Temporal Sigmoid Belief Networks for Sequence Modeling.

Zhe Gan, Chunyuan Li, Ricardo Henao, David E. Carlson, Lawrence Carin

Proceedings of the Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, 2015

Scalable Deep Poisson Factor Analysis for Topic Modeling.

Zhe Gan, Changyou Chen, Ricardo Henao, David E. Carlson, Lawrence Carin

Proceedings of the 32nd International Conference on Machine Learning, 2015

Learning Deep Sigmoid Belief Networks with Data Augmentation.

Zhe Gan, Ricardo Henao, David E. Carlson, Lawrence Carin

Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, 2015

2009

A General Geo-spatial Multi-scale Conceptual Model for Automatic Generalization.

Yaoming Yang, Xiaoan Tang, Zhe Gan, Haiyan Jiang

Proceedings of the 2009 International Conference on Environmental Science and Information Application Technology, 2009

Research on the Integration Techniques of Task-Oriented Geospatial Information Service for Battlefield.

Zhe Gan, Xiaoan Tang, Yaomin Yang, Haiyan Yang, Maoyin Sun

Proceedings of the 2009 International Conference on Environmental Science and Information Application Technology, 2009