📰 Recent News

• I will be at EMNLP from Nov. 11 - Nov. 16. Hope to chat with you!
• [Nov. 1] I gave a talk at UCLA NLP Seminar. It was great to see some old and new friends there!
• [10/07/2024] Please check our papers that are going to be presented at EMNLP 2024:
  • InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Context
  • Self-Contradictory Reasoning Evaluation and Detection
  • Images Speak Louder than Words: Understanding and Mitigating Bias in Vision-Language Model from a Causal Mediation Perspective
  • "You Gotta be a Doctor, Lin": An Investigation of Name-Based Bias of Large Language Models in Employment Recommendations
• [09/21/2024] I'm giving a talk at Human-Center AI conference. Thanks for the invitation!
• [09/19/2024] Congrats for the collaborators for the acceptance at EMNLP! See everyone there!
• [2024 August] Xiaoyuan Yi and I gave a tutorial on LLM Alignment at IJCAI 2024! The slides can be accessed here.

Publications (selected)

[Google Scholar] [Full List]

• InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Context
  Ziyi Liu, Abhishek Anand, Pei Zhou, Jen-tse Huang, Jieyu Zhao. EMNLP 2024.

• Self-Contradictory Reasoning Evaluation and Detection
  Ziyi Liu, Soumya Sanyal, Isabelle Lee, Yongkang Du, Rahul Gupta, Yang Liu, Jieyu Zhao. EMNLP 2024 Findings.

• Images Speak Louder than Words: Understanding and Mitigating Bias in Vision-Language Model from a Causal Mediation Perspective
  Zhaotian Weng, Zijun Gao, Jerone Andrews, Jieyu Zhao. EMNLP 2024.

• "You Gotta be a Doctor, Lin": An Investigation of Name-Based Bias of Large Language Models in Employment Recommendations
  Huy Nghiem, John Prindle, Jieyu Zhao, Hal Daumé III. EMNLP 2024.

• TrustLLM: Trustworthiness in Large Language Models
  Lichao Sun, Yue Huang, Haoran Wang, Siyuan Wu, ... , Jieyu Zhao, ..., Yan Liu, Yanfang Ye, Yinzhi Cao, Yong Chen, Yue Zhao. ICML 2024.

• A dynamic approach to long-term fairness in sequential decision-making
  Yuancheng Xu, Chenghao Deng, Yanchao Sun, Ruijie Zheng, Xiyao Wang, Jieyu Zhao, Furong Huang. ICML 2024.

• Fair Abstractive Summarization of Diverse Perspectives
  Yusen Zhang, Nan Zhang, Yixin Liu, Alexander Fabbri, Junru Liu, Ryo Kamoi, Xiaoxin Lu, Caiming Xiong, Jieyu Zhao, Dragomir Radev, Kathleen McKeown, Rui Zhang. NAACL 2024.

• Safer-Instruct: Aligning Language Models with Automated Preference Data
  Taiwei Shi, Kai Chen, Jieyu Zhao. NAACL 2024.

• A Rose by Any Other Name would not Smell as Sweet: Social Bias in Name Mistranslations
  Sandra Sandoval, Jieyu Zhao, Marine Carpuat, and Hal Daume III. EMNLP 2023.

• Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems
  Yixin Wan, Jieyu Zhao, Aman Chadha, Nanyun Peng, and Kai-Wei Chang. EMNLP-Finding, 2023.

• Mind What You Measure For: A Study on Reliability of Prompt-Based Bias Measurement
  Ruyuan Zuo, and Jieyu Zhao. WiNLP 2023.

• TACO: Temporal Latent Action-Driven Contrastive Loss For Visual Reinforcement Learning

  Ruijie Zheng, Xiyao Wang, Yanchao Sun, Shuang Ma, Jieyu Zhao, Huazhe Xu, Hal Daume, Furong Huang. NeurIPS, 2023.

• SODAPOP: Open-Ended Discovery of Social Biases in Social Commonsense Reasoning Models
  Haozhe An, Zongxia Li, Jieyu Zhao, Rachel Rudinger. EACL 2023.

• Investigating Ensemble Methods for Model Robustness Improvement of Text Classifiers
  Jieyu Zhao, Xuezhi Wang, Yao Qin, Jilin Chen, Kai-Wei Chang. EMNLP Findings, 2022.

• Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions?
  Jieyu Zhao, Daniel Khashabi, Tushar Khot, Ashish Sabharwal, and Kai-Wei Chang. ACL Findings, 2021.

• Double Perturbation: On the Robustness of Robustness and Counterfactual Bias Evaluation
  Chong Zhang, Jieyu Zhao, Huan Zhang, Kai-Wei Chang, and Cho-Jui Hsieh. NAACL 2021.

• Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer

  Jieyu Zhao, Subhabrata Mukherjee, Saghar Hosseini, Kai-Wei Chang, Ahmed Hassan Awadallah.

  Conference of the Association for Computational Linguistics. ACL 2020.

• Mitigating Gender Bias Amplification in Distribution by Posterior Regularization

  Shengyu Jia*, Tao Meng*, Jieyu Zhao, Kai-Wei Chang.

  Conference of the Association for Computational Linguistics. ACL 2020.

• Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations

  Tianlu Wang, Jieyu Zhao, Mark Yatskar, Kai-Wei Chang, Vicente Ordonez.

  International Conference on Computer Vision. ICCV 2019.

• Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods [code] [podcast]

  Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, Kai-Wei Chang.

  North American Chapter of the Association for Computational Linguistics. NAACL 2018.

• Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints [code], [slides]

  Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, Kai-Wei Chang.

  Conference on Empirical Methods in Natural Language Processing. EMNLP 2017. (Best Long Paper Award)

  Press: Wired: Machines Taught By Photos Learn a Sexist View of Women

Talks

• Tutorial: AI Meets Values: History, Essence, and Recent Advances of Big Model's Value Alignment. IJCAI 2024.
• Tutorial: Trustworthy and Responsible AI. AAAI 2023.
• Detect and mitigate bias in NLP. AKBC KG-BIAS work shop, 2020.

Education / Experience

• Postdoc, University of Maryland, College Park. 2022.01 - 2023.06
• Ph.D., University of California, Los Angeles. 2017.9 - 2021.12
• Google Research Intern, NYC. 2021.06 - 2021.09
• AI2 Research Intern, Seattle. 2020.09 - 2020.12
• Microsoft Research Intern, Redmond. 2020.06 - 2020.09
• Microsoft Research Intern, Redmond. 2019.06 - 2019.09

Awards (selected)

• CI Fellowship, 2021
• EECS Rising Star, 2021
• Microsoft PhD Fellowship, 2020
• SoCalNLP Symposium Best Poster Award, 2018
• UCLA Graduate Division Fellowships, 2017
• EMNLP 2017 Best Long Paper Award, 2017
• Outstanding Graduate of Beijing City, 2016