Course Staff

Logistics

• Office hours: Check the office hours above. If you have special requests, contact the course staff individually.
• Assignments: Assignments should be submitted through Google Drive.
• Discussions: We will be using Slack for general course-related questions and announcements. If you have an individual matter to discuss, email me directly (please put “[CSCI 699]” in the subject line) or come to my office hours.

Prerequisites

• Familiarity with natural language processing and/or machine learning. Ideal pre/co-requisites are CSCI 544 (Applied Natural Language Processing) or CSCI 567 (Machine Learning).
• We will not teach any NLP concepts/techniques in this class!
• Programming skills. We will mainly use python with PyTorch, but you can use any other libraries for your final project.

Schedule

All assignments are due by 11:59pm on the indicated date.

Week	Date	Topic	Related Readings	Assignments
1		Tue Aug 22	Introduction (slides)
		Thu Aug 24	Introduction (slides)
2	Tue Aug 29	Project Examples
		Human Subjects Research	Paper1, Paper2
	Thu Aug 31	Social Biases in NLP	Paper1, Paper2, Paper3
3	Tue Sep 5	Bias Evaluation	Paper1, Paper2, Paper3
	Thu Sep 7	Models vs morality	Paper1, Paper2, Paper3
4	Tue Sep 12	Harms of LLMs in downstream tasks	Paper1, Paper2, Paper3
	Thu Sep 14	Bias Mitigation	Paper1, Paper2, Paper3	Project Proposal deadline
5	Tue Sep 19	Guest Lecture: What happened in Industry Research (Sunipa Dev)	Paper1
	Thu Sep 21	Harms of LLMs: Bias and Stereotype	Paper1, Paper2, Paper3
6	Tue Sep 26	Hate Speech	Paper1, Paper2
		Bias & Stereotype & Harm summary
	Thu Sep 28	Midterm Presentation Workshop
7	Tue Oct 3	Midterm Presentation Workshop
	Thu Oct 5	Guest Lecture: LLM memorization (Eric Wallace)	Paper1
8	Tue Oct 10	Privacy	Paper1, Paper2, Paper3
	Thu Oct 12	Fall Recess (No Class)
9	Tue Oct 17	Human-Centered AI	Paper1, Paper2, Paper3
	Thu Oct 19	Guest Lecture: Human& AI Interaction (Weiyan Shi)
		Bias in Dialogue	Paper1
10	Tue Oct 24	Guest Lecture: Human-Centered AI (Sherry Wu)
		Microaggression	Paper1
	Thu Oct 26	Multilingual Biases	Paper1, Paper2, Paper3	Midterm Report Due
11	Tue Oct 31	Multimodal biases	Paper1, Paper2, Paper3
	Thu Nov 2	Reduce bias in language models	Paper1, Paper2, Paper3
12	Tue Nov 7	Guest Lecture: Interpretation (Hanjie Chen)	Paper1
	Thu Nov 9	Alignment with humans	Paper1, Paper2, Paper3
13	Tue Nov 14	Trade offs between different metrics	Paper1, Paper2, Paper3
	Thu Nov 16	Summary and Final Project Presentations
14	Tue Nov 21	Final Project Presentations
	Thu Nov 23	Thanksgiving (No Class)
15	Tue Nov 28	Final Project Presentations
	Thu Nov 30	Final Project Presentations
16	Tue Dec 5	No Class
	Thu Dec 7	No Class		Project Report Due

Grading

Grades will be based on attendance (10%), paper presentation (30%), and a course project (60%).

Attendance and Discussion (10% total):

• In-Person Attendance

Paper Reading and Discussion (30% total).

• Present paper, lead class discussion
• Signup as reviewers, peer review others’ presentation

Course Project (60% total):

• Project Proposal (10%)
• Midterm Report (10%)
• Final Presentation (20%)
• Final Report (20%)

Late days

You have 4 late days you may use on any assignment. Each late day allows you to submit the assignment 24 hours later than the original deadline. If you are working in a group for the project, submitting the project proposal or midterm report one day late means that each member of the group spends a late day.

Paper Presentation

Paper presentation will help students to develop the skills to give research talk to others. Each student will present 2 papers to the class. The student will prepare the slides for the paper and lead the discussion. Each week, there will be another student signed up as the feedback provider (reviewer). Reviewer will provide the feedback to the instructor or the TAs. Grading rubrics: correctness of the content (40%), clarity (20%), discussion (20%), slides & presentation skills (20%).

Final Project

The final project can be done individually or in groups of up to 3. This is your chance to freely explore machine learning methods and how they can be applied to a task of our choice. You will also learn about best practices for developing machine learning methods—inspecting your data, establishing baselines, and analyzing your errors.

Each group needs to finish one research project related to the class topics. There should be a “deliverable” result out of the project, meaning that your project should be self-complete and reproducible (scientifically correct. A typical successful project could be: 1) a novel and sound solution to an interesting research problem, 2) correct and meaningful comparisons among baselines and existing approaches, 3) applying existing techniques to a new application. We will not penalize negative results, as long as your proposed approach is thoroughly explored and justified. Overall, the project should showcase the student’s ability to think critically, conduct rigorous research, and apply the concepts learned in the course to address a relevant problem in the field of NLP ethics.

Students should use the standard *ACL paper submission template to finish their writing report regarding the course project.

• Project proposal (10%). Students are expected to finish a 2-page long project proposal by Week 4. The proposal should articulate the research question, justify the significance of the research, and provide evidence of the student’s knowledge and understanding of the research literature. A timeline for the project will be highly recommended to be included in the proposal.
• Midterm progress report (10%). By Week 10, the students should finish a ~4-page progress report. The report should provide a clear statement about the research goal (could be different from the original one), a concise overview of the work completed so far, including any challenges encountered and solutions implemented and a report of some initial results.
• Final presentation (20%). During the last two week, the students will make a 30-minute presentation about their project. It should include the research goal, the motivation, related work, their methodology and results. There will be a 5-minute QA session for other students to ask questions.
• Final project report (20%). Students will write a final project report to describe the details about their research. The report should follow the NLP conference paper format, including the abstract, introduction, related work, result demonstration and discussion section. If the result is negative, it won’t be penalized but the students should highlight their analysis about what could be the possible reasons. The report should be in total 8 pages (excluding the reference).

Resources

The following courses are relevant:

• UW: Linguistics 575: Ethics in NLP
• Berkeley: CS 294: Fairness in Machine Learning
• CMU: Computational Ethics for NLP