Natural Language Processing

Spring 2024, Mon/Wed, 6:30-8:00 PM, Dwinelle 145

Welcome to Natural Language Processing (NLP)! This course introduces students to NLP and exposes them to the variety of methods available for reasoning about text in computational systems. NLP is deeply interdisciplinary, drawing on both linguistics and computer science, and helps drive much contemporary work in text analysis. We will focus on major algorithms used in NLP for various applications (text classification, parsing, question answering, machine translation, etc.) and on the linguistic phenomena those algorithms attempt to model. Students will implement algorithms and create linguistically annotated data on which those algorithms depend*.

Textbooks

  • [SLP3] Dan Jurafsky and James Martin, Speech and Language Processing (3nd ed. draft); Available here
  • [NNNLP] Yoav Goldberg, Neural Network Methods for Natural Language Processing (2017); Available on campus/VPN here
  • [NLA] James Pustejovsky and Amber Stubbs, Natural Language Annotation for Machine Learning (2012); Available on campus/VPN here

Grade Distribution

INFO 159:

  • Homework: 25%
  • Annotation Project: 25%
  • Weekly Quizzes: 10%
  • Exams = max(exam1, exam2): 20%
  • NLP subfield survey: 20%

INFO 259:

  • Homework: 20%
  • Annotation Project: 20%
  • Weekly Quizzes: 10%
  • Exams = max(exam1, exam2): 20%
  • Final Project: 30%
    • Proposal: 5%
    • Midterm report: 5%
    • Final report: 15%
    • Presentation: 5%

Course Calendar**

Date
Topic/Slides
Readings
Jan. 17
Introduction
Browse NLP-progress
Jan. 22
Words, Embeddings
SLP Ch 6
Jan. 24
Text Classification 1: Logistic Regression
SLP Ch 5
Jan. 29
Text Classification 2: Feed-forward, Convolution Neural Networks
SLP Ch 7.1:7.6, NNLP Ch 13.
Jan. 31
Text Classification 3: Attention & Transformers
SLP Ch 10
Feb. 5
Corpora & Annotation
NLA Ch 6
Feb. 7
Language Modeling 1: Ngram and Log Linear Models
SLP Ch 3
Feb. 12
Language Modeling 2: Contextual Embeddings
SLP Ch 10
Feb. 14
Language Modeling 3: BERT, GPT, Prompting
Xie and Min 2022
Feb. 19
Presidents’ Day (No Class)
Feb. 21
Exam 1
Feb. 26
Statistical Sequence Labeling
SLP Ch 8
Feb. 28
Neural Sequence Labeling
SLP Ch 9.3-9.6
Mar. 4
Syntax & CFG Parsing
SLP Ch 17
Mar. 6
Dependency Parsing
SLP Ch 18
Mar. 11
Semantic Role Labeling
SLP Ch 24
Mar. 13
Wordnet, WSD, Supersenses
SLP Ch 23
Mar. 18
Coreference Resolution
SLP Ch 26
Mar. 20
Exam 2
Mar. 25
Spring Break
Mar. 27
Spring Break
Apr. 1
Information Extraction
SLP Ch 19
Apr. 3
Question Answering
SLP Ch 14
Apr. 8
Machine Translation
SLP Ch 13
Apr. 10
Text Generation: Dialogue
SLP Ch 15
Apr. 15
Ethics & Safety in NLP+Guest Lec:Olya Gurevich Co-founder PaperMoon AI
Bender et al, 2021
Apr 17
NLP For Low Resource Languages+Guest Lec: Sara Hooker, VP of Research-Cohere
Hedderich et al, 2021
Apr. 22
More NLP Applications and Conclusion
Talk by Dan Klein:Are LLMs the Beginning or End of NLP?
Apr. 24
Final Project Presentations

*A major part of course design and contents have been drawn from its earlier offerings by Prof. David Bamman.

**Course calendar is subject to change.