Disclaimer: This class is undergoing redevelopment, and the syllabus may change as the semester progresses.
Required readings are mostly drawn from the 3rd (forthcoming) edition the 2nd (2008) edition of Jurafsky and Martin's Speech and Language Processing textbook. Note that the numbering of the chapters has changed between those editions.
Optional readings are often more advanced. "MS" refers to chapters in Manning and Schütze (1999), Foundations of Statistical Natural Language Processing (you may need to use a campus machine to access the links to the chapters below) or to original research papers (you can find many more on the ACL anthology). I also recommend the Handbook of Computational Linguistics and Natural Language Processing (you also need to be on the campus network to access this site).
Lecture slides and Assignments are also linked to on this page. We typically release assignments in the evening of the day they come out. Assignments assume familiarity with Python 3.
| Wed, 08/28 | 01 | Introduction | ||
| What is NLP? What will you learn in this class? How will we teach this class? | ||||
| Required reading: Ch.1 (2nd Ed). | ||||
| Optional reading: Python tutorial (sec. 1-5), Jelinek (2009), Ferrucci et al. (2010) | ||||
| Fri, 08/30 | 02 | Regular Expression and Tokenization | ||
| Review of finite-state automata, Finite-state transducers, tokenization | ||||
| Required reading: Ch. 2 | ||||
| (Video) | 03 | Language Models; Intro to Probability Models for NLP (video) | ||
| Review of basic probability. How do we apply these ideas to NLP? | ||||
| N-gram language models. Evaluation: Perplexity and Word Error Rate. | ||||
| Required reading: Ch. 3 | ||||
| Optional reading: MS, Ch. 2 | ||||
| Wed, 09/11 | 04 | Introduction to Classification for NLP | ||
| Binary and Multiclass classification for NLP. Naive Bayes | ||||
| Required reading: Ch. 4 | ||||
| Fri, 09/13 | 05 | Logistic Regression for NLP | MP1 out | |
| Another way to do classification in NLP. | ||||
| Required reading: Ch. 5 | ||||
| Wed, 09/18 | 06 | From Logistic Regression to Neural Nets | ||
| Feedforward networks, neural language models | ||||
| Required reading: Ch. 7 | ||||
| Fri, 09/20 | 07 | More basic neural nets for NLP | ||
| More on neural models for NLP | ||||
| Required reading: Ch. 7 | ||||
| Wed, 09/25 | 08 | Distributional similarities and Vector Semantics | ||
| Representing words as vectors | ||||
| Required reading: Ch. 6 | ||||
| Fri, 09/27 | 09 | Word Embeddings and basic intro to RNNs | MP2 out | (MP1 due 09/30)|
| Introduction to POS tagging | ||||
| Required reading: Ch. 8 | ||||
| Wed, 10/02 | 10 | Part of Speech Tagging I | ||
| POS tagging | ||||
| Required reading: Ch. 8 | ||||
| Fri, 10/04 | 11 | More on POS tagging and Sequence Labeling | [4Cr] Proposal due | |
| Required reading: Ch. 8 | ||||
| Wed, 10/09 | 12 | Review for midterm | ||
| NB: Please go over the material before class by yourself | ||||
| Fri, 10/11 | 13 | Midterm Exam | ||
| In-class midterm | ||||
| Wed, 10/16 | 15 | Machine Translation I | ||
| Introduction to Machine Translation | ||||
| Fri, 10/18 | 16 | Machine Translation II | MP2 due. MP3 out | |
| More on Machine Translation | ||||
| Optional reading: Brown et al. (1990), Lopez (2008), Koehn et al., | ||||
| Och& Ney (2004), Wu (1997), Chiang (2007) www.statmt.org | ||||
| Wed, 10/23 | 17 | Syntax and Parsing I: Constituencies and Dependencies | ||
| Formal Grammars for English | ||||
| Required reading: Ch 12 | ||||
| Optional reading: MS, Ch. 3, Woods (2010) | ||||
| Fri, 10/25 | 18 | Syntax and Parsing II: Constituency Parsing | ||
| PCFGs and CKY Parsing | ||||
| Required reading: Ch 13 | ||||
| Optional reading: Collins' notes, Chi & Geman (1998), Schabes et al. (1993), | ||||
| Schabes & Pereira (1992), Stolcke (1995), Marcus et al. (1993), Collins (1997), | ||||
| Johnson (1998), Klein & Manning (2003), Petrov & Klein (2007), Hindle & Rooth | ||||
| Fri, 11/01 | 19 | Syntax and Parsing III: Dependency Parsing | ||
| Dependency Grammar and Shift-Reduce Dependency Parsing | ||||
| Required reading: Ch. 13 (3rd ed), McDonald & Nivre (2007) | ||||
| Optional reading: Nivre & Scholz (2004), Kubler et al. (2009), Nivre (2010), McDonald & Nivre (2011) | ||||
| Wed, 11/06 | 20 | Interlude: More on RNNs | ||
| More on RNNs: seq2seq models | ||||
| Required reading: Ch 10 | ||||
| Fri, 11/08 | 21 | Syntax and Parsing IV: Expressive Grammars | [4Cr] Progress Report due | |
| Going beyond CFGs (with a focus on categorial grammars) | ||||
| Optional reading: Abney (1997), Miyao & Tsujii (2008), Joshi and Schabes (1997), | ||||
| Steedman & Baldridge (2011), Schabes & Shieber, Schabes & Waters (1995), | ||||
| Bangalore & Joshi (1999), Hockenmaier & Steedman (2007), Clark & Curran (2007) | ||||
| Wed, 11/13 | 22 | Sentence Semantics I: Compositional Semantics | ||
| What is the meaning of a sentence, and how can we represent it? Basic predicate logic and lambda calculus | ||||
| Required reading: Ch. 14 | ||||
| Optional reading: Blackburn & Bos (2003), The Lambda Calculator | ||||
| Fri, 11/15 | 23 | Thesaurus-based Lexical Semantics | ||
| Lexicographic approaches to Lexical Semantics (WordNet etc.) | ||||
| Required reading: Ch. 19 | ||||
| Wed, 11/20 | 24 | Verb Semantics and Semantic Role Labeling | ||
| Basic intro to verb semantics: events, thematic roles, SRL | ||||
| Required reading: Ch. 20 | ||||
| Fri, 11/22 | 25 | Discourse | ||
| Referring expressions/coreference, rhetorical/discourse relations | ||||
| Required reading: Ch. 22 and Ch. 23 | ||||
| Wed, 12/04 | 27 | Dialogue | ||
| Fri, 12/06 | 28 | Review for final exam | MP4 due. | |
| We'll go over the material after the midterm | ||||
| Wed, 12/11 | 29 | Final exam (in-class) | ||