Resources

Useful Machine Learning Textbooks

• Mathematics for Machine Learning
• A First Course in Machine Learning
• Deep Learning (2016)
• Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow
• Speech and Language Processing. Dan Juraksky and James Martin. (2025) 3rd ed.

Neural Networks

Papers

• Learning Representations by back-propagating errors. Rumelhart, David E; Hinton, Geoffrey E, and Williams, Ronald J (Sept. 1985)
• Automatic differentiation in machine learning: a survey. Atilim Gunes Baydin et al. (2018)

Best Practices in Deep Learning

Papers

• Layer Normalization. Jimmy Lei Ba et al. (2016)

Language and Sequence Modeling

Papers

• Finding Structure in Time. Jeffrey L. Elman (1990)
  Early paper that introduced RNNs.

Other

• [Blog] The Unreasonable Effectiveness of Recurrent Neural Networks

• A Neural Probablistic Language Model Yoshua Bengio et al (2003).
  Contributions: Proposed using neural networks on a large scale for language modeling instead of previous approaches, i.e., n-grams. Does some funky stuff the direct concantenation of embeddings before the softmax layer and uses a fixed sliding window.

Transformers and Pretraining

Papers

• Transformer Attention is All You Need
• BERT BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• T5 Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel et al. (2019)

Other

• The Annotated Transformer
• Stanford CS224N Notes on Transformers
• [Blog] The Illustrated BERT, ELMo, and co. by Jay Alammar

Algorithms for Natural Language Processing

Papers

• A Simple LSTM model for Transition-based Dependency Parsing. Mohab Elkaref and Bernd Bohnet (2017)
• Deep Unordered Composition Rivals Syntactic Methods for Text Classification. Iyyer, Mohit (2015)

Courses

• [Blog] NLP Course | For You
  Excellent NLP course for fundamental topics; draws heavily from research papers +1.
• [Blog] Modern Approaches in Natural Language Processing

Programming

Other

• [Blog] Unix for Poets

Deep Computer Vision

Machine Translation

Models

• Sequence to Sequence Learning with Neural Networks. Ilya Sutskever et al. (2014)
  • Contributions included 2 separate LSTM-LMs, a deep LSTM with 4 layers, and reversed the order of the source sequence.
  • Their encoder-decoder architecture still encodes the source sequence into a fixed-length vector.
• No Language Left Behind: Scaling Human-Centered Machine Translation. NLLB Team and Marta R. Costa-jussà et al. (2022)
• MADLAD-400: A Multilingual And Document-Level Large Audited Dataset. Sneha Kudugunta et al. (2023)
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. Yonghui Wu et al. (2016)
• mT5: A massively multilingual pre-trained text-to-text transformer. Linting Xue et al. (2020)
  • Contributions: Extended the T5 model to a multilingual setting trained on a variant of common crawl (C4).
  • The Small and Base mT5 models may fall short on tasks on a given language compared to dedicated models on that language. The larger models are shown to empirically close the gap.
• Neural Machine Translation by Jointly Learning to Align and Translate. Dzmitry Bahdanau et al. (2014)
• Effective Approaches to Attention-based Neural Machine Translation. Thang Luong et al. (2015)

Papers

• Low-resource Languages: A Review of Past Work and Future Challenges. Alexandre Magueresse et al. (2020)

Other

• [Blog] The Bender Rule. by Emily Bender
• [Blog] A Neural Network for Machine Translation, at Production Scale. by Google

Large Language Models

Models

• Language Models are Few-Shot Learners. Tom B. Brown et al. (2020)

Retrieval-Augmented Generation

Papers

• Dense Passage Retrieval for Open-Domain Question Answering. Vladimir Karpukhin et al. (2020)

• OG-RAG: Ontology-Grounded Retrieval-Augmented Generation For Large Language Models. Kartik Sharma et al. (2024)

Human-Computer Interaction

Other

• Value Sensitive Design

Knowledge Representation

Papers

• What is an ontology?. Nicola Guarino et al. (2009)