[d-morrison]

Important

Stacked on #24 (recurrent networks), which is stacked on #23. Merge #23, then #24, then this one, each with a merge commit, retargeting each to main as its base merges.

Adds chapters/language-models.qmd, the week 7 Monday lecture on language modeling (plan module M5). It works on the opening of Coriolanus, the first 100,000 characters of Karpathy's tiny Shakespeare text. The local copy is chapters/data/shakespeare.txt, and tools/get-shakespeare.py rebuilds it. The text is public domain; char-rnn is MIT-licensed.

What the page does

Section	Exercise	Then	Result in the render
Counting	the chain rule for "the"	#def-language-model	a uniform model gives any 3 characters 1 in 226,981
Counting	the share of qs followed by u	#def-bigram-model (counting estimate)	100% of 32
Counting	validation positions whose pair never occurs in training	#def-add-one-smoothing	12 of 9,999 positions; after smoothing, P(u after q) = 0.35
Scoring	the average negative log probability	#def-perplexity	2.38 against log 61 = 4.11; perplexity 10.8 against 61
Recurrent model	train a GRU character model for 400 steps		validation perplexity 5.2 against 10.8
Recurrent model	the targets for a window	#def-teacher-forcing	the window moved one character on
Generating	the share of drawn words found in training		56%
Generating	softmax of logits divided by 0.5, 1 and 2	#def-temperature	word shares of 88%, 56% and 34% at temperatures 0.5, 1 and 1.5

• Each definition is followed by an example, and an assert guards each conclusion the prose draws. Training takes about 11 seconds on a CPU.
• Also changes:
  • _quarto-website.yml: the render list, navbar and sidebar.
  • chapters/calendar.qmd: the week 7 Monday cell.
  • references.bib: Jurafsky and Martin, SLP3 draft of Aug 19, 2026, sections 3.1.2, 3.3 and 3.6.1, each checked against the PDF; Hinton et al. 2015, section 2, eq. 1, checked; Karpathy's char-rnn repository and blog post.

Graded material

I searched dlg's exercises/, Homework/ and Quizzes/ for perplexity, bigram, smoothing, temperature, teacher forcing, language model, Shakespeare and char-rnn, and found nothing.
Homework 3 (Brian's Lab 5, an LSTM language model) is not written yet.
This page contains a complete character-level recurrent language model: training with teacher forcing, perplexity, and sampling with temperature. Swapping nn.GRU for nn.LSTM would answer an LSTM character model on this text, so Homework 3 needs to differ in data, token level or what it asks for. I have flagged this to Ezra.

Checks

• Rendered locally to HTML, PDF and slides; I checked every inline number against the rendered page.
• Ran check-term-links, check-new-line-breaks (290 added lines), check_example_after_def.py --fail, check_list_markers.py and typos locally; all clean on the chapter's files.
• Adversarial self-review (a separate subagent): 15 findings, all addressed in cd6dc00.
  • Add-one smoothing now has its own definition, and two exercises now come before the teacher-forcing and temperature definitions.
  • Added the data rebuild script.
  • The prompt is no longer counted in the word share, and new asserts cover the share claims.
  • Uses the shared \Prf, \condPr, \hPr and \vz macros.
  • Each reading now sits beside its content.
• Cross-model review (gpt-oss:120b via Ollama): 3 findings.
  • Rebutted: \expf is defined in latex-macros/macros.qmd, line 432.
  • Rebutted: @sol-gru-lm comes before the exercise that cites it.
  • Addressed: the Hinton et al. citation now names section 2 and attributes only the softened softmax to them.
• check-dois fails on lecun1998gradient, which IEEE answers with status 202 (see Add the convolutional networks chapter (week 5) #23). The SLP3 entry is now @misc, so the checker no longer asks it for a DOI.

🤖 Generated with Claude Code

https://claude.ai/code/session_019xfc9mZMigPGwaFDxGrYrf

[d-morrison]

Responses to the review of cd6dc00, fixed in 4eeb249:

• One div per subfile (addressed). The two reading callouts now live in their own subfiles, _perplexity_read.qmd and _sample_read.qmd, and language-models.qmd includes each one where its callout used to be.
• Math macros (addressed). The perplexity formula is now \expf{-\avgtof{t}{n}{\logf{\condPr{X_t = x_t}{x_1, \dots, x_{t-1}}}}}. I rendered it and checked the result.
• Line breaks (addressed). I split the lines flagged in _perplexity_def (lines 15 and 17), _temperature_def (lines 10 and 11), _bigram-model_def (line 5), _text-setup_sec, _perplexity_exm and _bigram-model_exm. The gha line-break checker passes over the 299 added lines.
• Temperature asserts (rebutted). The prose makes two claims: the share falls as the temperature rises, and most words are real at 0.5 while most are not at 1.5. Each assert backs one of those claims. If a different torch version changes the draw, I want the render to fail. Passing would leave the prose false. The seeds are fixed, and the local render passes (88%, 56%, 34%).
• Suggestions (deferred). I left get-shakespeare.py without a checksum. The Jurafsky and Martin section numbers (3.1.2, 3.3, 3.6.1) and the draft date were checked against the August 19, 2026 PDF before this round.

Posted by Claude Code (AI agent) --- not written by a human.

---

Generated by Claude Code

[d-morrison]

Cross-model review before merge. I ran gpt-oss:120b on Ollama cloud twice over this PR's changes since its previous cross-model review (ba4d72d..4eeb249). The second run included the evidence against the first run's findings. None of the findings held:

• Run 1: p_u_after_q undefined. It is defined in _bigram-model_exm.qmd.
• Run 1: exm-language-model missing. It is _language-model_exm.qmd line 1.
• Run 1: setup symbols defined twice. Each symbol is defined in one subfile only.
• Run 1: NaN rows in counted. Only rows t and q are read, and both are nonzero.
• Run 1: sample used before it is defined. It is a function in the exercise's own chunk, not a defined term.
• Run 1: SLP3 section 3.6.1. Section 3.6.1 is "Laplace Smoothing" in the Aug 19, 2026 draft.
• Run 1: "can carry" overstates GRU memory. "Can" describes what the recurrence allows, not perfect memory.
• Run 2: the print runs before p_u_after_q is assigned. Line 16 assigns it and line 17 prints it, and the page renders without error.

Posted by Claude Code (AI agent) --- not written by a human.

---

Generated by Claude Code

[d-morrison]

I addressed the review of cca2e5a in 6c288ff. In _perplexity_def.qmd, "which gives probability $1/K$ to each of $K$ symbols" now has a line of its own, and "has perplexity $K$." follows on the next line. The line-break checker passes on all 300 added lines.

The temperature asserts stay, for the reason in the previous reply. The SLP3 section numbers were checked against the PDF.

Posted by Claude Code (AI agent) --- not written by a human.

---

Generated by Claude Code

[github-actions]

Claude finished review — View run

---

I fetched PR 25 live just now. It is open, not a draft and not merged, and its head is still 6c288ff9345b49c4a85cd222157fa85c180bdf5d, the commit I reviewed. The review below therefore still applies.

Review of Morrison-Lab/dds PR 25 at 6c288ff

The only commit since the last round (cca2e5a..6c288ff) is 6c288ff. It touches chapters/_subfiles/language-models/_perplexity_def.qmd and splits "which gives probability $1/K$ to each of $K$ symbols has perplexity $K$" into two lines. The rendered output is unchanged. The commit adds no math, no code and no graded-item content. I did not render the pages or run the Python chunks.

The last posted verdict was Ready for merge, at 4eeb249: #25 (comment). It still stands. The new commit adds no violations.

Verdict

Ready for merge. The new commit is a formatting-only line break and adds no findings.

Structured Review Data (JSON)

{
  "schema_version": "1.1",
  "reviewer": "claude",
  "commit_sha": "6c288ff9345b49c4a85cd222157fa85c180bdf5d",
  "verdict": "CLEAN",
  "findings": [],
  "detailed_assessment": "The only change in range is a clause-level line break in _perplexity_def.qmd that renders identically and adds no math or code.",
  "holistic_assessment": "The change is a scoped follow-up to the earlier line-break findings, with no integration, regression or graded-content risk; Python chunks were not executed here."
}

Stopping Point: Clean stopping point reached --- session done; UMS executed; no follow-up items pending

Reviewed commit: 6c288ff

[github-actions]

💰 Cost: $0.2181 (review) — run

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.

_{2026-10-30 16:35 UTC}