Working syllabus

LLMs from First Principles

Three three-hour classes that give software engineers the mathematical and production intuition to understand, evaluate, and deliberately change LLM systems.

Proposed dates: Tuesday, November 3; Tuesday, November 10; and Tuesday, November 17, 2026. November 16 is a Monday.

The course contract

By the end, participants should be able to build an eval-driven release loop, narrate one complete neural-network training step, explain next-token prediction, and choose between code, prompting, retrieval, constrained decoding, tools, caching, and fine-tuning.

Prerequisite: comfort reading ordinary Python and basic algebra. No calculus or data-science background is assumed.

Class 1 · November 3

Shipping nondeterministic software

Core question: How can we safely iterate on something whose output changes from run to run?

0:00–0:20 · Production failureAn agent improves overall quality while regressing security cases and increasing latency. The class decides whether it should ship.

0:20–0:50 · Development lifecycleSpecification → examples → baseline → error taxonomy → intervention → evaluation → deployment → monitoring.

0:50–1:15 · What an eval isDeterministic checks, task outcomes, schemas, rubrics, human review, model graders, slices, latency, and cost.

1:25–1:55 · Useful statisticsSampling uncertainty, paired comparisons, repeated stochastic trials, and aggregate scores that conceal subgroup failures.

1:55–2:25 · LabCompare stored outputs from two versions of a bug-triage agent and construct a release gate.

2:35–3:00 · Production operating modelEvery change begins with a hypothesis; every hypothesis predicts movement in named slices.

Class 2 · November 10

From classifiers to next-token prediction

Core question: What physically happens during one step of neural-network training?

0:00–0:20 · ScoresUse a bug-routing classifier to make parameters, features, biases, logits, and tensor shapes concrete.

0:20–0:50 · LossCross-entropy is the penalty for assigning too little probability to the correct answer.

0:50–1:20 · Backprop by handForward pass → loss → local derivatives → parameter gradients → update → new prediction.

1:30–2:00 · NumPy labStable softmax, cross-entropy, analytical gradients, an SGD update, and finite-difference gradient checking.

2:00–2:25 · BatchesGradient noise, vectorization, throughput, memory, accumulation, and why gradients must be cleared.

2:35–3:00 · Language modelsA next-token model is repeated multiclass classification, using preceding context to produce vocabulary logits.

Class 3 · November 17

Modern LLM systems from the same mechanics

Core question: Which part of the system does each LLM technique actually change?

0:00–0:30 · InferenceGreedy decoding, temperature, top-p, and why sampling cannot repair a systematically wrong distribution.

0:30–1:00 · Structure and toolsConstrained decoding, schema validation, tool calls, and the application-controlled agent loop.

1:00–1:20 · CachingKV caches, cross-request prefix caching, application caching, invalidation, latency, and cost.

1:30–2:10 · Fine-tuningTask data, masked token loss, gradients, validation, overfitting, parameter-efficient tuning, checkpoints, and rollback.

2:10–2:30 · Intervention choiceWhen to use code, prompting, retrieval, constraints, tools, fine-tuning, or caching.

2:40–3:00 · Intake simulationConduct a fine-tuning interview and defend the proposed experiment, deployment gates, and rollback plan.

Teaching rhythm

Every mathematical concept follows the same sequence: production problem → concrete numbers → Python → notation → engineering consequence. Across each three-hour class, target roughly 35% explanation, 35% building and investigation, and 30% decision-making, discussion, and retrieval practice.