Eight gated modules. Each ends in a from-memory rewrite — no notes, no LLM. Every material is free.
ObjectiveBe able to say, without notes, what an optimizer is minimizing and why. You leave knowing what a cost function is, what a derivative tells you at a point, and why stepping against the gradient goes downhill — verified by hand-traced arithmetic, not by running code.
Three hand-traced tables for f(x)=x², x₀=5, five iterations each at η = 0.1, 1.1, 0.01 — columns x, f(x), f′(x), next x.
20 Khan linear-algebra problems by hand: vector addition and subtraction, magnitude, dot products.
10 derivatives by hand — x², 3x², x³, 5x, x²+3x — each also evaluated at a given point.
One page, no notes open: what a cost function is and why learning minimizes one; what the derivative tells you at a point; why the update subtracts η·f′(x); what the learning rate controls and its two failure modes.
All scanned or photographed into the repo. Zero lines of code.
From memory: define a gradient; x₁ = 4; why η=1.1 diverges; why the minus sign; a cost function example; [1,2,3]·[4,5,6] = 32.
ObjectiveTurn the algorithm you can already explain into code you own. You learn to translate an update rule into Python, to read a gradient in more than one variable, and to trust code only when it reproduces numbers you computed by hand.
gradient_descent.py — minimizes f(x)=x², a comment on every line, three saved loss-vs-iteration plots (η = 0.1, 1.1, 0.01) whose numbers match your Week 1 tables exactly.gd_2d.py — same loop on f(x,y)=x²+3y² using ∇f = [2x, 6y], with the descent path drawn on a contour plot.
12 Khan problems on partial derivatives and the gradient, by hand.
Half a page: why the gradient points in the direction of steepest ascent.
Rewrite log: gradient_descent.py deleted and rebuilt once from an empty file, committed separately.
Shape of Wx for 2×3 · 3×1. ∇f at (1,1) = [2,6]. Rewrite gradient_descent.py from an empty file and run it.
ObjectiveLearn what an expectation is and how a network computes its own gradients. You leave able to define E[g(X)], and to derive and implement backprop through a hidden layer from first principles rather than calling a framework.
nn_backprop.py — one hidden layer, NumPy only (no autograd), explicit forward pass and hand-derived backward pass, trained on XOR or a 2D toy set until loss is visibly near zero, with a printed loss curve.
15 solved problems from Blitzstein ch. 4, taken from the "Expectations and variances" and "LOTUS" headings — worked by hand, scanned into the repo.
Define E[g(X)] and compute it on 3 outcomes. Write dL/dW for the hidden layer. Say why activations are cached.
ObjectiveBridge probability and learning. You learn why sampling averages converge to expectations, and derive the one identity that makes policy gradients possible: the score-function trick that lets you differentiate through a random choice.
mc_estimate.py — samples X ~ U[0,1], estimates E[X²], plots the estimate against sample size and shows it settling on 1/3.
One-page derivation of ∇θE[f(x)] = E[f(x)·∇θlog pθ(x)], every step written out, with the line where ∇p = p·∇log p is used marked.nn_backprop.py deleted and rewritten from an empty file, reaching ≥95% on XOR.
Name the identity ∇p = p∇log p and show where it applies. Say why you cannot differentiate through sampling.
ObjectiveAssemble everything into a working policy-gradient agent. You learn how a policy is parameterised, how trajectories are collected under the current Gymnasium API, and how return-weighted log-probabilities become a loss an optimizer can minimize.
import gymnasium as gym
obs, info = env.reset()
obs, r, terminated, truncated, info = env.step(a)
done = terminated or truncated
Solved = mean return ≥ 475 over 100 episodes. The code constant says 500; the docs say 475 — use 475 and note the discrepancy. Pin your version.
reinforce.py — CartPole-v1 via Gymnasium: a 2-layer MLP policy, episode collection, discounted returns, the −Σ log π(a|s)·G loss, and a saved return-vs-episode curve trending past 100.requirements.txt with the Gymnasium version pinned.
Two paragraphs — one each on the Victores TEO paper and the García & Fernández safe-RL survey intro: what problem, what method, what result.
What you sum over a trajectory; why the loss carries a minus sign; what G multiplies and why.
ObjectiveUnderstand why REINFORCE is noisy and what fixes it. You learn what variance costs you, why subtracting a learned baseline is free of bias, and why on-policy data expires the moment the policy updates.
reinforce.py extended with a learned value-function baseline, hitting a 475+ mean return over 100 consecutive episodes, with before/after curves side by side.
Written answers to all four ownership questions, one paragraph each, Q3 showing where E[∇log π] = 0 is used.
README — what the repo is, how to run it, what the curves show.
Three email drafts (Victores, Fernández, one optional): specific interest tied to a named recent paper, repo link, one concrete question. Unsent.
All four ownership questions from memory — see below. For Q3, show explicitly where E[∇log π] = 0 is used.
ObjectiveMake the work legible to someone else. You learn to package a result so a stranger reproduces it in one command, and to write a research email that reads as specific interest rather than a request.
A public GitHub repo containing: reinforce.py runnable as one command, requirements.txt with pinned versions, a README with install → run → expected output, and a results section showing the 475+ CartPole curve. Plus two emails sent and all drafts saved in the repo.
Send in the second half of August so mail lands as staff return. Two or three sentences of specific interest referencing a recent paper, a repo link, one concrete question about their current work.
A stranger clones the repo, runs one command, reproduces your curve without asking you anything. Watch someone follow only the README on a clean machine.
ObjectiveProve ownership under time pressure. You learn the difference between having completed the course and being able to rebuild its central artifact from an empty file — and to summarise a real lab paper in one page.
Reproduce a working REINFORCE from an empty file in under 90 minutes.
No copy-paste, empty file, timer running — the run must pass on CartPole-v1, and you commit it with the elapsed time in the message.
Plus one page on the Victores TEO paper: the problem, the simulator and library stack, which algorithms were compared, the result, and one thing you would do differently. If the timed rewrite fails you have completed the course but not owned it — repeat modules 05–06 before Semester 1.
Explain to an imagined first-year, in five sentences, what REINFORCE does and why it works. If any sentence needs notes, you are not done.
Why does the gradient contain ∇log π?
Why is REINFORCE high-variance?
What does a baseline do, and why is it unbiased?
Why is REINFORCE on-policy?
Four semesters to PhD applications. PPO and SAC live here, not in module 05.
ppo.py · 37 PPO details · Sutton 2000. Gate: rewrite the PPO update from memory, reproduce a MuJoCo baseline.
2026–27 call open 21 Jul → 22 Sep 2026. €2,000, 2,111 grants. Needs GPA ≥ 7.25 and 75% of load passed. Verify on the BOE.
All free. The only paid dependency is GPU time for Isaac Lab.