Module 5 Lab: The Verification Capstone
Download the starter pack (zip) · Module 5 slides (PDF)
This module’s lab has no new dataset and no new pipeline to build. Instead, you take a pipeline you already have — from Module 3’s EDGAR text panel or Module 4’s WRDS (or public-fallback) funda-style pipeline — and harden it against the rubric below until it would survive a coauthor, a referee, or your own future self picking it up cold. Everything you need is in lab5_starter.zip: the rubric reproduced below (this is your grading instrument, not just a suggestion list), an audit-command card, DECISIONS.md/LOG.md templates, a container quick-start for the environment stretch, and a tiny worked example of the results.tex / \input{} pattern. Nothing in the starter pack is data — you’re pointing it at a pipeline you already built across this course.
Pick your pipeline
Choose Module 3’s EDGAR text pipeline (CIK lookup → filing extraction → DuckDB store → keyword or section-level result) or Module 4’s WRDS funda pipeline (or its public-FSDS fallback, if you were on that track) — whichever you have in better shape right now. You are not building anything new in this module; you are making the existing thing defensible. If neither pipeline is in a runnable state, spend the first ten minutes getting it back to “it ran once” before you start hardening it — the rubric assumes a working baseline, not a rescue project.
The capstone rubric
This is the exact rubric you’ll self-grade against at the end of the lab, and the one the instructor spot-checks live in the debrief. It has four sections; the fourth is a stretch, not a requirement.
Reproducibility
- The pipeline reruns start-to-finish from your README’s instructions alone, in a clean clone of the repository (a fresh folder, not your working directory) — no step that only works because you remember something the README doesn’t say.
- Git history shows meaningful commits at every real checkpoint (download, clean, measure, figure/table), each with a message that says why, not just what — not one giant commit that reads as a single undifferentiated blob.
Verification
DECISIONS.md,LOG.md, and a sample-attrition table are all present, and you have cross-checked them against the commit history rather than taken them on faith. Pick one specific claim fromDECISIONS.md, find the commit (or commits) it should correspond to, and write down — in your own notes, not just in your head — whether the diff actually supports the claim.- Two sanity checks are recorded with sources: one benchmark comparison (a number checked against something Claude didn’t touch) and one plausibility check (does the pattern make sense given what you know about the setting).
- One number is traced end-to-end: from the raw data, through the script that computes it, into a
results.texfile, and into a rendered output — with zero hand-typing anywhere along that chain. Write down which number you traced and the file path at each stage.
Hygiene
- Both credential-audit commands (
git log --all --full-history -- .envand agrep -rfor your project’s credential names) are run against this repository, and both outputs are pasted into your lab notes — not just described as “clean,” shown. .gitignoreis correct: credentials, raw licensed data, and any WRDS-derived files are excluded, and agit log --all -- .env-style check confirms none of them were ever committed.- If your pipeline used WRDS, the Duo/SSH tunnel is confirmed down, not left open.
- One banal writing artifact was delegated to Claude (for example, a variable-definition table generated directly from your metadata table) and one meaningful artifact was written by hand: a three-sentence note stating your pipeline’s contribution or a specific limitation, in your own words, that you could defend without the document in front of you.
Environment (stretch)
- Rerun the pipeline — or a meaningful slice of it — inside
claude-containeroragent-safehouse, and paste the blocked-access proof: the command you ran that should fail outside the mounted project folder, and the “operation not permitted”-style output confirming it did.
Environment stretch: quick-start
If you’re attempting the container stretch, lab_starter/container_quickstart.md has the two paths — claude-container (Docker-based) and agent-safehouse (lighter, Mac-native) — with the exact commands for each. The point of the stretch is not speed; it’s producing one concrete piece of evidence that the sandbox boundary is real, the same kind of proof you saw in this morning’s lecture (cat ~/.ssh/id_ed25519 failing with “Operation not permitted” from inside the sandbox). A pipeline that runs identically inside the container, plus one blocked-access command that fails exactly as it should, is the whole deliverable for this section — don’t try to move your entire course’s work into the container.
Debrief and exit ticket
Come back together for the live spot-check: the instructor will pull up three or four repositories and, together, audit one DECISIONS.md claim against its commit log in front of the room. Notice what’s easy to verify and what isn’t — that gap is the verification-debt 2×2 from this morning, applied to your own work instead of a hypothetical.
After the spot-check, each student states, out loud, one banal task they will always delegate and one meaningful task they will never let an agent originate — the delegation rule from this morning’s writing framework, now committed to publicly rather than just written down. This folds in the coauthor-handoff question directly: what would a real coauthor need to see in this repository before they’d trust it without re-doing your work themselves?
Complete the standard three-question exit ticket (one concept that clicked, one thing that broke, one thing you’d want to verify before trusting a similar result), plus this module’s specific item: your one-sentence delegation rule, in writing.