WEBVTT Kind: captions Language: en NOTE Module 5: Writing, Safety, and the Verification Capstone narrated walkthrough. Generated from the narration script and per-segment audio durations (0.7 s gap per slide). s01-001 00:00:00.000 --> 00:00:03.635 This is Module 5 of Claude Code for Accounting Research: writing, safety, s01-002 00:00:03.685 --> 00:00:06.928 and the verification capstone. I'm John Barrios. s01-003 00:00:06.978 --> 00:00:12.171 In this module we work on keeping the writing in your paper your own, s01-004 00:00:12.221 --> 00:00:15.790 and on leaving a repo that a coauthor could pick up and trust. s02-005 00:00:16.540 --> 00:00:18.877 So here's how the live session runs. s02-006 00:00:18.927 --> 00:00:23.675 We open with about fifteen minutes on writing as thinking and where to draw the s02-007 00:00:23.725 --> 00:00:24.625 line on what you delegate. s02-008 00:00:24.675 --> 00:00:29.350 Then twenty-five minutes of demos: the editor prompt, a clean-room referee, s02-009 00:00:29.400 --> 00:00:34.155 the style skills, the referee DAG, and auditing a response letter. s02-010 00:00:34.205 --> 00:00:37.162 Then another twenty-five minutes on verification debt, sandboxes, s02-011 00:00:37.212 --> 00:00:41.437 permission rules and hooks, credentials, and paper trails, s02-012 00:00:41.487 --> 00:00:43.322 and a ten-minute git demo. s02-013 00:00:43.372 --> 00:00:46.228 A break, then eighty minutes of capstone lab, s02-014 00:00:46.278 --> 00:00:49.947 where you harden a pipeline from Module 3 or Module 4. s02-015 00:00:49.997 --> 00:00:53.850 We close with a debrief, the exit ticket, and the Module 6 readings. s03-016 00:00:54.600 --> 00:00:57.982 A lot of the thinking in a paper happens while you write it. s03-017 00:00:58.032 --> 00:00:59.950 If you can't get a sentence to work, s03-018 00:01:00.000 --> 00:01:02.995 often you haven't worked out the argument behind it yet. s03-019 00:01:03.045 --> 00:01:05.552 Say you give the model a vague prompt, s03-020 00:01:05.602 --> 00:01:09.142 like help me explain why this identification strategy is credible. s03-021 00:01:09.192 --> 00:01:13.285 You get back a fluent paragraph, you agree with it, and you paste it in. s03-022 00:01:13.335 --> 00:01:16.755 The model made the decisions in that paragraph, and you didn't. So test it. s03-023 00:01:16.805 --> 00:01:19.471 Can you explain that paragraph out loud, s03-024 00:01:19.521 --> 00:01:22.686 without the document or the model in front of you? s03-025 00:01:22.736 --> 00:01:25.896 And if you're too tired to say what you think, stop for the day, because a rushed, s03-026 00:01:25.946 --> 00:01:29.990 vague prompt just gets you the model's generic default. s04-027 00:01:30.740 --> 00:01:35.041 The common failure is mistaking fluency for having thought it through. s04-028 00:01:35.091 --> 00:01:38.781 A smooth paragraph doesn't tell you the argument in it is yours. s04-029 00:01:38.831 --> 00:01:41.762 A language model is built to produce text that reads well, s04-030 00:01:41.812 --> 00:01:44.203 whether or not the claim underneath holds up. s04-031 00:01:44.253 --> 00:01:48.912 So "this sounds right" and "I could defend this in a seminar" are two different s04-032 00:01:48.962 --> 00:01:49.863 tests. s04-033 00:01:49.913 --> 00:01:53.410 Before a paragraph goes into the paper, you want to pass the second one. s05-034 00:01:54.160 --> 00:01:58.784 The framework for the writing half of this module is the line between banal and s05-035 00:01:58.834 --> 00:01:59.662 meaningful writing. s05-036 00:01:59.712 --> 00:02:03.752 Banal writing is writing whose content is already set by something else. s05-037 00:02:03.802 --> 00:02:09.479 Variable-definition tables, data appendices, table notes, robustness-panel headers. s05-038 00:02:09.529 --> 00:02:11.890 The data already determine what they say, so delegate them. s05-039 00:02:11.940 --> 00:02:13.335 The agent is faster, s05-040 00:02:13.385 --> 00:02:18.565 and it doesn't get bored the tenth time it writes the same kind of sentence. s05-041 00:02:18.615 --> 00:02:23.039 Meaningful writing is the identification argument, the statement of contribution, s05-042 00:02:23.089 --> 00:02:28.449 the limitations section, any sentence where you make the argument. Protect those. s05-043 00:02:28.499 --> 00:02:31.685 There's no fixed list, and you decide where the line falls. s05-044 00:02:31.735 --> 00:02:35.995 So write down one task you'll always delegate and one you'll never let an agent s05-045 00:02:36.045 --> 00:02:36.350 start. s06-046 00:02:37.100 --> 00:02:41.133 The failure here is assuming you already know where your line is. s06-047 00:02:41.183 --> 00:02:43.179 The hard cases are in the middle. s06-048 00:02:43.229 --> 00:02:45.837 Take a paragraph describing a robustness check. s06-049 00:02:45.887 --> 00:02:48.564 It looks routine, because it just describes a table, s06-050 00:02:48.614 --> 00:02:53.064 but it contains a judgment call about which alternative specification matters s06-051 00:02:53.114 --> 00:02:55.183 and why. So use this rule. s06-052 00:02:55.233 --> 00:02:58.469 If a sentence makes you decide something rather than report it, s06-053 00:02:58.519 --> 00:03:01.130 it's on the meaningful side, however short it is. s07-054 00:03:01.880 --> 00:03:06.413 There's a second reason to protect the meaningful side, and it's about voice. s07-055 00:03:06.463 --> 00:03:09.716 LLMs take weak writing and make it competent. s07-056 00:03:09.766 --> 00:03:14.223 They also take distinctive writing and pull it toward the same generic register. s07-057 00:03:14.273 --> 00:03:16.230 Accounting journals have house styles, s07-058 00:03:16.280 --> 00:03:20.388 so a JAR paper and a TAR paper don't read the same, and that helps a little. s07-059 00:03:20.438 --> 00:03:26.436 But within a journal, papers can still drift toward the model's default voice. s07-060 00:03:26.486 --> 00:03:30.270 The style skills you'll see in a few minutes are meant to push back against that s07-061 00:03:30.320 --> 00:03:30.630 drift. s08-062 00:03:31.380 --> 00:03:33.484 The first demo is the editor prompt. s08-063 00:03:33.534 --> 00:03:36.430 You ask for comments, not rewrites. s08-064 00:03:36.480 --> 00:03:40.828 The prompt on the slide asks for feedback in the style of a New York Times editor, s08-065 00:03:40.878 --> 00:03:42.787 sharp on argument and clarity, s08-066 00:03:42.837 --> 00:03:46.411 as inline comments at the places where the argument is weak, unclear, s08-067 00:03:46.461 --> 00:03:47.992 or could be tightened. s08-068 00:03:48.042 --> 00:03:52.918 And it says twice: don't edit my writing, don't rewrite any sentences. s08-069 00:03:52.968 --> 00:03:57.081 Without that "do not," a capable model hands you back a polished rewrite, s08-070 00:03:57.131 --> 00:03:59.405 and you start accepting instead of thinking. s08-071 00:03:59.455 --> 00:04:03.869 With it, the model points to the weak spots and you do the rewriting yourself. s08-072 00:04:03.919 --> 00:04:06.758 And don't soften it to comments plus suggested edits, s08-073 00:04:06.808 --> 00:04:09.810 because a suggested edit takes one click to accept. s09-074 00:04:10.560 --> 00:04:12.864 The editor prompt has a weakness. s09-075 00:04:12.914 --> 00:04:15.418 Run it in the session where you wrote the section, s09-076 00:04:15.468 --> 00:04:20.387 and the model has heard all your justifications, like why the sample starts in 2004, s09-077 00:04:20.437 --> 00:04:22.692 so it'll tend to accept your design. s09-078 00:04:22.742 --> 00:04:25.154 A real referee has heard none of that. s09-079 00:04:25.204 --> 00:04:29.963 So run the critique separately, in a fresh session or a sub-agent like in Module 4, s09-080 00:04:30.013 --> 00:04:31.620 with the draft as its only input. s09-081 00:04:31.670 --> 00:04:32.684 Make it read-only, s09-082 00:04:32.734 --> 00:04:37.822 and have it write its report to a review folder so you can answer it point by point. s09-083 00:04:37.872 --> 00:04:40.062 Keep it on the paper, not the authors, s09-084 00:04:40.112 --> 00:04:43.372 because comments about the authors give you nothing to fix. s09-085 00:04:43.422 --> 00:04:47.821 This builds on Scott Cunningham's Referee 2 persona in MixtapeTools and the s09-086 00:04:47.871 --> 00:04:51.310 Editor persona in Alessandro Spina's Claude Code materials. s10-087 00:04:52.060 --> 00:04:53.265 Here's the prompt. s10-088 00:04:53.315 --> 00:04:57.075 It tells the model it's refereeing for the Journal of Accounting Research, s10-089 00:04:57.125 --> 00:04:59.814 that it only has the manuscript in the paper folder, s10-090 00:04:59.864 --> 00:05:01.978 and that it can't edit any file. s10-091 00:05:02.028 --> 00:05:06.281 It reads the identification and sample-construction sections and writes its s10-092 00:05:06.331 --> 00:05:08.282 report to a file in the review folder. s10-093 00:05:08.332 --> 00:05:10.603 The main instruction is the sample check. s10-094 00:05:10.653 --> 00:05:12.745 For every filter the paper states, s10-095 00:05:12.795 --> 00:05:16.677 does the attrition it implies line up with the N in Table 1? s10-096 00:05:16.727 --> 00:05:20.431 And which filters never have their effect on the sample reported? s10-097 00:05:20.481 --> 00:05:24.210 Accounting papers often describe one sample and build another. s10-098 00:05:24.260 --> 00:05:28.646 A reviewer who never saw your cleaning script has to rebuild the sample from the s10-099 00:05:28.696 --> 00:05:30.330 text, which is what a journal referee does. s11-100 00:05:31.080 --> 00:05:34.382 The failure here is a referee that agrees with you. s11-101 00:05:34.432 --> 00:05:36.817 Models tend to tell you what you want to hear. s11-102 00:05:36.867 --> 00:05:40.816 If the report is mostly praise, or it backs down as soon as you object, s11-103 00:05:40.866 --> 00:05:42.843 it isn't doing a referee's job. s11-104 00:05:42.893 --> 00:05:47.326 So ask for the three most serious problems instead of an overall assessment. s11-105 00:05:47.376 --> 00:05:50.546 And when you disagree with a point, write your answer in the report file, s11-106 00:05:50.596 --> 00:05:53.350 not in the chat, where the model can just concede. s12-107 00:05:54.100 --> 00:05:56.734 Demo 2 is the style skills. s12-108 00:05:56.784 --> 00:05:59.479 Barrios voice is built from my published papers. s12-109 00:05:59.529 --> 00:06:01.697 It picks up real patterns, like sentence length, s12-110 00:06:01.747 --> 00:06:06.357 stating a coefficient as a dollar amount, and not stacking hedges. s12-111 00:06:06.407 --> 00:06:07.790 It also sometimes overdoes them, s12-112 00:06:07.840 --> 00:06:11.430 so a passage reads like a caricature of how I write. s12-113 00:06:11.480 --> 00:06:15.529 And it reflects how I wrote when it was built, so it needs updating. s12-114 00:06:15.579 --> 00:06:19.580 Econ humanizer asks whether a passage sounds like a language model. s12-115 00:06:19.630 --> 00:06:23.088 It flags inflated claims of importance, stacked hedges, s12-116 00:06:23.138 --> 00:06:27.389 and sentences like "this paper contributes to the growing literature." The latex s12-117 00:06:27.439 --> 00:06:30.996 tables skill handles the routine table formatting. s12-118 00:06:31.046 --> 00:06:35.704 As in Module 3, think of each skill as a written procedure the model follows, s12-119 00:06:35.754 --> 00:06:36.450 like an SOP. s13-120 00:06:37.200 --> 00:06:38.730 Demo 3 is the referee DAG. s13-121 00:06:38.780 --> 00:06:43.043 Say you get an R&R with four referee reports that overlap and sometimes s13-122 00:06:43.093 --> 00:06:44.380 contradict each other. s13-123 00:06:44.430 --> 00:06:48.936 The strategic revision skill splits every comment into a task and works out s13-124 00:06:48.986 --> 00:06:50.698 which tasks depend on others. s13-125 00:06:50.748 --> 00:06:55.194 You can't rewrite the conclusion before you've run the robustness check it s13-126 00:06:55.244 --> 00:06:56.936 depends on. s13-127 00:06:56.986 --> 00:06:58.292 Then it groups what can run in parallel, s13-128 00:06:58.342 --> 00:07:02.660 and flags the critical path and any conflicting requests. s13-129 00:07:02.710 --> 00:07:05.232 You can follow the same steps without the skill. s13-130 00:07:05.282 --> 00:07:08.533 Break the revision into tasks, map what depends on what, s13-131 00:07:08.583 --> 00:07:10.990 and find the conflicts before you start writing. s14-132 00:07:11.740 --> 00:07:16.484 Once the revision is done, the response letter tells the referees what changed, s14-133 00:07:16.534 --> 00:07:21.389 and an LLM will write "we now report" whether or not anyone ran the analysis. s14-134 00:07:21.439 --> 00:07:26.310 So audit the letter in three columns: the claim, where it is in the manuscript, s14-135 00:07:26.360 --> 00:07:28.451 and the commit that implements it. s14-136 00:07:28.501 --> 00:07:32.892 Adding firm fixed effects to Table 4 has a location and a commit. s14-137 00:07:32.942 --> 00:07:36.929 Clustering at the industry level is in the table notes, with no commit. s14-138 00:07:36.979 --> 00:07:38.572 That empty cell is the dangerous case, s14-139 00:07:38.622 --> 00:07:43.256 because the text was changed to match the letter and the numbers weren't. s14-140 00:07:43.306 --> 00:07:46.582 Claude fills in the columns from the documents and git log, s14-141 00:07:46.632 --> 00:07:48.647 and leaves a cell empty rather than guess. s14-142 00:07:48.697 --> 00:07:50.770 You check every empty cell. s14-143 00:07:50.820 --> 00:07:54.410 This is adapted from the R&R mode of Spina's Editor persona. s15-144 00:07:55.160 --> 00:07:57.921 Now we move from writing to verification. s15-145 00:07:57.971 --> 00:08:02.832 Every task has two costs, the cost to make it and the cost to verify it. s15-146 00:08:02.882 --> 00:08:06.772 Agentic tools cut the first a lot and the second much less. s15-147 00:08:06.822 --> 00:08:08.829 The gap is verification debt, AI output you haven't checked. s15-148 00:08:08.879 --> 00:08:14.936 On the grid, the danger zone is the top left, cheap to make and expensive to verify. s15-149 00:08:14.986 --> 00:08:16.719 The top right is expensive both ways, s15-150 00:08:16.769 --> 00:08:20.394 where slow production forced checking as you went. s15-151 00:08:20.444 --> 00:08:24.472 Take a keyword count over ten years of 10-K Risk Factor sections. s15-152 00:08:24.522 --> 00:08:29.177 Claude builds it in ten minutes, but showing it measures disclosure change, s15-153 00:08:29.227 --> 00:08:34.773 and not boilerplate copied forward, takes an hour of hand-coding a sample. s15-154 00:08:34.823 --> 00:08:39.188 The more you let the agent run unsupervised, the faster that debt piles up. s15-155 00:08:39.238 --> 00:08:43.670 So with a partner, take three minutes and place two of your own tasks on the grid. s16-156 00:08:44.420 --> 00:08:48.106 Before any procedural fix, get the environment right. s16-157 00:08:48.156 --> 00:08:51.602 Ask what the agent can see, and the worst it could do with that. s16-158 00:08:51.652 --> 00:08:53.473 The ladder has four rungs. s16-159 00:08:53.523 --> 00:08:56.378 Manual approval is fine on your normal machine. s16-160 00:08:56.428 --> 00:09:00.188 Auto, where a classifier screens actions and you review the diffs, s16-161 00:09:00.238 --> 00:09:02.596 runs on your machine or in a container. s16-162 00:09:02.646 --> 00:09:07.270 YOLO, where permission checks are skipped, belongs in a sandbox or container only, s16-163 00:09:07.320 --> 00:09:09.066 never on your host. s16-164 00:09:09.116 --> 00:09:13.223 And unattended runs need an isolated container with restricted outbound network s16-165 00:09:13.273 --> 00:09:14.451 access. s16-166 00:09:14.501 --> 00:09:16.928 Skipping permissions in a sandbox is a different risk, s16-167 00:09:16.978 --> 00:09:20.176 because the agent sees only the folders you mount. Inside one, s16-168 00:09:20.226 --> 00:09:25.251 trying to read your SSH key returns "Operation not permitted." The kernel blocks it, s16-169 00:09:25.301 --> 00:09:28.530 so it doesn't depend on the model's judgment. s17-170 00:09:29.280 --> 00:09:33.204 Installing a skill puts someone else's instructions on your machine. s17-171 00:09:33.254 --> 00:09:35.338 So read it before you install it, s17-172 00:09:35.388 --> 00:09:38.615 the same way you'd read a script before you run it on your data. s17-173 00:09:38.665 --> 00:09:42.885 A Markdown file can still tell an agent to do something you wouldn't approve. s17-174 00:09:42.935 --> 00:09:44.624 And weigh where it came from. s17-175 00:09:44.674 --> 00:09:48.585 A skill from Anthropic or from your own group gets more trust than one from a s17-176 00:09:48.635 --> 00:09:51.310 little-known repo someone linked in a forum post. s18-177 00:09:52.060 --> 00:09:56.655 The failure here is treating the sandbox as your data governance. s18-178 00:09:56.705 --> 00:09:59.434 A container limits what the agent can do. s18-179 00:09:59.484 --> 00:10:02.497 It doesn't limit what the agent can see in a folder you mount, s18-180 00:10:02.547 --> 00:10:07.465 and it doesn't stop that folder's contents from being sent to the model during a s18-181 00:10:07.515 --> 00:10:08.428 normal session. s18-182 00:10:08.478 --> 00:10:12.235 So if a folder holds WRDS-licensed data, unpublished results, s18-183 00:10:12.285 --> 00:10:16.730 or anything with individual identifiers, mount only what the task needs. s19-184 00:10:17.480 --> 00:10:22.374 Permission rules are a cheap middle layer between trusting the model and running s19-185 00:10:22.424 --> 00:10:23.115 a container. s19-186 00:10:23.165 --> 00:10:25.609 The file on the left is a short settings.json. s19-187 00:10:25.659 --> 00:10:29.432 It blocks reads of your.env files and your SSH keys, s19-188 00:10:29.482 --> 00:10:33.064 blocks edits to the raw data folder, refuses recursive deletes, s19-189 00:10:33.114 --> 00:10:36.535 and makes any git push stop and ask you first. s19-190 00:10:36.585 --> 00:10:38.440 Commit it in the project's.claude folder, s19-191 00:10:38.490 --> 00:10:41.702 so every coauthor works under the same rules. s19-192 00:10:41.752 --> 00:10:45.175 The rules apply in every mode, and deny rules are checked first, s19-193 00:10:45.225 --> 00:10:48.097 so they hold even when permission checks are skipped. s19-194 00:10:48.147 --> 00:10:51.849 But they match tool calls and the literal command text, s19-195 00:10:51.899 --> 00:10:56.416 so calling rm by its full path, or wrapping it in bash -c, gets past them, s19-196 00:10:56.466 --> 00:10:59.703 and cat on the.env file reads it another way. s19-197 00:10:59.753 --> 00:11:03.590 So set them up, and keep using a sandbox where you need real isolation. s20-198 00:11:04.340 --> 00:11:08.455 You've now met most of the ways to shape what Claude Code does, s20-199 00:11:08.505 --> 00:11:09.485 and they come in two kinds. s20-200 00:11:09.535 --> 00:11:14.802 The top half of the table asks: CLAUDE.md, path-scoped rules, skills, s20-201 00:11:14.852 --> 00:11:16.085 and sub-agents. s20-202 00:11:16.135 --> 00:11:19.893 The model reads them and usually complies, but not every time, s20-203 00:11:19.943 --> 00:11:24.242 and a long session or a compacted context makes that less likely. s20-204 00:11:24.292 --> 00:11:29.836 The bottom half enforces: permission rules, hooks, and a container or sandbox. s20-205 00:11:29.886 --> 00:11:32.440 Those run whether the model agrees or not. s20-206 00:11:32.490 --> 00:11:36.428 So if something has to happen every time, put it in the bottom half. s20-207 00:11:36.478 --> 00:11:42.489 Writing "never modify raw data" in CLAUDE.md is a request the model usually follows. s20-208 00:11:42.539 --> 00:11:45.810 A deny rule on editing the raw data folder is enforced. s21-209 00:11:46.560 --> 00:11:50.679 A hook is the most flexible enforcer, because it's your own script. s21-210 00:11:50.729 --> 00:11:54.816 This one is registered to run before every shell command Claude issues. s21-211 00:11:54.866 --> 00:11:56.822 If the command is a git commit, s21-212 00:11:56.872 --> 00:12:01.343 it checks the staged changes for the string WRDS_PASSWORD, and if it finds it, s21-213 00:12:01.393 --> 00:12:04.591 it blocks the commit with a message. s21-214 00:12:04.641 --> 00:12:08.404 Claude Code sends each proposed command to the script as JSON. s21-215 00:12:08.454 --> 00:12:11.865 Exit code 2 blocks the command and shows Claude the message, s21-216 00:12:11.915 --> 00:12:12.931 and exit code 0 lets it run. s21-217 00:12:12.981 --> 00:12:16.656 It only checks commits Claude makes, though. s21-218 00:12:16.706 --> 00:12:18.423 Commits you type in your own terminal skip it, s21-219 00:12:18.473 --> 00:12:21.230 so you still need the audit on the next slide. s22-220 00:12:21.980 --> 00:12:26.306 Run two commands from the root of any project before you call it clean. s22-221 00:12:26.356 --> 00:12:31.931 The first asks git whether a.env file was ever committed, anywhere in the history. s22-222 00:12:31.981 --> 00:12:35.339 The second searches every file in the project for WRDS_PASSWORD, s22-223 00:12:35.389 --> 00:12:39.479 and you widen that to the other credential names your project uses. s22-224 00:12:39.529 --> 00:12:40.695 Both should come back empty. s22-225 00:12:40.745 --> 00:12:44.359 If either one finds something, treat the credential as compromised. s22-226 00:12:44.409 --> 00:12:50.636 Change the password or key first, then remove it from the git history. s22-227 00:12:50.686 --> 00:12:52.085 Deleting the current copy isn't enough, s22-228 00:12:52.135 --> 00:12:56.410 because anyone who clones the repo can still find it in an old commit. s23-229 00:12:57.160 --> 00:13:01.472 Ask for the paper trail up front, in the same prompt that describes the task: s23-230 00:13:01.522 --> 00:13:06.097 a DECISIONS.md, a LOG.md, and a sample-attrition table. Then check them, s23-231 00:13:06.147 --> 00:13:12.232 because an LLM will write a DECISIONS entry saying work was done when it wasn't. s23-232 00:13:12.282 --> 00:13:15.049 Picture a leverage panel like the Module 4 one. s23-233 00:13:15.099 --> 00:13:19.857 The entry says it added total liabilities over total assets as a robustness check, s23-234 00:13:19.907 --> 00:13:22.709 results consistent, see Table A2. s23-235 00:13:22.759 --> 00:13:25.071 The entry is confident and specific, s23-236 00:13:25.121 --> 00:13:28.665 but no Table A2 script exists anywhere in the repo, and no commit matches. s23-237 00:13:28.715 --> 00:13:34.619 So check each specific claim against the commit history before you rely on it. s23-238 00:13:34.669 --> 00:13:36.349 Use the documentation to find where to look, s23-239 00:13:36.399 --> 00:13:39.750 and then confirm the work in the code and the commits. s24-240 00:13:40.500 --> 00:13:43.263 A related rule, and it predates AI: s24-241 00:13:43.313 --> 00:13:46.786 nobody types a number into the manuscript by hand. s24-242 00:13:46.836 --> 00:13:51.305 The pipeline script computes the result, say a coefficient or a sample size, s24-243 00:13:51.355 --> 00:13:52.458 and writes it to a results.tex file. s24-244 00:13:52.508 --> 00:13:54.048 The paper's source never types that number. s24-245 00:13:54.098 --> 00:13:59.589 It pulls it in with a LaTeX input command that points to the results file. s24-246 00:13:59.639 --> 00:14:04.489 Rerun the pipeline and the paper updates at the next compile. s24-247 00:14:04.539 --> 00:14:07.984 What's different with an LLM in the loop is that, without this rule, s24-248 00:14:08.034 --> 00:14:11.962 a model drafting a results paragraph will write in a plausible number that s24-249 00:14:12.012 --> 00:14:13.276 nothing computed. s24-250 00:14:13.326 --> 00:14:15.010 The input pattern rules that out. s25-251 00:14:15.760 --> 00:14:19.952 Demo 4 is git, which you've been waiting on since Module 1. s25-252 00:14:20.002 --> 00:14:23.497 The five commands are on the handout, so the demo shows them in use. s25-253 00:14:23.547 --> 00:14:27.703 After a real Claude session, I run git diff against the previous commit, s25-254 00:14:27.753 --> 00:14:30.948 and you see a twelve-line diff you can actually review. s25-255 00:14:30.998 --> 00:14:32.700 Then I do one rollback, live. s25-256 00:14:32.750 --> 00:14:37.036 The habit to build is reviewing one commit at a time. s25-257 00:14:37.086 --> 00:14:41.270 At the start of a project, tell Claude to commit at every meaningful step. s25-258 00:14:41.320 --> 00:14:43.011 That's one sentence in your opening prompt, s25-259 00:14:43.061 --> 00:14:47.230 and it turns one big change into a series of small ones you can check. s26-260 00:14:47.980 --> 00:14:49.414 Here's what that looks like. s26-261 00:14:49.464 --> 00:14:52.667 It's a stylized transcript, not a screenshot. s26-262 00:14:52.717 --> 00:14:56.010 The commit message at the top says why the change was made: s26-263 00:14:56.060 --> 00:15:01.762 keep firm-years whose only missing values are in columns we don't use. s26-264 00:15:01.812 --> 00:15:04.807 One file changed, clean_fsds.py, five lines added and seven removed. s26-265 00:15:04.857 --> 00:15:08.767 The old step dropped rows with any missing value. s26-266 00:15:08.817 --> 00:15:12.556 The new one drops only rows missing total assets or sales, s26-267 00:15:12.606 --> 00:15:15.364 and converts the assets column to a number. s26-268 00:15:15.414 --> 00:15:20.263 A twelve-line commit with a message like that is the size of change you can s26-269 00:15:20.313 --> 00:15:21.737 check properly. s26-270 00:15:21.787 --> 00:15:26.403 An eight-hundred-line pipeline handed over all at once is much harder to review s26-271 00:15:26.453 --> 00:15:26.730 well. s27-272 00:15:27.480 --> 00:15:31.468 Once the course is over, here's a sensible order for what to set up. s27-273 00:15:31.518 --> 00:15:35.467 Every project, from day one: a CLAUDE.md with your data, conventions, s27-274 00:15:35.517 --> 00:15:37.806 and verification rules, a progress log, and git. s27-275 00:15:37.856 --> 00:15:42.918 When a correction repeats, the third time you give Claude the same instruction, s27-276 00:15:42.968 --> 00:15:43.906 make it a skill, s27-277 00:15:43.956 --> 00:15:49.115 and add permission rules once the project holds credentials or licensed data. s27-278 00:15:49.165 --> 00:15:52.950 When a pattern repeats across projects, that's when you add sub-agents, hooks, s27-279 00:15:53.000 --> 00:15:54.833 and global skills. s27-280 00:15:54.883 --> 00:15:58.268 Build each piece when a real repetition or risk calls for it, not in advance. s27-281 00:15:58.318 --> 00:16:02.923 And keep in mind that how much autonomy you give the agent and how much you s27-282 00:16:02.973 --> 00:16:05.046 configure are separate choices. s27-283 00:16:05.096 --> 00:16:09.670 This order echoes the three-tier adoption structure in Spina's Claude Code brownbag. s28-284 00:16:10.420 --> 00:16:11.850 Now the capstone. s28-285 00:16:11.900 --> 00:16:14.843 Pick your Module 3 EDGAR panel or your Module 4 pipeline, s28-286 00:16:14.893 --> 00:16:19.788 from WRDS or the public fallback, whichever is in better shape. s28-287 00:16:19.838 --> 00:16:21.663 You won't build anything new today. s28-288 00:16:21.713 --> 00:16:25.844 You harden that pipeline against the rubric: reproducibility, verification, hygiene, s28-289 00:16:25.894 --> 00:16:28.642 and the environment stretch. Before you start, s28-290 00:16:28.692 --> 00:16:33.097 take thirty seconds and write down one task you'll always delegate and one s28-291 00:16:33.147 --> 00:16:35.746 you'll never let an agent start. s28-292 00:16:35.796 --> 00:16:37.570 You'll be asked for both again at the end. s29-293 00:16:38.320 --> 00:16:41.586 The rubric has three required parts and a stretch. s29-294 00:16:41.636 --> 00:16:46.407 Reproducibility: the pipeline reruns from a clean clone using only the README, s29-295 00:16:46.457 --> 00:16:50.246 and the git history has meaningful commits at every checkpoint. s29-296 00:16:50.296 --> 00:16:53.210 Verification: you've checked DECISIONS, LOG, s29-297 00:16:53.260 --> 00:16:55.526 and the attrition table against the commits, s29-298 00:16:55.576 --> 00:16:58.550 recorded two sanity checks with sources, s29-299 00:16:58.600 --> 00:17:03.254 and traced one number from the raw data through results.tex to the output, s29-300 00:17:03.304 --> 00:17:05.146 with nothing typed by hand. s29-301 00:17:05.196 --> 00:17:09.610 Hygiene: both audit commands run and the output pasted, a correct.gitignore, s29-302 00:17:09.660 --> 00:17:13.157 the WRDS tunnel closed if you used WRDS, s29-303 00:17:13.207 --> 00:17:17.619 one banal artifact delegated and one meaningful one written by hand. s29-304 00:17:17.669 --> 00:17:22.497 The stretch is a rerun inside a container, with the blocked-access output pasted. s29-305 00:17:22.547 --> 00:17:25.950 You grade yourself, and I spot-check three or four projects live. s30-306 00:17:26.700 --> 00:17:29.717 Before you leave, you get the Module 6 readings. s30-307 00:17:29.767 --> 00:17:35.646 The required set is excerpts from Korinek, 2023, excerpts from Bloom, Jones, s30-308 00:17:35.696 --> 00:17:40.605 Van Reenen, and Webb, 2020, Macnamara and coauthors, 2024, s30-309 00:17:40.655 --> 00:17:44.788 the Nature editorial from 2023, and Lusher, Yang, and Carrell, 2023. s30-310 00:17:44.838 --> 00:17:51.671 If you want an accounting angle, you can swap in Kim, Muhn, and Nikolaev, 2024, s30-311 00:17:51.721 --> 00:17:53.276 or Dong, Stratopoulos, and Wang, 2024. s30-312 00:17:53.326 --> 00:17:55.075 The whole packet is under ninety minutes. s30-313 00:17:55.125 --> 00:17:58.516 Read it overnight. s30-314 00:17:58.566 --> 00:18:03.288 The position memo prompt is "What I will still be paid for in 2036," and debate s30-315 00:18:03.338 --> 00:18:05.610 positions get assigned in class. s31-316 00:18:06.360 --> 00:18:07.988 We close with the debrief. s31-317 00:18:08.038 --> 00:18:09.965 I put three or four projects on screen, s31-318 00:18:10.015 --> 00:18:13.781 and together we check one DECISIONS claim against the commit log. s31-319 00:18:13.831 --> 00:18:15.814 Then you share your delegation rule, s31-320 00:18:15.864 --> 00:18:19.042 one task you always delegate and one you never do, s31-321 00:18:19.092 --> 00:18:23.542 and we talk about what a coauthor would need to see before trusting your repository. s31-322 00:18:23.592 --> 00:18:27.496 The exit ticket is the standard three questions, one thing that clicked, s31-323 00:18:27.546 --> 00:18:28.670 one thing that broke, s31-324 00:18:28.720 --> 00:18:32.230 and one thing you'd want to verify before trusting a similar result, s31-325 00:18:32.280 --> 00:18:34.310 plus your delegation rule in writing. s32-326 00:18:35.060 --> 00:18:37.998 This last page is the course operating system, s32-327 00:18:38.048 --> 00:18:41.524 a one-pager of what to keep doing after the course ends. s32-328 00:18:41.574 --> 00:18:44.895 Across the top are the three mantras from Module 1: s32-329 00:18:44.945 --> 00:18:50.343 files persist and context doesn't, trust but verify the way you would with an RA, s32-330 00:18:50.393 --> 00:18:55.200 and be specific about the file, operation, aggregation, and output. s32-331 00:18:55.250 --> 00:18:58.713 Below that are the five git commands, the paper-trail files, s32-332 00:18:58.763 --> 00:19:02.826 the two-command credential audit, and a hygiene checklist. s32-333 00:19:02.876 --> 00:19:06.921 Module 6 comes back to a harder version of the delegation question, s32-334 00:19:06.971 --> 00:19:09.810 applied to what a PhD is for. See you there.