Module 3 Lab: EDGAR Pipeline, Validity Protocol, First Skill
This lab has three parts, and each one builds on the last. First you build a real 10-K text pipeline end to end. Then you stop and write (prose, not code) a validity protocol for the measure you just built. Finally, you convert the whole pipeline into your first SKILL.md. The starter pack (lab3_starter.zip) contains a pinned list of twenty CIKs across four industry groups, a front-loaded prompt template, a SKILL.md skeleton with anatomy comments already in place, a paper-reader skill skeleton for the Part C stretch task, and this lab’s verify checklist. No filing data ships with the starter pack. EDGAR is public, so you fetch everything live, the same way you would for a real project.
Part A: Pipeline (45 min in a live session; plan on 60–90 minutes on your own)
Start from an empty project folder, as in every lab in this course. Unzip the starter pack and copy pinned_ciks.csv into that folder, since the prompt refers to it by name. Open the front-loaded prompt template from the starter pack and fill in the pieces specific to your run: the topic you’ll measure (pick one from the menu below), the fiscal years (2021–2023 is a reasonable default), your name and email for the User-Agent, and the output paths. The template already states the operational facts from the module notes: the CIK list, the rate limit, the User-Agent requirement, and the cache path. You should not need to work any of them out again. If you find yourself typing a fact the template already states, stop and fill in the template instead of writing a fresh prompt from scratch.
Pick one topic from the menu for your keyword measure: tariffs (trade policy exposure), cyber (cybersecurity and data-breach risk), climate (climate and physical-risk disclosure), or supply chain (supply-chain disruption risk). Any of the four works for this module’s pipeline. Pick the one you would most plausibly use in your own research, since you will read the actual sentences in Part B.
Build the pipeline in the order the module notes describe: CIK resolution and filing-index retrieval for the pinned firm list, cached raw-HTML download, Item 1A section extraction with a fallback chain of regex patterns, a load into DuckDB, and then one result table with your keyword measure by firm-fiscal-year. Four requirements are mandatory. First, the cache must prevent re-downloading a filing that already succeeded. Test this by re-running the pipeline once and confirming zero new network requests for filings already in the cache. Second, the pipeline must produce a quality report that states the extraction rate and gives a one-line diagnosis of what went wrong for every failure (a real edge case, a regex gap, a malformed filing). Third, the extraction rate must reach at least 95 percent before you move to Part B. Two pinned banks will not yield to a heading regex in any year. Wells Fargo’s 10-K answers Item 1A by pointing to the risk-factor section of its Annual Report to Shareholders (Exhibit 13 in the same filing), and Citigroup’s 10-K has a “Risk Factors” section but no “Item 1A” heading. Together they are six of the sixty filings, so you cannot reach the floor without handling them: fetch the right document or match the section title, and say which you did in the quality report. If the rate falls short, you are seeing the module’s central lesson in your own data. Read the quality report, ask Claude why the remaining filings are missing, and fix what you can before you accept the rest as real edge cases. Fourth, write a README that lists the named scripts, in order, that rebuild the result table from an empty folder, as in Module 2.
Part B: Validity check (15 min)
This part is short, and you write it rather than code it. The judgment here is the one piece of this lab you cannot delegate, and the goal is to practice stating that judgment explicitly, not to produce more code. Pick one firm from your panel and compute the year-over-year textual similarity between its Risk Factors section this year and the same section last year. Ask Claude to add a short named script for this that computes Jaccard similarity on word 5-gram shingles (cosine on word counts is a fine alternative). Read both sections side by side. Classify what you see. Is the year-over-year change in your keyword count driven by boilerplate (largely unchanged language, a high similarity score, a keyword count that moved because a paragraph got slightly longer)? Or is it driven by a meaningful rewrite, where the similarity score is lower and the keyword signal comes from the new language?
Then write, in three sentences, the audit-sample protocol you would run before this measure went into a real paper’s main table: which fifty filings you would pull and how you’d stratify the sample (by measure value, firm size, and industry, not a convenience sample of whichever firms are easiest to read), who would code them and against what rubric decided in advance, and what you’d report if more than one person coded any part of the sample. The deliverable is the protocol, not a claim that your measure is valid. Fifty filings are not enough to prove validity at any real scale. Writing “this measure is valid” without having run the audit is the validity failure the module notes warn about.
Part C: The skill (20 min)
Convert the pipeline you just built into edgar-panel/SKILL.md, installed at project scope (.claude/skills/) rather than globally. The pipeline is tied to this project’s data, and at project scope a coauthor who clones your repository gets the skill automatically. Use the five-step workflow from the module notes: notice the repetition (you just wrote this procedure out once already, in Part A’s prompt), name the inputs (CIK list, form type, fiscal-year range), design the output structure (DuckDB file, metadata.csv, quality report) before writing the skill body, let Claude draft the SKILL.md from a short spec covering those three things, then test-fire it. The file must start with --- on line 1. Claude Code reads the frontmatter only when the opening --- is the first line, so delete the skeleton’s comment blocks before you save.
Do not skip the test-fire. Open a fresh session, not the one you built the skill in, and confirm the skill triggers on a natural request that does not name the tool (something like “get me tariff counts from the risk factors of these two CIKs, 2021–2023”). A skill that fires shows up in the transcript as Claude loading edgar-panel by name. If instead you see Claude reading your old scripts from Part A, the skill did not fire. (You can always run it on purpose with /edgar-panel, but that does not test the description.) Then run it on two CIKs not in your pinned list and confirm it reproduces the same output structure. If it does not fire, check the likely cause from the module notes: does your description field read as a trigger condition or as a title? If it reads as a title, rewrite it.
Once your skill fires correctly, install one or two skills from the course’s Skills Catalog, whichever fit your own work. Then read the wrds skill’s SKILL.md as a mature example. Its description lists the phrases a real request uses (“query WRDS”, “access Compustat”, “get CRSP data”), and you will use it in Module 4. Look at how its description is phrased and how its steps are structured. That is the standard your own skill is working toward, even if this module’s version does not reach it yet.
Stretch, if you finish early: build a second skill, paper-reader, from paper_reader_SKILL_skeleton.md in the starter pack. Test it in a fresh session on one paper from your own reading list. Then check every number in its notes (each N and each coefficient) against the PDF’s tables, and write down how many were wrong.
Wrap-up and exit ticket
Write down where your extraction rate landed and whether each failure was a real edge case or a regex gap a better fallback chain could have caught earlier. Then read your description cold, as if someone else wrote it, and ask whether it would match a request that doesn’t name the tool. When the course runs with a group, two or three descriptions are read aloud without names and the room votes on whether each would trigger; if you are working with others, trade descriptions and do the same.
Answer the standard three-question exit ticket (one concept that clicked, one thing that broke, one thing you’d want to verify before trusting a similar result), plus this module’s item: what would your second skill be? Name one other recurring procedure in your own research workflow that you would convert next, and describe its inputs and output structure.