Faster junior drafting needs an apprenticeship control
Better assisted drafts do not establish stronger professional judgement. Assess the capacity benefit while separately testing the unaided review capability the practice needs to maintain.
Keep development and test groups separate before comparing performance. No measured results are shown.
An AI drafting assistant may help a junior professional produce a stronger first draft with less effort. That creates an opportunity for additional work, faster service or more focused senior review. The practice needs to measure whether the benefit survives the full path to an accepted document.
The practice also depends on professional judgement. Producing a sound document with assistance and identifying defects without it are different capabilities. A training plan should assess both rather than infer the second from improved assisted output.
Use assistance where it improves work, while retaining representative unaided review tasks and reasoned feedback. The operating decision should account for present capacity and the development of reviewers able to identify substantive errors when assistance is wrong or unavailable.
Assisted quality and acquired judgement are different results
A September 2026 field experiment makes the distinction unusually visible. The researchers randomly assigned 133 practising patent lawyers at eleven US intellectual-property firms to receive a custom AI drafting assistant or remain in a control group. Blinded patent lawyers at another firm scored benchmark work for enforceability, accuracy, strategic ambiguity, completeness and clarity. Ninety-one participants completed the full three-month protocol. Autor and colleagues, 2026
With the assistant, lawyers produced better patent drafts. The treatment group scored 0.34 standard deviations higher after ten days and 0.38 higher after ninety days. The first task took about ten minutes less against a control-group average of 112 minutes. The estimated time saving at ninety days was similar in minutes but statistically uncertain. Junior lawyers captured most of the measured speed gain and larger improvements in assisted drafting quality.
The final exercise removed the assistant. Participants had to redline a flawed patent application, a task designed to expose the judgement behind the edit. Treated senior lawyers outperformed senior controls. Treated juniors showed no average improvement over junior controls, and their results spread out. There were fewer middling scores, but more poor scores as well as more good ones.
That result does not establish that AI damaged junior learning. The experiment had no baseline test of unassisted skill, lasted three months and could not observe every participant’s tool use. It does establish something a practice can act on: better assisted output is not evidence that professional judgement has developed. The two outcomes need separate tests.
| If the pilot measures output only | If the pilot measures work and capability |
|---|---|
| Give every draft to the assistant and count time saved | Name the drafting steps AI may assist and the review steps the person must still perform |
| Treat partner corrections as ordinary rework | Code corrections by defect type and record whether the junior can explain the repair |
| Compare the final document with the old average | Blind-score assisted drafts and separate unassisted review tasks |
| Expand when usage and output quality rise | Expand when capacity improves and unaided judgement holds or develops |
Put the assistant at the first-draft boundary
Start where the work is bounded enough to evaluate. In a patent practice, that could mean producing dependent claims or a detailed description from an approved invention disclosure and a named set of source material. In another professional service, it could be the first version of a standard analysis or client document. The useful boundary has known inputs, a review step and defects that experienced practitioners can classify.
Before AI, a junior produces the first draft, a senior reviews it and the junior repairs the work. After AI, the junior prepares the inputs, directs the first draft, verifies every substantive statement and explains proposed changes to the reviewer. The senior still decides whether the work is fit to leave the firm. The difference is where the blank-page effort goes and how quickly the draft reaches a reviewable state.
Do not automate the feedback loop away. If the assistant silently repairs each error, the final document can improve while the junior receives no account of what was wrong. Require the reviewer to identify the defect and the junior to record the reason for the repair. That record becomes both coaching material and an evaluation set for later versions of the system.
An earlier randomised experiment with law students found the same reason to separate speed from quality. GPT-4 access produced large and consistent time savings across realistic legal tasks, while improvements in legal-analysis quality were slight and uneven. The participants were students rather than practising lawyers, and the study measured immediate work rather than skill after sustained use. It supports the measurement distinction without resolving the apprenticeship question. Choi, Monahan and Schwarcz, 2024
Make the capacity gain appear in an operating measure
Minutes saved on a benchmark are a mechanism, not a commercial outcome. The practice owner needs to decide where the released effort will go. It might support more matters with the same team, shorten the interval to a client-ready draft, reduce partner correction or move senior attention towards the few decisions that carry the most consequence. Name one before the pilot starts.
Measure matters reaching an agreed review stage per lawyer, elapsed time to that stage and senior review minutes per accepted draft. Read those measures beside substantive defect rates, revision rounds, missed deadlines, client-requested corrections and any downstream event that reveals weak work. A faster first draft followed by more partner repair has moved effort rather than released it.
Then measure capability separately. Give participants periodic, representative work without the assistant and score it blind using the same rubric. Include a review task, because finding and repairing a defect is different from generating plausible text. Segment the results by experience and prior performance. The average can hide the people whose assisted output rose while their unaided judgement did not.
- 01 Bound Choose one repeatable drafting step with approved inputs and an accountable reviewer.
- 02 Baseline Record elapsed time, review effort, defect types and accepted output before assistance.
- 03 Assist Let the junior direct and verify the draft while the senior keeps the release decision.
- 04 Test Blind-score assisted output and periodic unassisted review work as separate outcomes.
- 05 Decide Expand, narrow or redesign according to capacity, quality and skill by cohort.
The capability assessment should sample judgement the service actually requires. Use representative review tasks and a consistent rubric, and distinguish them from tests of factual recall. If performance changes, investigate the training and task allocation before expanding the assisted workflow.
What the Inference Institute can help decide
The useful engagement defines the workflow boundary, the quality rubric and the comparison before a platform is selected. It maps where source material enters, where AI can draft, which assertions require verification, what a senior must decide and which unaided tasks represent durable professional judgement.
The result is a pilot specification a managing partner can use: one class of work, eligible users, permitted inputs, review rules, capacity measures, quality measures and a stopping condition. It also makes the operating choice explicit. The practice can use the released effort for additional work, faster service or deeper review, then test whether that outcome actually occurred.
What this does not tell you
The patent study is a working paper published in September 2026. Its sample was small, its participating firms already performed work for Google, several authors were Google employees and Google funded the direct experimental costs. The tasks were benchmarks, while the on-the-job measures covered fewer people and were too imprecise to establish a production effect. The results do not forecast another firm’s capacity, quality or skill development.
The study also does not identify the best teaching method. An unassisted review sample can reveal a gap. Closing it may require worked examples, supervised reasoning, different task allocation or more deliberate feedback. Those choices belong to the practice and the profession.
Choose a bounded drafting task where better first work could improve a defined operating measure. Assess that benefit alongside unaided review performance and the limitations of the comparison. Expansion should follow evidence about accepted work and professional capability, rather than assisted draft quality alone.