Google-led trial finds AI improves patent drafts but junior lawyers gain little lasting judgment

A three-month randomized trial followed 133 practicing patent lawyers at 11 U.S. firms, testing both AI-assisted drafting and professional judgment without the tool. The researchers gave participating lawyers access to a custom AI assistant for patent writing. At each firm, two-thirds of the lawyers were assigned early access; the control group received AI-use training but could not use the assistant during the study. Blinded patent attorneys evaluated the work, and the participating firms had regular, non-exclusive business relationships with Google.

Lawyers completed drafting exercises after 10 and 90 days, using simulated inventor materials. On those tasks, access to AI raised scores by 0.34 standard deviations at 10 days and 0.38 at 90 days; the reported statistical results were p = 0.03 and p = 0.01, respectively. The Google researchers said those gains corresponded to increases of 10 and 11 percentile points compared with control-group scores. Expert graders assessed five dimensions: enforceability, accuracy, strategic ambiguity, completeness and clarity. The score changes reflected fewer poor results and more good ones, without a noticeable rise in excellent scores. On the 10-day task, junior lawyers completed drafts 18 minutes faster than the control group’s average of 124 minutes.

At 90 days, participants also had to mark up and correct an error-filled patent application without AI, a task intended to test professional judgment. Lawyers in the treatment group scored 0.32 standard deviations above controls overall, with p = 0.04. The difference was concentrated among senior lawyers with at least seven years of experience, who scored 0.45 standard deviations higher, with p = 0.02. Junior lawyers showed no average improvement; their scores split toward both more poor and more good results, with fewer mediocre ones.

For AI builders, the trial distinguishes improved work while using an assistant from gains in judgment that remain when it is unavailable. In this study, stronger assisted drafts did not translate into an average improvement for junior lawyers on the unaided review task. The researchers suggest that foundational expertise may help workers turn AI-assisted practice into lasting skill. These results come from a three-month experiment in patent drafting, rather than a test across professions or over a longer period.