Judge Analytics: Lex Machina vs Trellis vs Custom Agent

By Jude Lee · · Comparison

Two attorneys comparing printed court orders and docket data on a laptop in a small firm conference room

What judicial analytics actually tells you

Strip the marketing away and judicial analytics is descriptive statistics over docket records: how often a given judge granted motions to dismiss, how long cases in that courtroom took to reach disposition, which firms appear most often, what the typical scheduling pattern looks like. It is useful context. It is not prediction, and it is emphatically not a model of how your judge will rule on your facts.

The second, newer thing people mean by “judge analytics” is an AI reading task: give an assistant the last thirty orders a judge issued on summary judgment, and ask for a structured brief — the standard she recites, the language she repeats, the arguments she rejects, the formatting she enforces. That is a text-comprehension job, and it sits close to the work motion-drafting assistants already do, with one advantage: the output is verifiable, because every claim points to an order you can open and read.

Analytics tells you what happened in cases that reached a public docket. It never tells you why — and “why” is the part you are actually paid for.

Where the underlying data comes from

Federal dockets come from PACER, the federal judiciary’s access system; per-page fees, caps, and the quarterly waiver threshold are published on the PACER site and change periodically, so check the current fee schedule rather than relying on anyone’s summary. The Free Law Project’s CourtListener and its RECAP Archive mirror a large and growing slice of federal filings for free, with an API.

State trial courts are the hard part. There is no national system; coverage is county-by-county, and some counties publish little more than a minute entry. That fragmentation is the whole reason this market has separate vendors, and it is why “does it cover my courthouse” beats every feature comparison.

How the off-the-shelf field actually differs

Lex Machina (LexisNexis) built its reputation on federal docket analytics, organized by practice area, with structured outcome and timing data and expanding state coverage. Trellis went the other direction, concentrating on state trial court records and judge pages in the counties where small and mid-size firms actually litigate. Westlaw sells litigation analytics inside its research subscription; its judge and court analytics have historically been deepest in federal district and appellate courts, with state coverage varying by jurisdiction — confirm yours on the current product page. The field is wider than those three: Bloomberg Law offers litigation analytics to its subscribers, and Docket Alarm (vLex/Fastcase) sells docket search and analytics with a different court mix again. Each vendor has been shipping natural-language query layers over this data — check the current product page, since this moves fast, and verify what any AI layer is actually reading before you rely on it.

The honest small-firm read: if you are a federal litigator, the federal-first databases earn their keep. If you are in three county courthouses, pull up a vendor’s page for your judge during the demo and see whether it is thin. A thin page is not a bug you can fix with a custom build either — if the county doesn’t publish orders, nobody has them.

Off-the-shelf analytics subscription
Buy when you need breadth: many judges, many jurisdictions, structured outcome and timing charts, no engineering. You get normalized data and someone else’s maintenance burden. You pay per seat, you inherit their coverage map, and you cannot ask it about your own firm’s history in front of that judge.
Custom judge-brief agent
Build when you need depth in a few courts: the agent reads orders you already have — PACER downloads, RECAP, your own prior filings and rulings — and produces a repeatable judge brief in your format. Budget for ownership, not a project: when a county site changes its layout, someone re-fixes the fetcher that week; when the model version changes, someone re-runs the skill against a saved set of graded briefs to confirm the output still holds. That is a named person with recurring hours — an internal technologist or a retained developer on a small monthly commitment — not a one-time build.

Building the judge-brief agent: MCP plus a skill

MCP (Model Context Protocol) is an open standard for giving an AI assistant governed access to specific tools and data. A small custom MCP server over your firm’s matter data can expose exactly three or four tools: search CourtListener for orders by judge and motion type, fetch a PDF from your document system, pull your firm’s prior filings before that judge, and write a draft memo back to the matter folder. Read-only by default; nothing in this workflow needs write access to a court system.

A skill is a packaged, reusable instruction set — the firm’s standard for how a judge brief gets produced every time: which sections, which sources count, what to do when there are fewer than ten relevant orders, how to cite each assertion back to a specific document. Skills are what turn a clever one-off prompt into something your associates can run without re-inventing the ask.

  1. Pick one judge and one motion type

    Scope it absurdly narrowly — say, summary judgment before the judge on your biggest open case. Breadth is what kills these pilots.

  2. Assemble the corpus by hand first

    Collect twenty to forty orders from PACER, RECAP, or your own files. If you cannot find them manually, an agent will not conjure them.

  3. Write the skill as a template, not a prompt

    Define the output sections: recited standard, recurring language, arguments rejected, procedural preferences, local rule enforcement, open questions. Require a document citation on every line.

  4. Wire up read-only access via MCP

    Start with CourtListener’s API and your document management system. Keep the agent’s permissions tight — see our take on read-only versus write access for legal agents.

  5. Have a partner grade the first three briefs

    Score each line: accurate, overstated, or unsupported. If more than a handful are unsupported, your corpus is too small or your skill is too loose.

  6. Write down the escalation rule

    When the agent’s summary and the lawyer’s reading of the order conflict, the human read wins by default and the agent’s line is struck. The supervising lawyer on the matter adjudicates — not the person who built the skill — and every disagreement gets logged so the skill can be tightened.

Where these systems break

Sampling bias is the first failure. Published orders skew toward contested matters; most cases settle, and settlements leave almost no analyzable trace. A “grant rate” computed over whatever happened to be published is not a base rate.

Second, correlation masquerading as insight. A judge who denies most motions to dismiss may simply draw a docket of better-pleaded cases. The agent will happily write a confident sentence about judicial tendency from data that supports no such thing. Your skill should instruct it to report counts and caveats, not conclusions.

Third, citation integrity. Any AI that summarizes orders can misattribute a quote to the wrong case. If this output ever feeds a draft brief, run it through a citation verification step and a human read. The ABA’s Formal Opinion 512, Generative Artificial Intelligence Tools (issued July 2024), and the technology-competence commentary to Model Rule 1.1 both put verification squarely on the lawyer; read them alongside your own state bar’s guidance, which may be stricter.

Modeling the payback without inventing numbers

There is no credible published figure for what a judge brief saves a small firm, so here is the arithmetic shape instead of a headline. Plug in your own measured numbers.

Assume first-pass order reading and local-practice research takes H hours per motion — measure it on your last three, don’t guess. Assume a blended rate R you supply for whoever actually does that work. Assume the agent removes a share S of the reading-and-summarizing time but not the judgment calls, and adds V hours of verification you will never remove.

Net recovered hours per motion = (H × S) − V. Multiply by motions per year (M), then subtract annual cost: subscription seats, or build plus the recurring maintenance ownership described above.

Worked shape: if H = 6, S = 0.5, V = 1, you recover 2 hours per motion. At M = 10, that’s 20 hours × R per year against your annual tool cost. If 20R doesn’t clear the cost line, the tool has to justify itself on better-aimed motions rather than hours — and that is not measurable, so don’t let a vendor put a dollar sign on it.

The numbers below are not findings — they are the only three figures we think are defensible for this workflow.

(H × S) − V
Recovered hours per motion — your measured hours, your assumptions
Worked example, not a benchmark
3 motions
Minimum sample to baseline your current prep time before buying anything
Editorial recommendation
100% of cited orders
Share a human should open and read before relying on the output
Editorial recommendation

Choosing between the three

Buy a subscription if you litigate across many courts, need structured outcome data for client reporting, and your judges show up with real depth on the vendor’s pages. Build a custom agent if your practice is concentrated in a few courts, you already have a pile of orders and prior filings, you want firm-specific memory the vendors cannot give you — and you can name the person who owns it next year. Stay manual if you appear before four judges you already know cold; an associate reading ten orders is cheap, and this is a case where the automation does not need to exist.

Most firms end up with a hybrid: a subscription for breadth, a skill for the judge brief, and a hard rule that no number leaves the building without a human reading the order behind it.

Where is your firm losing billable hours?

Get a free automation audit: we map your intake-to-invoice workflow and show you exactly what's worth automating — before you spend a dollar.

Get a free automation audit

Related guides