AI Medical Record Review: PI Platforms vs Custom Agent

By Jude Lee · · Comparison

Paralegals and an attorney reviewing medical records and a case timeline on a laptop in a small law firm

The job you are actually automating

A plaintiff’s-side record review is not one task. It’s about six, and they fail differently:

  1. Ingest and OCR. Faxed, scanned, rotated, sometimes handwritten. Bad OCR poisons everything downstream.
  2. De-duplicate. Providers send the same discharge summary four times across three productions.
  3. Index by provider and date range. So you know what you have before you know what it says.
  4. Extract the chronology. Date, provider, encounter type, findings, treatment, work restrictions — each entry tied to a Bates or PDF page.
  5. Flag the gaps. Missing records, treatment gaps, prior-condition references, IME contradictions.
  6. Summarize for a purpose — demand package, mediation brief, expert letter, deposition prep.

Steps 1–3 are mostly mechanical and automate cleanly. Step 4 is where language models genuinely shine and also where they invent things. Steps 5–6 are lawyer judgment with AI as a drafting assistant, not a decision-maker.

Three ways to put AI on it — and one that isn’t AI

Purpose-built PI platform

Upload the production, get back an indexed chronology with page citations, often plus a draft demand. The vendor has already solved OCR of ugly scans, provider normalization, and duplicate detection — the boring parts that take months to build. You get support, an audit trail, and a BAA. You give up control of the output format, pay per matter or per page, and inherit whatever the model decides is “significant.” For a two-to-eight-lawyer PI firm with steady volume, this is usually the fastest path to value.

Custom agent on your own stack

An agent that pulls the record PDFs out of your DMS, runs the extraction, writes the chronology back to the matter, and updates your case-management fields. Makes sense when the chronology has to feed something else — a litigation database, a settlement model, an intake scoring system — or when you handle a record type the platforms handle poorly. Costs real money and real project time up front, and you own the maintenance.

The third option gets overlooked: a general assistant plus a written skill. A skill is a packaged, versioned instruction set that teaches an AI assistant to do one job the same way every time — your chronology column order, your date-format rule, your “cite every entry to a page or omit it” rule, your list of red-flag phrases. It lives in a file your firm edits, not in a vendor’s prompt. Cost is close to your existing AI subscription. Two caveats: you don’t get industrial OCR or duplicate detection, so you’re pre-processing yourself; and consumer or individual subscription tiers generally do not come with a business associate agreement and can carry different retention and training terms than enterprise or business tiers. Confirm in writing that your plan’s terms cover PHI before the first upload, not after. We walk through the mechanics of writing a skill in AI skills for repeatable document review.

The fourth option is a human one, and it still wins more often than the AI conversation admits: outsourcing to a legal-nurse consultant, contract nurse reviewer, or chronology vendor. If you handle only a handful of record-heavy matters a quarter, the fixed cost of learning any tool never amortizes. If your records are unusual — workers’ comp panels, foreign-language charts, dense handwritten therapy notes — a trained reviewer degrades more gracefully than any model. And a human who has read the whole file can sit with you for deposition prep and answer questions the chronology doesn’t cover.

The chronology is not the deliverable. The verified chronology is the deliverable, and verification time is the number that decides whether any of this pays.

Where AI has actually stuck in small firms

The uses that survive contact with a real practice are narrower than the marketing suggests: first-draft correspondence and routine documents, search across the firm’s own files, discovery and record triage, intake response speed, and time-entry cleanup. Those are the jobs where output is checkable in less time than doing it from scratch. Record review sits squarely in that set — a paralegal can spot-check a cited page in seconds.

AI still disappoints on anything requiring an assertion about a source you haven’t verified, anything where the model must decide what’s legally material, and anything with a hard external deadline and no human gate. Fabricated citations, confident wrong dates, and silent omissions are failure modes of unchecked output, not arguments against the technology. Our oversight-model breakdown covers how to structure the check.

Focusing the effort where the value is

Our view: in a typical record production, a small fraction of the pages carry nearly all of the case value — the ED note, the imaging report, the first documented complaint, the work restriction, the surgical recommendation. The rest is billing sheets, repeated intake forms, and vitals.

That suggests the highest-return automation isn’t “summarize everything.” It’s index everything, surface candidate key pages, and let a human read those closely. Tools that try to produce a finished narrative from page one tend to smooth over exactly the contradictions you’re being paid to find. Structure first, prose last.

Modeling the payback yourself

Don’t trust anyone’s published hour-savings number, including a vendor’s. Build your own from inputs you already have:

InputWhere to get itExample placeholder
Baseline review hours per matterTime entries on your last five closed record-heavy matters___ hrs
AI-assisted hours per matterMeasured on those same matters during a pilot___ hrs
Verification hours per matterMeasured in the pilot — it never goes to zero___ hrs
Loaded cost of whoever does the workSalary + benefits + overhead ÷ productive hours$___ /hr
Tool cost per matterVendor quote, or subscription ÷ matters per month$___

Work it as: (baseline hours − AI-assisted hours − verification hours) × loaded hourly cost, minus tool cost per matter. Use your own effective hourly value rather than any published salary average. Then add the part firms forget: what the recovered hours get reallocated to. If a paralegal’s freed hours go to demand packages that go out weeks earlier, the value shows up in cycle time and case throughput, not in a labor line item. If they go to nothing in particular, the savings are theoretical. Our ROI walkthrough has the full worksheet.

Choosing, without pretending there’s one right answer

No single product is right for every firm, and any list that names one is selling something. The useful question is per-job:

  1. Under ~10 record-heavy matters a quarter

    Use a general assistant with a written chronology skill, plus decent OCR software — but only after confirming your AI plan’s terms permit PHI and you have a BAA in place. If they don’t, either upgrade the plan or send the work to an outsourced reviewer. Don’t buy a platform for volume you don’t have.

  2. Steady PI volume, no engineering appetite

    Buy the purpose-built platform. First, hand-build the chronology for one closed matter yourself so you have ground truth. Then run the tool on it and count two things: entries it missed, and citations pointing to the wrong page. Any miss on a treatment gap or a prior-condition reference is disqualifying regardless of the overall accuracy rate. Score on that, not on how good the prose reads.

  3. Records must flow into other systems

    Consider a custom agent with MCP access to your DMS and practice-management system, so it reads the matter file and writes structured results back. The approach is the same one described in building an MCP server over your matter data.

  4. Always

    Keep the human gate on the chronology before it leaves the firm. Every entry that will appear in a demand, brief, or expert letter gets its cited page opened.

Where is your firm losing billable hours?

Get a free automation audit: we map your intake-to-invoice workflow and show you exactly what's worth automating — before you spend a dollar.

Get a free automation audit

Related guides