Benchmarks

One hundred rebuilt verdicts, scored against the judge.

100 US civil jury verdicts rebuilt from their public records, each scored by 250 simulated juries. A further 900 documented verdicts are listed below as a coverage catalog; those are not an accuracy measure and are excluded from the score.

direction, rebuilt suite: 76/100
trial judge, same task: 77.9%
when it commits: 59/77
catalog, not scored: 900

What these numbers mean

Each case here already has a real jury verdict. We run it through 250 simulated juries. Direction is how often those juries favored the same side as the real one: 76 of 100 on the rebuilt suite.

The honest comparison is not 100%. In a federal study of roughly 400 jury trials, the trial judge who sat through the whole case predicted the verdict 77.9% of the time, and judges and trial attorneys agreed with each other only weakly on which side the evidence favoured. Jury outcomes carry a large irreducible component. We publish where we sit against that, not against perfection.

It is not equally good on both sides. It calls 49 of 64 plaintiff verdicts correctly and 26 of 35 defence verdicts. Those are worth reading separately, because this suite is 64 plaintiff wins to 35 defence wins — so a caller that read nothing and always said plaintiff would score 64% on it. The headline above is the average of the two columns, not a single skill, and if you are defending a case the second number is yours.

Direction is a threshold, so it is less precise than it looks. A case counts as matched when its plaintiff share lands on the real verdict's side of 50%. 5 of the 100 rebuilt cases sit within two points of that line — each is close to a coin flip, and each carries a full point of the headline. Running more juries does not settle them: the same suite scored at 250, 500, 1,000 and 2,000 juries moves across a four-point range, because those cases are genuinely at the boundary rather than under-sampled. Read the figure as roughly plus or minus three points, and prefer when it commits above, which excludes the rows the engine itself calls too close.

The % plaintiff on each row is simply the share of those 250 juries, not a grade. 100 cases are rebuilt from the public record and carry the score. The other 900 catalog rows borrow their feature vectors from a same-program template case rather than from their own record, so they show coverage, not accuracy, and are excluded from every figure above.

Real: P plaintiff, D defense, H hung.
Called: The side 250 simulated juries favored.
% plaintiff: Share of those juries, not a score.
Result: Match same side. Miss other side. Even is 45-55%.

1000/1000

CaseProgramShareCalledResult

How to read the score, including the misses

Direction on the rebuilt suite is 76 of 100. Even means the plaintiff share sat within five points of 50%, so the engine did not commit. On the stress rows where it does commit, the score is 59 of 77, and that is what the regression gate asserts on. Raw direction is not the gate: this suite is plaintiff-heavy, so a caller that always said plaintiff would score most of it without reading anything. We track per-class recall and calibration instead.

The 100 scored rows are full public-record rebuilds. The 900 catalog rows are matters in the shape the console creates - parties, claims, facts, damages and a trial plan from the public caption - but their feature vectors are inherited from one of 19 template cases chosen by program label, not derived from their own record. Most of a catalog row's call follows from that template, which is why those rows are listed for coverage and excluded from the score.

Real juries also split inside these programs. If even real juries split on near-identical records, the reliable move is a paired test: the same juries hear both versions of your case.

Compensatory damages on a single plaintiff are in scope. Punitive and multi-plaintiff awards are not. A row with no plaintiff share is marked not seated. It is never scored as a 0% defense miss.

Now run your case.

The same engine, on the record you are actually trying.

Read the method Open the console