> For the complete documentation index, see [llms.txt](https://docs.zerve.ai/zerve-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.zerve.ai/zerve-docs/superagents/batches/how-judging-works.md).

# How judging works

How judges compare and rank work in Ideation, Tournament and judged Hill-climb batches.

When there's no single number to decide which piece of work is best, superagents use **judges**: agents whose only job is to compare entrants and decide which is better. Judging is used in [Ideation](/zerve-docs/superagents/batches/ideation.md), [Tournament](/zerve-docs/superagents/batches/tournament.md) and judged [Hill-climb](/zerve-docs/superagents/batches/hill-climb.md) batches.

## Head-to-head comparisons

Judges don't score each entrant on its own. Instead, they compare two entrants at a time against written **judging criteria**, such as accuracy, clarity and how actionable a memo is, and decide which is better. Comparing pairs is generally more reliable than asking for an absolute score.

An **ensemble** of judges takes part, rather than a single one, so no one judge's preferences decide the outcome.

## Choosing which pairs to compare

Comparing every entrant against every other would take too long with many entrants. Instead, a rating system called **Glicko-2** keeps a running estimate of each entrant's strength, and uses it to pick the comparisons that will be most informative next, such as two entrants whose ratings are close.

## Settling the final ranking

When judging is done, a statistical model called **Bradley-Terry** turns all the comparisons into a final ranking. Each entrant gets a strength score with a **confidence range**, so you can see not only which entrant ranked first but how clearly it beat the others.

While a batch is still running, rankings are **provisional** and may change.

## Budgets

Judging can't go on forever. Every judged batch has a limit: a maximum number of comparisons, a maximum amount of time, or both.

## Judges and the team

Judges are created only for the batch they're judging. They appear on the Map while they work, but they don't count toward the superagent's team size.

## Reading the results

The **ranking explorer** shows the outcome:

| What it shows                     | How to read it                                                                                         |
| --------------------------------- | ------------------------------------------------------------------------------------------------------ |
| **Strength and confidence range** | How strong each entrant is. Overlapping ranges mean the judges couldn't clearly separate two entrants. |
| **Wins, losses and draws**        | Each entrant's record across its comparisons                                                           |
| **Head-to-head grid**             | How every entrant fared against every other                                                            |

For a judged hill-climb, the **run explorer** shows the leading entrant in each round, how the ratings changed over time, and whether the judges' verdicts were settling down.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.zerve.ai/zerve-docs/superagents/batches/how-judging-works.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.