Eight Steps to Evaluate an LLM Before Production — the Process GitHub Used to Cut False Positives 95%

“This prompt feels better” is where most LLM evaluation stops. On 2026-08-25 GitHub published how they do it as disciplined engineering. Verified 2026-09-08.

The result first: on their own secret scanning, they reported a 95% reduction in false positives on the evaluated offline dataset, while keeping recall within a defined guardrail.

That second clause carries the weight — cutting false positives is easy if you accept misses; holding recall at the same time is what makes it an improvement.

1. The eight steps, in order

GitHub’s lifecycle: product decision → representative dataset → offline evaluation → error analysis → targeted change → regression evaluation → online experiment, with feedback loops throughout.

1. Define the product goal before touching the model

Write down three things:

• Primary success metric (e.g. reduce false positives)
• Safety constraint (e.g. minimum recall threshold)
• Operational guardrails (latency, cost, reliability)

Without those three lines, every later comparison lacks a standard.

2. Change one variable at a time

Record for every run: prompt version, model version, key metrics (precision, recall, latency), and the known baseline.

GitHub’s rule is to “change one major variable at a time” — otherwise you learn that something improved without learning what did it.

3. Make the eval data look like production

Keep offline evaluation close to the real workflow: surrounding context, ambiguous cases, realistic distractions — not clean isolated examples.

This is the common failure: idealised items produce a high score, then real inputs break it.

4. Have humans look at the labels

GitHub’s question: “Does it match the question the evaluation is trying to answer?”

Production labels capture workflow outcomes, which are not necessarily ground truth. Sample and verify manually.

5. Use a hybrid dataset

Stack three sources: production data + synthetic examples targeting rare failure patterns + academic benchmarks and open datasets to fill coverage gaps.

6. Classify errors instead of counting them

Sample failures manually and categorise by source: model, prompt, input framing, pipeline, dataset, labelling.

This step decides what you fix next. Skip it and every problem looks like “the model isn’t good enough.”

7. Use LLM-as-judge for triage

Automated classification routes cases three ways: clear cases handled automatically, low-confidence or high-impact cases to humans, periodic sampling to catch systematic errors.

Note the ordering: the judge is a triage tool that saves human effort, not a replacement for step 4.

8. Always compare against a baseline

A number without a baseline says nothing. That is exactly how GitHub reported theirs: 95% fewer false positives, on the evaluated offline dataset, relative to a baseline — not “it got better.”

2. Readiness checklist

As published:

• Product decision and success metrics clearly defined
• Evaluation data resembles the production workflow
• Prompt, model and dataset versions recorded
• False positives and negatives reviewed by category
• Offline results compared against a known baseline

All five, or it has not been evaluated.

3. The scaled-down version for individuals

You are not GitHub and do not need the full pipeline. The logic still holds at personal scale:

1. Write down the one thing it must do well, and the one thing you will not sacrifice (step 1).
2. Collect ten real examples — from your actual work, not imagined ones (step 3).
3. Change one thing at a time and re-run the same ten (step 2).
4. Categorise the failures: model limitation, unclear instruction, or missing context (step 6).

Step 2 alone settles most “which one feels better” arguments.

4. One last thing

GitHub’s framing: evaluation is continuous engineering, not a one-time gate before launch.

Models update, input distributions drift, users invent uses you never considered. Keep those ten examples and re-run them on every change — the cheapest and most effective sentence in the whole piece.

Related: what AI2 found by dismantling 34,000 benchmark questions and the AI jargon glossary covering harnesses and hill climbing.

Source read directly on 2026-09-08: GitHub’s official post How to evaluate LLMs before production (2026-08-25). The eight-step lifecycle, per-step principles, readiness checklist and the secret-scanning result (“95% reduction in false positives on the evaluated offline dataset while keeping recall within our defined guardrail”) are all as published. We did not reproduce the evaluation; the 95% is GitHub’s own offline result. Section 3 is our adaptation.