LLM-as-a-Judge & Verifier Models
TrialFactorial rates LLM-as-a-Judge as Trial (2026). Atelier's help assistant uses a verifier: a second model call checks every claim in a draft answer against the documentation for the installed version, removes what the docs do not back, and checks the shortened answer again.
In the LLM-as-a-judge pattern a separate model call grades or checks another model's output. It is used offline to score results in evals, and at runtime as a verifier that checks an answer against its sources before a user sees it. The judge works best with a narrow question and only the evidence it needs.
Updates
Atelier's help assistant uses a verifier: a second model call checks every claim in a draft answer against the documentation for the installed version, removes what the docs do not back, and checks the shortened answer again. If it still cannot confirm the answer, the user gets an honest "I couldn't confirm that" with the closest page. In a test set of 20 real questions run three times each, it answered correctly in 59 of 60 runs. The verifier only sees the draft and the docs, not the conversation, which keeps it from trusting the first model's reasoning.