OpenRouter 官方最新动态：How to Gate Pull Requests on LLM Evals in CI

来源：OpenRouter 官方动态 | 发布日期：2026-10-01T00:00:00.000Z

核心更新概览

How to Gate Pull Requests on LLM Evals in CI

详细内容记录

How to Gate Pull Requests on LLM Evals in CI What you need before you can gate anything Test an agent that calls tools with Ori Eval Compare a plain script with eval platforms Changing one line in a support agent’s system prompt can ship an agent that tells customers the refund window is 30 days when your policy says 14. Nothing in a normal CI pipeline checks what the model says, so the build passes and the first person to see the wrong answer is a customer. Gating a pull request on a fixed eval set works the same way as gating on a failing unit test. You keep test cases in the repository, run them when a prompt changes, and block the merge when too many fail. In this guide, you write an eval set for a support agent and score it with a script that calls OpenRouter. You measure how much the results move between runs with nothing changed, then wire the script into GitHub Actions as a required status check. A fixed eval set is a list of test cases committed to your repository. It changes only through a reviewed pull request.

更多技术细节可访问官方原文：https://openrouter.ai/blog/tutorials/how-to-gate-pull-requests-on-llm-evals-in-ci/。