peptiderepo 3820bdfd28 ci: reviewer cost cap + 3-model A/B harness (#3) ...

* ci: cost cap + 3-model A/B harness on CTO reviewer

Two enhancements to the PR reviewer:

1. Monthly spend cap: tracked in Repo Variables (CTO_REVIEW_MONTH,
   CTO_REVIEW_SPEND_CENTS, CTO_REVIEW_CAP_CENTS). Default cap $10/month.
   Auto-resets on month rollover. Posts a skip-comment on PRs when cap hit.

2. A/B harness: reviews every PR with 3 models in parallel
   (Claude Haiku 4.5, DeepSeek V3.1, Kimi K2 Thinking). Comment shows
   all 3 verdicts side-by-side so we can compare quality/cost on real
   PRs before picking a permanent default. Drop to 1 model by editing
   the MODELS list after A/B window closes.

Agent-Session: cowork-reviewer-ab-20260414

* ci: use CTO_ADMIN_TOKEN for repo variables + restore formal APPROVE/REQUEST_CHANGES

Default GITHUB_TOKEN cannot mutate Repository Variables (always 403).
Use a PAT-backed secret CTO_ADMIN_TOKEN for the budget counter steps.
Also restores formal APPROVE/REQUEST_CHANGES verdicts now that repo
setting can_approve_pull_request_reviews is enabled.

Agent-Session: cowork-reviewer-ab-20260414

* ci: address review feedback — per-model timeout + simpler cost math

- Per-model timeout of 60s so one hung model does not block siblings
- Simplify cost_cents math (cost_usd * 100 directly, no double-rounding)

Defer: fine-grained PAT for CTO_ADMIN_TOKEN (requires manual OAuth device flow).

Agent-Session: cowork-reviewer-ab-20260414

* ci: stop exposing CTO_ADMIN_TOKEN as env var (security cleanup)

Pass the PAT only via github-token param. continue-on-error on the
budget step so a missing/expired secret degrades gracefully instead
of failing the whole workflow.

Agent-Session: cowork-reviewer-ab-20260414

2026-04-14 17:35:10 +07:00

19 KiB

Raw Blame History

View Raw