Compare commits

..

3 Commits

Author SHA1 Message Date
peptiderepo
9366a017b8 fix: restore 3 bootstrap constants dropped in v0.19.0 — admin 500 hotfix (v0.20.1)
Prod incident 2026-06-12: all plugin admin pages 500'd after the v0.20.0
deploy (Undefined constant PRAUTOBLOGGER_DEFAULT_RUN_CEILING_USD then
..._REQUEST_JSON_RETENTION_DAYS at class-activator.php:71-72 on the
migration path). Root cause: PR #156 (v0.19.0) deleted the v0.18.x trio
from the constants block; tests/bootstrap.php fallbacks masked it in CI.
Values restored verbatim from v0.18.3 (22ae7e7). Prod hot-patch applied
15:1x UTC, verified (board+dossier 200, db 1.3.0); this makes it canonical.

Agent-Session: cto-constants-hotfix-20260612
2026-06-12 15:23:06 +00:00
peptiderepo
2d641153c7 ci(review): drop multi-model panel to Haiku-only (#162)
QA A/B decision 2026-06-12 (CEO-ratified): Haiku-only. Evidence: convo/cto/threads/2026-06-qa-ab-review/. QA verdict per repo in qa-reviews/.

Agent-Session: cto-panel-merge-20260612
2026-06-12 21:53:35 +07:00
peptiderepo
ac2f20de7e feat: M3 edit + single-step re-run — input versioning, human-modified audit, staged re-run (v0.20.0) (#161)
QA: REJECT@6ed639d (CTO rebase header P1) -> APPROVE@6727e07 (qa-reviews/prautoblogger/2026-06-12-6727e07.md)
CPO guardrails 1-5 verified hard-AC. CEO conditional greenlight 2026-06-12 in chat (QA-APPROVE condition met).

Agent-Session: cto-m3-merge-20260612
2026-06-12 21:46:49 +07:00
3 changed files with 109 additions and 216 deletions

View File

@@ -1,20 +1,17 @@
name: CTO Review name: CTO Review
# Automated PR review powered by Claude + DeepSeek + Kimi (via OpenRouter) # Automated PR review powered by Claude Haiku (via OpenRouter).
# plus Neurometric PR-Summarizer (a Qwen3-4B task-specialist SLM).
# #
# During the A/B window (until further notice), every PR gets reviewed by FOUR models # A/B panel decision 2026-06-12: panel dropped to Haiku-only.
# in parallel. The PR comment shows each model's verdict side-by-side so we can compare # Evidence: convo/cto/threads/2026-06-qa-ab-review/02-analyst-findings.md
# quality/cost on real PRs before picking a permanent default. # DeepSeek: high false-positive rate (absint()="code exec", PLUGIN_DIR="dir traversal").
# Kimi: ~60% model failure rate across 13 PRs sampled.
# Neurometric: summarizer only, zero unique-real findings, unreliable heuristic verdict.
# Haiku: 0 failures, lowest noise, equivalent real-signal catch rate vs 4-voice panel.
# #
# Neurometric is a "prose voice": it does not support OpenAI tool-calling, so its # Does NOT merge. Posts formal APPROVE/REQUEST_CHANGES/COMMENT reviews.
# contribution is raw markdown + a heuristic verdict. It is never the primary voice.
# #
# Does NOT merge. Does NOT post formal APPROVE/REQUEST_CHANGES reviews — always COMMENT # Requires repo secret: OPENROUTER_API_KEY
# (GITHUB_TOKEN can't formally approve PRs without repo-setting toggle).
#
# Requires repo secrets: OPENROUTER_API_KEY, NEUROMETRIC_API_KEY (optional; the
# Neurometric voice self-skips with an inline error note if the key is missing).
# #
# Cost cap: tracked in Repository Variables (CTO_REVIEW_SPEND_CENTS, CTO_REVIEW_CAP_CENTS). # Cost cap: tracked in Repository Variables (CTO_REVIEW_SPEND_CENTS, CTO_REVIEW_CAP_CENTS).
# Default cap is 1000 cents ($10/month). Auto-resets on the 1st of each month. # Default cap is 1000 cents ($10/month). Auto-resets on the 1st of each month.
@@ -125,8 +122,8 @@ jobs:
HEAD_SHA="${{ github.event.pull_request.head.sha }}" HEAD_SHA="${{ github.event.pull_request.head.sha }}"
# Write the diff to a file, not a step output / env var. Passing a # Write the diff to a file, not a step output / env var. Passing a
# ~400KB diff through the process env (execve argv+envp) exceeds the # ~400KB diff through the process env (execve argv+envp) exceeds the
# runner ARG_MAX and fails the next step with "Argument list too long" # runner ARG_MAX and fails the next step with "Argument list too long".
# (issue #110). File transport also sidesteps the ~1MB step-output cap. # File transport also sidesteps the ~1MB step-output cap.
git diff --no-color "$BASE_SHA" "$HEAD_SHA" | head -c 400000 > /tmp/cto-review-diff.txt git diff --no-color "$BASE_SHA" "$HEAD_SHA" | head -c 400000 > /tmp/cto-review-diff.txt
FILES=$(git diff --name-only "$BASE_SHA" "$HEAD_SHA") FILES=$(git diff --name-only "$BASE_SHA" "$HEAD_SHA")
SIZE=$(wc -c < /tmp/cto-review-diff.txt) SIZE=$(wc -c < /tmp/cto-review-diff.txt)
@@ -151,12 +148,11 @@ jobs:
echo "EOF_CONV" echo "EOF_CONV"
} >> "$GITHUB_OUTPUT" } >> "$GITHUB_OUTPUT"
- name: Call 4 models in parallel (OpenRouter + Neurometric) - name: Call Haiku via OpenRouter
if: steps.budget.outputs.skip != 'true' if: steps.budget.outputs.skip != 'true'
id: review id: review
env: env:
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }} OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
NEUROMETRIC_API_KEY: ${{ secrets.NEUROMETRIC_API_KEY }}
PR_TITLE: ${{ github.event.pull_request.title }} PR_TITLE: ${{ github.event.pull_request.title }}
PR_BODY: ${{ github.event.pull_request.body }} PR_BODY: ${{ github.event.pull_request.body }}
PR_AUTHOR: ${{ github.event.pull_request.user.login }} PR_AUTHOR: ${{ github.event.pull_request.user.login }}
@@ -184,24 +180,17 @@ jobs:
fi fi
python3 - <<'PY' python3 - <<'PY'
import json, os, sys, urllib.request, concurrent.futures import json, os, sys, urllib.request
api_key = os.environ["OPENROUTER_API_KEY"] api_key = os.environ["OPENROUTER_API_KEY"]
# A/B harness: four models. Each gets the same prompt. # Haiku-only reviewer. A/B panel dropped 2026-06-12 per QA data:
# Primary model's verdict drives the heading. Others are labeled. # DeepSeek high false-positive rate, Kimi ~60% failure rate,
# Edit this list to change the roster (or drop to 1 after A/B window closes). # Neurometric zero unique-real findings. Haiku retained as sole voice.
# MODEL_SLUG = "anthropic/claude-haiku-4.5"
# provider_type semantics: MODEL_DISPLAY = "Claude Haiku 4.5"
# - "openrouter_tools": OpenAI tool-calling via OpenRouter; structured verdict. INPUT_PER_M_USD = 1.00
# - "neurometric_prose": Neurometric marketplace SLM; prose output + heuristic verdict. OUTPUT_PER_M_USD = 5.00
MODELS = [
# (slug, display name, input_per_M_USD, output_per_M_USD, is_primary, provider_type)
("anthropic/claude-haiku-4.5", "Claude Haiku 4.5", 1.00, 5.00, True, "openrouter_tools"),
("deepseek/deepseek-chat-v3.1", "DeepSeek V3.1", 0.15, 0.75, False, "openrouter_tools"),
("moonshotai/kimi-k2-thinking", "Kimi K2 Thinking", 0.60, 2.50, False, "openrouter_tools"),
("neurometric/pr-summarizer", "Neurometric PR-Summarizer (Qwen3-4B)", 0.00, 0.00, False, "neurometric_prose"),
]
system = """You are the CTO and lead reviewer for the peptiderepo project. You are reviewing a pull request opened by an autonomous agent or human contributor. system = """You are the CTO and lead reviewer for the peptiderepo project. You are reviewing a pull request opened by an autonomous agent or human contributor.
@@ -264,218 +253,100 @@ jobs:
} }
}] }]
PER_MODEL_TIMEOUT = 60 # seconds; one hung model should not block siblings body = {
"model": MODEL_SLUG,
"max_tokens": 2048,
"messages": [
{"role": "system", "content": system},
{"role": "user", "content": user_message}
],
"tools": tools,
"tool_choice": {"type": "function", "function": {"name": "submit_review"}}
}
req = urllib.request.Request(
"https://openrouter.ai/api/v1/chat/completions",
data=json.dumps(body).encode(),
headers={
"Authorization": f"Bearer {api_key}",
"content-type": "application/json",
"HTTP-Referer": "https://github.com/peptiderepo",
"X-Title": f"peptiderepo CTO reviewer ({MODEL_DISPLAY})"
},
method="POST"
)
# Heuristic verdict inference for prose-voice models that can't do tool-calling. error = None
# Keyword order matters: block markers beat approve markers. review = None
PROSE_BLOCK_MARKERS = [ cost_cents = 0.0
"do not merge", "do not approve", "should not be merged", "reject this", ptok = 0
"critical security", "critical issue", "critical vulnerability", ctok = 0
"blocker", "blocking issue", "request_changes", "request changes",
"**critical**", "sql injection", "must be addressed before",
]
PROSE_APPROVE_MARKERS = [
"safe to merge", "ready to merge", "no blocking issues", "lgtm",
"approve this", "looks good to merge", "no concerns",
]
def infer_verdict_from_prose(text): try:
t = (text or "").lower() with urllib.request.urlopen(req, timeout=60) as resp:
if any(m in t for m in PROSE_BLOCK_MARKERS): data = json.loads(resp.read())
return "REQUEST_CHANGES" except Exception as e:
if any(m in t for m in PROSE_APPROVE_MARKERS): error = str(e)[:300]
return "APPROVE" data = {}
return "COMMENT"
def call_openrouter_tools(slug, display, in_rate, out_rate, is_primary):
body = {
"model": slug,
"max_tokens": 2048,
"messages": [
{"role": "system", "content": system},
{"role": "user", "content": user_message}
],
"tools": tools,
"tool_choice": {"type": "function", "function": {"name": "submit_review"}}
}
req = urllib.request.Request(
"https://openrouter.ai/api/v1/chat/completions",
data=json.dumps(body).encode(),
headers={
"Authorization": f"Bearer {api_key}",
"content-type": "application/json",
"HTTP-Referer": "https://github.com/peptiderepo",
"X-Title": f"peptiderepo CTO reviewer ({display})"
},
method="POST"
)
try:
with urllib.request.urlopen(req, timeout=PER_MODEL_TIMEOUT) as resp:
data = json.loads(resp.read())
except Exception as e:
return {
"slug": slug, "display": display, "is_primary": is_primary,
"provider_type": "openrouter_tools",
"error": str(e)[:300],
"cost_cents": 0,
}
if not error:
choice = data.get("choices", [{}])[0].get("message", {}) choice = data.get("choices", [{}])[0].get("message", {})
tool_calls = choice.get("tool_calls", []) tool_calls = choice.get("tool_calls", [])
if not tool_calls: if not tool_calls:
return { error = f"No tool_call in response. Raw content: {str(choice)[:300]}"
"slug": slug, "display": display, "is_primary": is_primary, else:
"provider_type": "openrouter_tools", try:
"error": f"No tool_call in response. Raw content: {str(choice)[:300]}", review = json.loads(tool_calls[0]["function"]["arguments"])
"cost_cents": 0, except Exception as e:
} error = f"Malformed tool args: {e}"
try: if not error:
review = json.loads(tool_calls[0]["function"]["arguments"]) usage = data.get("usage", {})
except Exception as e: ptok = usage.get("prompt_tokens", 0)
return { ctok = usage.get("completion_tokens", 0)
"slug": slug, "display": display, "is_primary": is_primary, cost_usd = (ptok * INPUT_PER_M_USD / 1_000_000) + (ctok * OUTPUT_PER_M_USD / 1_000_000)
"provider_type": "openrouter_tools", cost_cents = cost_usd * 100
"error": f"Malformed tool args: {e}",
"cost_cents": 0,
}
usage = data.get("usage", {}) # Build comment
ptok = usage.get("prompt_tokens", 0) if error:
ctok = usage.get("completion_tokens", 0) heading_verdict = "ERROR"
cost_usd = (ptok * in_rate / 1_000_000) + (ctok * out_rate / 1_000_000) lines = [f"## CTO Review — {heading_verdict}", ""]
cost_cents = cost_usd * 100 lines.append(f"Model failed: `{error}`")
return {
"slug": slug, "display": display, "is_primary": is_primary,
"provider_type": "openrouter_tools",
"review": review, "prompt_tokens": ptok, "completion_tokens": ctok,
"cost_cents": cost_cents,
}
def call_neurometric_prose(slug, display, is_primary):
nm_key = os.environ.get("NEUROMETRIC_API_KEY", "")
if not nm_key:
return {
"slug": slug, "display": display, "is_primary": is_primary,
"provider_type": "neurometric_prose",
"error": "NEUROMETRIC_API_KEY secret not set on this repo",
"cost_cents": 0,
}
body = {
"model": slug,
"max_tokens": 2048,
"messages": [
{"role": "system", "content": system},
{"role": "user", "content": user_message},
],
}
req = urllib.request.Request(
"https://api.neurometric.ai/v1/chat/completions",
data=json.dumps(body).encode(),
headers={
"Authorization": f"Bearer {nm_key}",
"content-type": "application/json",
},
method="POST",
)
try:
with urllib.request.urlopen(req, timeout=PER_MODEL_TIMEOUT) as resp:
data = json.loads(resp.read())
except Exception as e:
return {
"slug": slug, "display": display, "is_primary": is_primary,
"provider_type": "neurometric_prose",
"error": str(e)[:300],
"cost_cents": 0,
}
prose = data.get("choices", [{}])[0].get("message", {}).get("content") or ""
verdict = infer_verdict_from_prose(prose)
return {
"slug": slug, "display": display, "is_primary": is_primary,
"provider_type": "neurometric_prose",
"review": {"verdict": verdict, "prose": prose},
"prompt_tokens": 0, "completion_tokens": 0,
"cost_cents": 0,
}
def call_model(slug, display, in_rate, out_rate, is_primary, provider_type="openrouter_tools"):
if provider_type == "neurometric_prose":
return call_neurometric_prose(slug, display, is_primary)
return call_openrouter_tools(slug, display, in_rate, out_rate, is_primary)
results = []
with concurrent.futures.ThreadPoolExecutor(max_workers=len(MODELS)) as pool:
futures = [pool.submit(call_model, *m) for m in MODELS]
for f in concurrent.futures.as_completed(futures):
results.append(f.result())
# Order: primary first, then alphabetical for stable comparison
results.sort(key=lambda r: (not r.get("is_primary"), r["display"]))
# Build combined comment
primary = next((r for r in results if r.get("is_primary") and "review" in r), None)
heading_verdict = primary["review"]["verdict"] if primary else "MIXED"
lines = [f"## CTO Review — {heading_verdict}", ""]
lines.append(f"_A/B harness active: {len(results)} models reviewing in parallel. Primary verdict drives heading; others are advisory._")
lines.append("")
total_cents = 0.0
for r in results:
lines.append(f"---")
primary_tag = " (primary)" if r.get("is_primary") else ""
lines.append(f"### {r['display']}{primary_tag}")
if "error" in r:
lines.append("")
lines.append(f"⚠️ Model failed: `{r['error']}`")
lines.append("")
continue
rev = r["review"]
lines.append("") lines.append("")
if r.get("provider_type") == "neurometric_prose": lines.append(f"_reviewer: {MODEL_DISPLAY}_")
lines.append(f"**Verdict:** {rev['verdict']} _(heuristic, inferred from prose)_") else:
lines.append("") heading_verdict = review.get("verdict", "COMMENT")
# Cap at 6KB to keep PR comments readable; 4B model output is rarely over this lines = [f"## CTO Review — {heading_verdict}", ""]
prose = rev.get("prose", "") lines.append(f"**Verdict:** {heading_verdict}")
if len(prose) > 6000:
prose = prose[:6000] + "\n\n_…output truncated for comment length…_"
lines.append(prose)
lines.append("")
lines.append("_free tier; per-call token usage not tracked_")
lines.append("")
continue
lines.append(f"**Verdict:** {rev.get('verdict', 'COMMENT')}")
lines.append("") lines.append("")
lines.append(rev.get("summary") or "_(model returned no summary)_") lines.append(review.get("summary") or "_(model returned no summary)_")
lines.append("") lines.append("")
if rev.get("concerns"): if review.get("concerns"):
lines.append("**Concerns:**") lines.append("**Concerns:**")
for c in rev["concerns"]: for c in review["concerns"]:
sev = c.get("severity", "UNKNOWN").upper() sev = c.get("severity", "UNKNOWN").upper()
f = f"`{c['file']}`" if c.get("file") else "" f = f"`{c['file']}`" if c.get("file") else ""
lines.append(f"- **{sev}** {f} — {c['issue']}") lines.append(f"- **{sev}** {f} — {c['issue']}")
lines.append("") lines.append("")
if rev.get("positives"): if review.get("positives"):
lines.append("**Positives:** " + "; ".join(rev["positives"])) lines.append("**Positives:** " + "; ".join(review["positives"]))
lines.append("") lines.append("")
lines.append(f"_tokens in/out: {r['prompt_tokens']}/{r['completion_tokens']} · cost: ${r['cost_cents']/100:.4f}_") lines.append(f"_tokens in/out: {ptok}/{ctok} · cost: ${cost_cents/100:.4f}_")
total_cents += r["cost_cents"]
lines.append("") lines.append("")
lines.append("---") lines.append("---")
lines.append(f"_Total review cost: **${total_cents/100:.4f}**. A/B roster configured in `.github/workflows/cto-review.yml`._") lines.append(f"_Total review cost: **${cost_cents/100:.4f}**. Reviewer: {MODEL_DISPLAY}. Roster configured in `.github/workflows/cto-review.yml`._")
with open("/tmp/review-body.md", "w") as f: with open("/tmp/review-body.md", "w") as f:
f.write("\n".join(lines)) f.write("\n".join(lines))
with open(os.environ["GITHUB_OUTPUT"], "a") as f: with open(os.environ["GITHUB_OUTPUT"], "a") as f:
f.write(f"added_cents={int(round(total_cents))}\n") f.write(f"added_cents={int(round(cost_cents))}\n")
f.write(f"skipped=0\n") f.write(f"skipped=0\n")
f.write(f"primary_verdict={heading_verdict}\n") f.write(f"primary_verdict={heading_verdict}\n")
PY PY
- name: Post PR review (formal APPROVE / REQUEST_CHANGES / COMMENT) - name: Post PR review (formal APPROVE / REQUEST_CHANGES / COMMENT)
# Uses primary model's verdict as the formal event. Falls back to COMMENT # Uses Haiku's verdict as the formal event. Falls back to COMMENT
# if the primary model failed or returned something unexpected. # if the model failed or returned something unexpected.
if: steps.budget.outputs.skip != 'true' && steps.review.outputs.skipped != '1' if: steps.budget.outputs.skip != 'true' && steps.review.outputs.skipped != '1'
uses: actions/github-script@v7 uses: actions/github-script@v7
env: env:

View File

@@ -5,6 +5,20 @@ All notable changes to PRAutoBlogger will be documented in this file.
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project uses [Semantic Versioning](https://semver.org/). and this project uses [Semantic Versioning](https://semver.org/).
## [0.20.1] - 2026-06-12
### Fixed
- **Restore three bootstrap constants deleted by the v0.19.0 constants-block regression**
(`PRAUTOBLOGGER_DEFAULT_RUN_CEILING_USD` 0.50, `PRAUTOBLOGGER_DEFAULT_REQUEST_JSON_RETENTION_DAYS` 14,
`PRAUTOBLOGGER_DEFAULT_REASONING_MAX_TOKENS` 2048 — verbatim v0.18.3 values). PR #156 (v0.19.0)
dropped them from `prautoblogger.php`; the gap was latent because `tests/bootstrap.php` defines
its own fallbacks (CI green) and no exercised v0.19.x runtime path read them (prod reasoning
disabled). v0.20.0's activator defaults + run-state ceiling read them on the admin
migration path -> fatal 500 on all plugin admin pages immediately after the v0.20.0 deploy
(2026-06-12; front-of-site unaffected; hot-patched on prod within minutes, this commit makes
it canonical). Follow-up owed: CI guard asserting every referenced PRAUTOBLOGGER_* constant
is production-defined or defined()-guarded (maintenance thread).
## [0.20.0] - 2026-06-12 ## [0.20.0] - 2026-06-12
### Added ### Added

View File

@@ -9,7 +9,7 @@
* Plugin Name: PRAutoBlogger * Plugin Name: PRAutoBlogger
* Plugin URI: https://peptiderepo.com/prautoblogger * Plugin URI: https://peptiderepo.com/prautoblogger
* Description: Monitors social media for trending topics, generates SEO-friendly blog posts using AI, and publishes them on a daily schedule with full cost tracking and self-improvement metrics. * Description: Monitors social media for trending topics, generates SEO-friendly blog posts using AI, and publishes them on a daily schedule with full cost tracking and self-improvement metrics.
* Version: 0.20.0 * Version: 0.20.1
* Requires at least: 6.0 * Requires at least: 6.0
* Requires PHP: 7.4 * Requires PHP: 7.4
* Author: PeptideRepo * Author: PeptideRepo
@@ -35,8 +35,16 @@ if ( ! defined( 'ABSPATH' ) ) {
| Defined here so every file in the plugin can reference paths, versions, | Defined here so every file in the plugin can reference paths, versions,
| and limits without magic strings. | and limits without magic strings.
*/ */
define( 'PRAUTOBLOGGER_VERSION', '0.20.0' ); define( 'PRAUTOBLOGGER_VERSION', '0.20.1' );
define( 'PRAUTOBLOGGER_DB_VERSION', '1.3.0' ); define( 'PRAUTOBLOGGER_DB_VERSION', '1.3.0' );
define( 'PRAUTOBLOGGER_DEFAULT_RUN_CEILING_USD', 0.50 );
define( 'PRAUTOBLOGGER_DEFAULT_REQUEST_JSON_RETENTION_DAYS', 14 );
// v0.18.1: ceiling on "thinking" tokens per reasoning-enabled LLM call. The
// request's completion budget (max_tokens) is RAISED by this amount so
// reasoning can never consume the visible-content budget and return an empty
// message (the 2026-06-11 empty-draft incident). SETTINGS-backed
// (prautoblogger_reasoning_max_tokens); 0 disables the cap (pure effort mode).
define( 'PRAUTOBLOGGER_DEFAULT_REASONING_MAX_TOKENS', 2048 );
define( 'PRAUTOBLOGGER_PLUGIN_FILE', __FILE__ ); define( 'PRAUTOBLOGGER_PLUGIN_FILE', __FILE__ );
define( 'PRAUTOBLOGGER_PLUGIN_DIR', plugin_dir_path( __FILE__ ) ); define( 'PRAUTOBLOGGER_PLUGIN_DIR', plugin_dir_path( __FILE__ ) );
define( 'PRAUTOBLOGGER_PLUGIN_URL', plugin_dir_url( __FILE__ ) ); define( 'PRAUTOBLOGGER_PLUGIN_URL', plugin_dir_url( __FILE__ ) );