peptide-repo/prautoblogger
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: empty-content guard + finish_reason=length handling + reasoning budget (v0.18.1)

Browse Source 
Reasoning models at high effort could spend the entire max_tokens budget
on thinking and return zero visible content with finish_reason=length;
the empty article flowed writer -> editor (reject) -> empty draft post,
with the cost booked as success (2026-06-11 incident: prod run acf24029,
draft post 921; diagnosis: convo pipeline-v2-phase1 seq 12).

- Empty-completion guard at the provider seam (new
  OpenRouter_Completion_Guard): empty/whitespace content is a failure;
  one retry with reasoning disabled (per-call override) when
  finish_reason=length or reasoning was active, then the stage fails.
- finish_reason=length always warning-logged (model, stage, tokens).
- Failed/retried attempts booked to generation_log with
  response_status=error + real token burn; run_decisions verdict rows
  (retry_reasoning_disabled / failed_empty_completion) for v0.18.0 runs.
- Writer-path guard in Content_Generator::execute_stage and
  belt-and-braces empty check in Publisher::create_post.
- Reasoning budget sanity: reasoning.max_tokens cap + completion
  headroom via new prautoblogger_reasoning_max_tokens setting (default
  constant 2048, 0 = legacy uncapped effort mode); cap replaces effort
  when active; governor reservations use the effective ceiling.
- Body assembly moved to Request_Builder::build_body (300-line cap).
- Tests: guard verdict matrix, builder budget shaping, writer/publisher
  guards, full provider retry path.

Agent-Session: prab-empty-draft-guard-20260611
This commit is contained in:
peptiderepo
2026-06-11 13:15:58 +00:00
committed by peptiderepo
parent f5658f1c97
commit da0250f3ba
19 changed files with 1196 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files

47
CHANGELOG.md
View File

@@ -5,6 +5,53 @@ All notable changes to PRAutoBlogger will be documented in this file.
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project uses [Semantic Versioning](https://semver.org/).
## [0.18.1] - 2026-06-11
Empty-draft hotfix (CTO diagnosis: convo thread `2026-06-pipeline-v2-phase1` seq 12; prod run
`acf24029`, draft post 921). A true reasoning model at a high reasoning effort could spend the
entire `max_tokens` completion budget on thinking, return zero visible content with
`finish_reason=length`, and the empty article flowed writer → editor (reject: "draft text is
missing") → empty draft post in the Review Queue, with the cost booked as `success`.
### Fixed
- **Empty-completion guard at the call seam** (`PRAutoBlogger_OpenRouter_Completion_Guard`):
a chat completion whose visible content is empty/whitespace is now a **failure**, never a
success. When the empty result co-occurs with `finish_reason=length` or an active reasoning
request, the call is retried **once** with reasoning disabled (per-call `reasoning`
override) before the stage fails. Failed/retried attempts land in `generation_log` with
`response_status='error'`, the failure note, and the real token burn; inside a v0.18.0 run a
`run_decisions` row records the verdict (`retry_reasoning_disabled` /
`failed_empty_completion`).
- **Writer-path guard**: a writing stage (outline/draft/polish/single-pass) whose output is
empty throws instead of passing an empty article to the editor (belt over the seam guard,
covers any future non-OpenRouter provider).
- **Publisher belt-and-braces**: `Publisher::create_post()` refuses to create any post —
publish or draft — whose content is empty after tag stripping. No empty draft post can ever
be created again.
- `finish_reason=length` is now always warning-logged with model, stage, and
prompt/completion/reasoning token counts (previously silent — the truncated `llm_research`
call in the incident run was invisible).
### Added
- **Reasoning token cap + completion headroom**: when a request enables reasoning, the
thinking budget is capped via OpenRouter's `reasoning.max_tokens` and the request's
`max_tokens` is **raised** by the same amount, so no reasoning effort (incl. `xhigh`) can
consume the visible-content budget. New setting `prautoblogger_reasoning_max_tokens`
(Settings → AI Models, default `PRAUTOBLOGGER_DEFAULT_REASONING_MAX_TOKENS` = 2048; 0
restores the previous uncapped effort mode). When the cap is active it replaces `effort`
(OpenRouter treats them as alternative budget controls); the `xhigh`/`high` effort options
stay available but are bounded by default. Cost-governor reservations use the effective
(raised) ceiling.
- LLM call sites now pass their pipeline stage to the provider (`stage` option, never sent
upstream) so guard warnings and failure rows are attributable: `analysis`, `review`,
`llm_research`, `image_prompt_rewrite`, `opik_eval_judge`, and the writer stages (which
also pass `prompt_key`).
### Notes
- `OpenRouter_Provider::send_chat_completion()` body assembly moved to
`OpenRouter_Request_Builder::build_body()` (300-line cap; behavior unchanged apart from the
reasoning budget above).
## [0.18.0] - 2026-06-12
Pipeline v2 — Phase 1 substrate (plan of record: CPO thread

12
includes/admin/class-settings-fields-extended.php
View File

@@ -78,7 +78,17 @@ class PRAutoBlogger_Settings_Fields_Extended {
'low' => __( 'Low — light reasoning, lower cost', 'prautoblogger' ),
'minimal' => __( 'Minimal — barely any reasoning', 'prautoblogger' ),
),
'description' => __( 'How much effort the model spends reasoning before answering. Higher effort = more reasoning tokens = higher cost. Only applies when reasoning is enabled above.', 'prautoblogger' ),
'description' => __( 'How much effort the model spends reasoning before answering. Higher effort = more reasoning tokens = higher cost. Only applies when reasoning is enabled above. When the Reasoning Token Cap below is set (default), the cap is sent instead of the effort level — set the cap to 0 to use effort only.', 'prautoblogger' ),
),
array(
'id' => 'prautoblogger_reasoning_max_tokens',
'label' => __( 'Reasoning Token Cap', 'prautoblogger' ),
'type' => 'number',
'section' => 'prautoblogger_models',
'default' => PRAUTOBLOGGER_DEFAULT_REASONING_MAX_TOKENS,
'min' => 0,
'max' => 32000,
'description' => __( 'Hard ceiling on thinking tokens per call when reasoning is enabled. The completion budget is raised by this amount so reasoning can never crowd out the article text — an uncapped high effort can spend the entire budget thinking and return an empty draft. 0 = no cap (uncapped effort mode, not recommended).', 'prautoblogger' ),
),
// ── Schedule & Budget ───────────────────────────────────────

1
includes/core/class-chief-editor.php
View File

@@ -64,6 +64,7 @@ class PRAutoBlogger_Chief_Editor {
'temperature' => 0.3,
'max_tokens' => 5000,
'response_format' => array( 'type' => 'json_object' ),
'stage' => 'review',
)
);

1
includes/core/class-content-analyzer.php
View File

@@ -78,6 +78,7 @@ class PRAutoBlogger_Content_Analyzer {
'temperature' => 0.5,
'max_tokens' => 8000,
'response_format' => array( 'type' => 'json_object' ),
'stage' => 'analysis',
)
);

21
includes/core/class-content-generator.php
View File

@@ -197,6 +197,12 @@ class PRAutoBlogger_Content_Generator {
PRAutoBlogger_Run_Stage_State::start( $this->run_id, $stage, '', $this->item_key );
}
// v0.18.1: stage metadata for the provider's empty-completion
// guard — warning logs and failed-attempt audit rows carry the
// real stage + registry key (never sent upstream).
$options['stage'] = $stage;
$options['prompt_key'] = $prompt_key;
$ctx = PRAutoBlogger_Opik_Trace_Context::current();
$span_id = $ctx->start_span(
@@ -223,6 +229,21 @@ class PRAutoBlogger_Content_Generator {
$options
);
// v0.18.1 writer-path guard (belt over the provider seam, which
// already throws for OpenRouter): no empty article may ever leave
// a writing stage — it would flow to the editor, be rejected as
// "draft text is missing", and land as an empty draft post.
if ( '' === trim( (string) $response['content'] ) ) {
throw new \RuntimeException(
sprintf(
/* translators: 1: writing stage name, 2: model identifier. */
__( 'Writing stage "%1$s" produced empty content (model %2$s) — failing the stage instead of passing an empty article downstream.', 'prautoblogger' ),
$stage,
(string) $response['model']
)
);
}
$ctx->end_span(
$span_id,
array(

1
includes/core/class-image-prompt-builder.php
View File

@@ -200,6 +200,7 @@ PROMPT;
array(
'temperature' => 0.7,
'max_tokens' => self::REWRITER_MAX_TOKENS,
'stage' => 'image_prompt_rewrite',
)
);

17
includes/core/class-publisher.php
View File

@@ -82,6 +82,23 @@ class PRAutoBlogger_Publisher {
?string $run_id = null,
?PRAutoBlogger_Cost_Tracker $cost_tracker = null
): int {
// v0.18.1 belt-and-braces: never create a post whose content is
// empty once tags and whitespace are stripped — an empty "draft
// for human review" (2026-06-11 incident, post 921) reviews
// nothing and hides the underlying generation failure. The
// provider/writer guards make this unreachable on OpenRouter
// paths; this protects every other path into post creation.
if ( '' === trim( wp_strip_all_tags( $content ) ) ) {
throw new \RuntimeException(
sprintf(
/* translators: 1: intended post status, 2: article title. */
__( 'Refusing to create %1$s post "%2$s": generated content is empty.', 'prautoblogger' ),
$post_status,
$idea->get_suggested_title()
)
);
}
// v0.18.0 idempotency: post creation is keyed by run + idea
// (check-before-insert) so a retried/resumed run cannot create a
// duplicate post. One run_id spans all N articles of a batch, so

1
includes/providers/class-llm-research-provider.php
View File

@@ -70,6 +70,7 @@ class PRAutoBlogger_LLM_Research_Provider implements PRAutoBlogger_Source_Provid
'temperature' => 0.7,
'max_tokens' => 8000,
'response_format' => array( 'type' => 'json_object' ),
'stage' => 'llm_research',
)
);

168
includes/providers/class-open-router-completion-guard.php Normal file
View File

@@ -0,0 +1,168 @@
<?php
declare(strict_types=1);
/**
* phpcs:ignore WordPress.Files.FileName.InvalidClassFileName -- class naming convention differs from WordPress standard
*
* Empty-completion guard for OpenRouter chat responses (v0.18.1).
*
* What: Treats a chat completion whose visible content is empty/whitespace
* as a FAILURE, never a success — per the LLM-call discipline:
* schema-validate every output, retry/fall back, never silently
* pass. The incident this prevents (2026-06-11, prod run acf24029,
* draft post 921): a true reasoning model at a high reasoning effort
* spent the entire max_tokens completion budget on thinking,
* finished with finish_reason='length', and emitted zero visible
* content — which flowed writer → editor (reject: "draft text is
* missing") → an empty draft post in the Review Queue, with the
* cost booked as 'success'.
*
* Behavior on every parsed response:
* - finish_reason='length' is warning-logged (model, stage, prompt/
* completion/reasoning tokens) whether or not content survived.
* - Empty content + (finish_reason='length' OR reasoning active on
* the request) → ONE retry with reasoning disabled for that call
* (per-call `reasoning` override), then fail the stage.
* - Every failed/retried attempt lands in generation_log with its
* REAL response_status ('error') and token burn, and — inside a
* v0.18.0 run — a run_decisions row records the verdict.
* Who triggers it: PRAutoBlogger_OpenRouter_Provider::send_chat_completion()
* on every successfully parsed response.
* Dependencies: Logger, Cost_Tracker (failure rows), Run_Context (run
* attribution), Audit_Writer (run_decisions verdicts).
*
* @see class-open-router-provider.php — Integration point; supplies the retry callable.
* @see class-open-router-response-parser.php — Produces the parsed array assessed here.
* @see class-open-router-request-builder.php — Reasoning token cap (first line of defense).
* @see core/class-audit-writer.php — run_decisions verdict rows.
*/
class PRAutoBlogger_OpenRouter_Completion_Guard {
/** Verdict recorded when an empty attempt is retried with reasoning disabled. */
public const VERDICT_RETRY = 'retry_reasoning_disabled';
/** Verdict recorded when a call finally fails for empty content. */
public const VERDICT_FAIL = 'failed_empty_completion';
/**
* Assess a parsed completion: pass it through, retry once, or fail loudly.
*
* @param array<string, mixed> $parsed Parsed response from Response_Parser::parse_success().
* @param array<string, mixed> $options Original call options. Reads the caller-metadata keys
* 'stage' / 'prompt_key' (audit attribution) and
* 'empty_retry' (marks the retry attempt — never retried twice).
* @param bool $reasoning_active Whether the dispatched request had reasoning enabled.
* @param callable $retry Re-dispatches the call with the options it is given.
* Invoked at most once, with reasoning disabled.
*
* @return array<string, mixed> A completion whose visible content is non-empty.
*
* @throws \RuntimeException When content is empty and the retry budget is spent or a retry
* cannot plausibly help.
*/
public function finalize( array $parsed, array $options, bool $reasoning_active, callable $retry ): array {
$stage = (string) ( $options['stage'] ?? 'unknown' );
$finish_reason = (string) ( $parsed['finish_reason'] ?? 'unknown' );
if ( 'length' === $finish_reason ) {
PRAutoBlogger_Logger::instance()->warning(
sprintf(
'Completion hit its max_tokens ceiling (finish_reason=length): model=%s, stage=%s, prompt_tokens=%d, completion_tokens=%d, reasoning_tokens=%d.',
(string) $parsed['model'],
$stage,
(int) $parsed['prompt_tokens'],
(int) $parsed['completion_tokens'],
(int) $parsed['reasoning_tokens']
),
'openrouter'
);
}
if ( '' !== trim( (string) ( $parsed['content'] ?? '' ) ) ) {
return $parsed; // Healthy completion — visible content present.
}
$already_retried = ! empty( $options['empty_retry'] );
// A reasoning-disabled retry can plausibly help when the budget was
// exhausted (length) or when reasoning was eating the budget.
$retry_can_help = 'length' === $finish_reason || $reasoning_active;
if ( $retry_can_help && ! $already_retried ) {
$note = sprintf(
'Empty completion (finish_reason=%s, completion_tokens=%d, reasoning_tokens=%d) — retrying once with reasoning disabled.',
$finish_reason,
(int) $parsed['completion_tokens'],
(int) $parsed['reasoning_tokens']
);
$this->record_attempt( $parsed, $options, $stage, self::VERDICT_RETRY, $note );
PRAutoBlogger_Logger::instance()->warning(
sprintf( 'Stage %s: %s (model %s)', $stage, $note, (string) $parsed['model'] ),
'openrouter'
);
$retry_options = $options;
$retry_options['reasoning'] = array( 'enabled' => false );
$retry_options['empty_retry'] = true;
return $retry( $retry_options );
}
$note = sprintf(
'Empty completion (finish_reason=%s, completion_tokens=%d, reasoning_tokens=%d)%s — failing the call.',
$finish_reason,
(int) $parsed['completion_tokens'],
(int) $parsed['reasoning_tokens'],
$already_retried ? ' after a reasoning-disabled retry' : ''
);
$this->record_attempt( $parsed, $options, $stage, self::VERDICT_FAIL, $note );
throw new \RuntimeException(
sprintf(
/* translators: 1: pipeline stage, 2: model identifier, 3: finish_reason value, 4: retry note. */
__( 'LLM call for stage "%1$s" returned empty content (model %2$s, finish_reason=%3$s)%4$s.', 'prautoblogger' ),
$stage,
(string) $parsed['model'],
$finish_reason,
$already_retried ? __( ' after a reasoning-disabled retry', 'prautoblogger' ) : ''
)
);
}
/**
* Book a failed/retried attempt: a generation_log row with the REAL
* response_status and token burn (the empty call still cost money),
* plus a run_decisions verdict for new-style runs. Both writes are
* self-healing no-ops on a half-migrated schema; neither can throw.
*
* @param array<string, mixed> $parsed Parsed response of the failed attempt.
* @param array<string, mixed> $options Call options ('prompt_key' attribution).
* @param string $stage Stage label for the log row.
* @param string $verdict VERDICT_RETRY or VERDICT_FAIL.
* @param string $note Human-readable failure note.
* @return void
*/
private function record_attempt( array $parsed, array $options, string $stage, string $verdict, string $note ): void {
$run_id = PRAutoBlogger_Run_Context::current_run_id();
$tracker = new PRAutoBlogger_Cost_Tracker();
if ( null !== $run_id && '' !== $run_id ) {
// Attribute the row to the surrounding run. Idempotent: the
// run context, ledger row, and prompt pins already exist.
$tracker->set_run_id( $run_id );
}
$tracker->log_api_call(
null,
$stage,
'OpenRouter',
(string) $parsed['model'],
(int) $parsed['prompt_tokens'],
(int) $parsed['completion_tokens'],
'error',
$note,
null,
isset( $options['prompt_key'] ) ? (string) $options['prompt_key'] : null
);
// run_decisions verdict — no-op outside a run or pre-1.2.0 schema.
PRAutoBlogger_Audit_Writer::record_decision( (string) ( $run_id ?? '' ), $stage, $verdict, $note );
}
}

76
includes/providers/class-open-router-provider.php
View File

@@ -19,8 +19,9 @@ declare(strict_types=1);
* @see class-open-router-config.php — Config helpers (base URL, cache TTL).
* @see class-open-router-pricing.php — Pricing and model list lookups.
* @see class-open-router-validator.php — Credential validation (delegated).
* @see class-open-router-request-builder.php — Request header building + cURL filter.
* @see class-open-router-request-builder.php — Request body/header building + cURL filter.
* @see class-open-router-response-parser.php — Response parsing + error classification.
* @see class-open-router-completion-guard.php — Empty-completion guard + reasoning-off retry (v0.18.1).
* @see class-cost-tracker.php — Called after every request to log token usage.
* @see ARCHITECTURE.md — Data flow diagram showing where this fits.
*/
@@ -40,8 +41,15 @@ class PRAutoBlogger_OpenRouter_Provider implements PRAutoBlogger_LLM_Provider_In
* temperature?: float,
* max_tokens?: int,
* response_format?: array{type: string},
* reasoning?: array{enabled: bool, effort?: string},
* } $options Optional parameters. Pass 'reasoning' to override the global setting.
* reasoning?: array{enabled: bool, effort?: string, max_tokens?: int},
* stage?: string,
* prompt_key?: string,
* empty_retry?: bool,
* } $options Optional parameters. Pass 'reasoning' to override the global
* setting. 'stage'/'prompt_key' are caller metadata for the
* empty-completion guard's logging + audit rows (never sent
* upstream); 'empty_retry' is set internally by the guard's
* single reasoning-disabled retry.
*
* @return array{
* content: string,
@@ -52,46 +60,39 @@ class PRAutoBlogger_OpenRouter_Provider implements PRAutoBlogger_LLM_Provider_In
* finish_reason: string,
* }
*
* @throws \RuntimeException On API error after retries exhausted.
* @throws \RuntimeException On API error after retries exhausted, or when the
* completion's visible content is empty/whitespace
* (v0.18.1 guard — after at most one reasoning-disabled retry).
* @throws PRAutoBlogger_Cost_Ceiling_Exception When the per-run cost governor blocks
* the call (run already halted).
*/
public function send_chat_completion( array $messages, string $model, array $options = array() ): array {
$builder = new PRAutoBlogger_OpenRouter_Request_Builder();
// Fetch + format-validate the key (moved to Request_Builder in
// v0.18.0 for the 300-line cap; behavior unchanged — throws the
// same messages on missing/corrupted keys).
$api_key = ( new PRAutoBlogger_OpenRouter_Request_Builder() )->resolve_api_key();
$api_key = $builder->resolve_api_key();
$body = array(
'model' => $model,
'messages' => $messages,
);
// Body assembly — incl. global/per-call reasoning and the v0.18.1
// reasoning token cap + completion headroom — lives in the builder.
$body = $builder->build_body( $messages, $model, $options );
if ( isset( $options['temperature'] ) ) {
$body['temperature'] = $options['temperature'];
}
if ( isset( $options['max_tokens'] ) ) {
$body['max_tokens'] = $options['max_tokens'];
}
if ( isset( $options['response_format'] ) ) {
$body['response_format'] = $options['response_format'];
}
// Reasoning mode: explicit caller override takes priority, then global setting.
if ( isset( $options['reasoning'] ) ) {
$body['reasoning'] = $options['reasoning'];
} elseif ( '1' === get_option( 'prautoblogger_reasoning_enabled', '0' ) ) {
$body['reasoning'] = array(
'enabled' => true,
'effort' => get_option( 'prautoblogger_reasoning_effort', 'medium' ),
);
}
// Whether the outgoing request has reasoning enabled — drives the
// empty-completion retry decision after parse.
$reasoning_active = isset( $body['reasoning'] ) && false !== ( $body['reasoning']['enabled'] ?? true );
// v0.18.0 — per-run cost governor: reserve the worst-case estimate
// BEFORE any dispatch. Throws (and the run is already halted) when
// the reservation would breach the run ceiling; returns null when
// this process is not inside a governed run (historical behavior).
$reservation = PRAutoBlogger_Cost_Governor::open_chat_reservation( $model, $messages, $options );
// v0.18.1: the estimate uses the EFFECTIVE completion ceiling from
// the built body (incl. reasoning headroom), not the raw option.
$govern_options = $options;
if ( isset( $body['max_tokens'] ) ) {
$govern_options['max_tokens'] = $body['max_tokens'];
}
$reservation = PRAutoBlogger_Cost_Governor::open_chat_reservation( $model, $messages, $govern_options );
$last_error = '';
@@ -101,7 +102,6 @@ class PRAutoBlogger_OpenRouter_Provider implements PRAutoBlogger_LLM_Provider_In
$cache_ttl = $config->get_cache_ttl_seconds();
$via_gateway = $config->is_via_gateway();
$builder = new PRAutoBlogger_OpenRouter_Request_Builder();
$request_headers = $builder->build_headers( $api_key, $via_gateway, $cache_ttl );
$curl_auth_filter = $builder->register_curl_auth_filter( $request_headers, $base_host );
@@ -190,7 +190,21 @@ class PRAutoBlogger_OpenRouter_Provider implements PRAutoBlogger_LLM_Provider_In
$this->estimate_cost( $model, (int) $parsed['prompt_tokens'], (int) $parsed['completion_tokens'] )
);
$reservation = null;
return $parsed;
// v0.18.1 — empty-completion guard: empty visible content
// is a FAILURE, never a success. Warns on finish_reason=
// length, retries ONCE with reasoning disabled when that
// can plausibly help, books failed attempts with their
// real status, and otherwise throws. The recursive retry
// manages its own reservation and cURL filter.
return ( new PRAutoBlogger_OpenRouter_Completion_Guard() )->finalize(
$parsed,
$options,
$reasoning_active,
function ( array $retry_options ) use ( $messages, $model ): array {
return $this->send_chat_completion( $messages, $model, $retry_options );
}
);
}
// All retries exhausted.

94
includes/providers/class-open-router-request-builder.php
View File

@@ -18,6 +18,100 @@ declare(strict_types=1);
*/
class PRAutoBlogger_OpenRouter_Request_Builder {
/**
* Build the JSON request body for a chat-completion call.
*
* Centralizes option handling (v0.18.1, moved from the provider for
* the 300-line cap): temperature, max_tokens, response_format, and
* the reasoning block — an explicit caller override takes priority
* over the global setting (that per-call override is how the
* empty-completion retry disables reasoning for its second attempt).
*
* Caller-metadata keys ('stage', 'prompt_key', 'empty_retry') are
* deliberately NOT copied into the HTTP body.
*
* @param array<int, array{role: string, content: string}> $messages Chat messages.
* @param string $model Model identifier.
* @param array<string, mixed> $options Call options.
*
* @return array<string, mixed> Request body ready for wp_json_encode().
*/
public function build_body( array $messages, string $model, array $options ): array {
$body = array(
'model' => $model,
'messages' => $messages,
);
if ( isset( $options['temperature'] ) ) {
$body['temperature'] = $options['temperature'];
}
if ( isset( $options['max_tokens'] ) ) {
$body['max_tokens'] = $options['max_tokens'];
}
if ( isset( $options['response_format'] ) ) {
$body['response_format'] = $options['response_format'];
}
// Reasoning mode: explicit caller override takes priority, then global setting.
if ( isset( $options['reasoning'] ) ) {
$body['reasoning'] = $options['reasoning'];
} elseif ( '1' === get_option( 'prautoblogger_reasoning_enabled', '0' ) ) {
$body['reasoning'] = array(
'enabled' => true,
'effort' => get_option( 'prautoblogger_reasoning_effort', 'medium' ),
);
}
return $this->apply_reasoning_budget( $body );
}
/**
* Reasoning budget sanity (v0.18.1): cap thinking, protect content.
*
* When the outgoing request enables reasoning and carries a completion
* ceiling, the thinking budget is capped via OpenRouter's
* `reasoning.max_tokens` and the completion ceiling is raised by the
* same amount — so no reasoning effort (incl. xhigh) can consume the
* entire completion budget and emit an empty message (the 2026-06-11
* empty-draft incident, prod run acf24029). The cap is SETTINGS-backed
* (`prautoblogger_reasoning_max_tokens`, default
* PRAUTOBLOGGER_DEFAULT_REASONING_MAX_TOKENS, 0 = pure effort mode:
* no cap, no headroom — the historical behavior). When the cap is
* active it replaces `effort` — OpenRouter treats the two as
* alternative budget controls and normalizes max_tokens per model.
*
* No-op when reasoning is absent/disabled on the request, when the
* request has no completion ceiling, or when the cap setting is 0.
*
* @param array<string, mixed> $body Assembled request body.
*
* @return array<string, mixed> Body with the reasoning budget applied.
*/
private function apply_reasoning_budget( array $body ): array {
if ( ! isset( $body['reasoning'] ) || false === ( $body['reasoning']['enabled'] ?? true ) ) {
return $body;
}
if ( ! isset( $body['max_tokens'] ) ) {
return $body; // No completion ceiling to protect.
}
if ( ! isset( $body['reasoning']['max_tokens'] ) ) {
$cap = absint( get_option( 'prautoblogger_reasoning_max_tokens', PRAUTOBLOGGER_DEFAULT_REASONING_MAX_TOKENS ) );
if ( $cap < 1 ) {
return $body; // Cap disabled — pure effort mode.
}
$body['reasoning']['max_tokens'] = $cap;
unset( $body['reasoning']['effort'] );
}
// Headroom: models that spend reasoning from the completion budget
// (e.g. deepseek/deepseek-v4-flash) must still have the caller's
// full max_tokens available for visible content.
$body['max_tokens'] = (int) $body['max_tokens'] + (int) $body['reasoning']['max_tokens'];
return $body;
}
/**
* Build request headers for OpenRouter API call.
*

1
includes/services/opik/class-opik-eval-scorer.php
View File

@@ -82,6 +82,7 @@ class PRAutoBlogger_Opik_Eval_Scorer {
array(
'temperature' => 0.1,
'max_tokens' => 200,
'stage' => 'opik_eval_judge',
)
);

10
prautoblogger.php
View File

@@ -9,7 +9,7 @@
* Plugin Name: PRAutoBlogger
* Plugin URI: https://peptiderepo.com/prautoblogger
* Description: Monitors social media for trending topics, generates SEO-friendly blog posts using AI, and publishes them on a daily schedule with full cost tracking and self-improvement metrics.
* Version: 0.18.0
* Version: 0.18.1
* Requires at least: 6.0
* Requires PHP: 7.4
* Author: PeptideRepo
@@ -35,7 +35,7 @@ if ( ! defined( 'ABSPATH' ) ) {
| Defined here so every file in the plugin can reference paths, versions,
| and limits without magic strings.
*/
define( 'PRAUTOBLOGGER_VERSION', '0.18.0' );
define( 'PRAUTOBLOGGER_VERSION', '0.18.1' );
define( 'PRAUTOBLOGGER_DB_VERSION', '1.2.0' );
define( 'PRAUTOBLOGGER_PLUGIN_FILE', __FILE__ );
define( 'PRAUTOBLOGGER_PLUGIN_DIR', plugin_dir_path( __FILE__ ) );
@@ -49,6 +49,12 @@ define( 'PRAUTOBLOGGER_MAX_RETRIES', 3 );
// the named get_option() fallbacks — never inline the literals.
define( 'PRAUTOBLOGGER_DEFAULT_RUN_CEILING_USD', 0.50 );
define( 'PRAUTOBLOGGER_DEFAULT_REQUEST_JSON_RETENTION_DAYS', 14 );
// v0.18.1: ceiling on "thinking" tokens per reasoning-enabled LLM call. The
// request's completion budget (max_tokens) is RAISED by this amount so
// reasoning can never consume the visible-content budget and return an empty
// message (the 2026-06-11 empty-draft incident). SETTINGS-backed
// (prautoblogger_reasoning_max_tokens); 0 disables the cap (pure effort mode).
define( 'PRAUTOBLOGGER_DEFAULT_REASONING_MAX_TOKENS', 2048 );
define( 'PRAUTOBLOGGER_RETRY_BASE_DELAY_SECONDS', 2 );
define( 'PRAUTOBLOGGER_API_TIMEOUT_SECONDS', 180 );
define( 'PRAUTOBLOGGER_CACHE_TTL_SECONDS', 3600 );

3
tests/bootstrap.php
View File

@@ -28,6 +28,9 @@ if ( ! defined( 'PRAUTOBLOGGER_DEFAULT_RUN_CEILING_USD' ) ) {
if ( ! defined( 'PRAUTOBLOGGER_DEFAULT_REQUEST_JSON_RETENTION_DAYS' ) ) {
define( 'PRAUTOBLOGGER_DEFAULT_REQUEST_JSON_RETENTION_DAYS', 14 );
}
if ( ! defined( 'PRAUTOBLOGGER_DEFAULT_REASONING_MAX_TOKENS' ) ) {
define( 'PRAUTOBLOGGER_DEFAULT_REASONING_MAX_TOKENS', 2048 );
}
// WordPress time constants used by CostTracker and ContentAnalyzer.
if ( ! defined( 'DAY_IN_SECONDS' ) ) {

164
tests/unit/Core/ContentGeneratorTest.php Normal file
View File

@@ -0,0 +1,164 @@
<?php
/**
* Tests for PRAutoBlogger_Content_Generator (v0.18.1 writer-path guard).
*
* Validates that a writing stage whose LLM output is empty/whitespace
* throws instead of passing an empty article downstream, that healthy
* output passes through, and that the stage metadata for the provider's
* empty-completion guard rides the options array.
*
* execute_stage() is exercised via reflection: generate() defines the
* PRAUTOBLOGGER_EVAL_MODE constant, which cannot be redefined across
* tests in one PHP process.
*
* @package PRAutoBlogger\Tests\Core
*/
namespace PRAutoBlogger\Tests\Core;
use PRAutoBlogger\Tests\BaseTestCase;
use Brain\Monkey\Functions;
class ContentGeneratorTest extends BaseTestCase {
private \PRAutoBlogger_Content_Request $request;
protected function setUp(): void {
parent::setUp();
\PRAutoBlogger_Prompt_Registry::flush_cache();
$this->stub_get_option( [
'prautoblogger_log_level' => 'error',
] );
// Content_Prompts::build_system() lists recent posts for the
// linking rules — none exist in the unit environment.
Functions\when( 'get_posts' )->justReturn( [] );
$idea = new \PRAutoBlogger_Article_Idea( $this->get_article_idea_fixture() );
$this->request = new \PRAutoBlogger_Content_Request(
$idea,
'single_pass',
'informational',
800,
2000,
'',
[],
''
);
}
protected function tearDown(): void {
\PRAutoBlogger_Prompt_Registry::flush_cache();
parent::tearDown();
}
/**
* Invoke the private execute_stage() on a generator wired to the
* given LLM response.
*
* @param array $llm_response Response the mock provider returns.
* @param array|null $captured Receives the options passed to the LLM.
* @return string Stage output.
*/
private function run_stage( array $llm_response, ?array &$captured = null ): string {
$llm = $this->createMock( \PRAutoBlogger_LLM_Provider_Interface::class );
$llm->method( 'send_chat_completion' )->willReturnCallback(
function ( array $messages, string $model, array $options ) use ( $llm_response, &$captured ) {
$captured = $options;
return $llm_response;
}
);
$llm->method( 'get_provider_name' )->willReturn( 'OpenRouter' );
$llm->method( 'estimate_cost' )->willReturn( 0.001 );
$cost_tracker = $this->createMock( \PRAutoBlogger_Cost_Tracker::class );
$generator = new \PRAutoBlogger_Content_Generator( $llm, $cost_tracker );
$method = new \ReflectionMethod( \PRAutoBlogger_Content_Generator::class, 'execute_stage' );
$method->setAccessible( true );
return $method->invoke(
$generator,
'draft',
'draft_generation',
'Write the article.',
$this->request,
'model/test',
[
'temperature' => 0.7,
'max_tokens' => 4000,
],
'content.single_pass'
);
}
/**
* Healthy stage output passes through unchanged.
*/
public function test_non_empty_stage_output_passes_through(): void {
$output = $this->run_stage( [
'content' => '<p>A real article.</p>',
'model' => 'model/test',
'prompt_tokens' => 100,
'completion_tokens' => 200,
] );
$this->assertSame( '<p>A real article.</p>', $output );
}
/**
* Empty stage output throws — no empty article may leave the writer.
*/
public function test_empty_stage_output_throws(): void {
$this->expectException( \RuntimeException::class );
$this->expectExceptionMessage( 'produced empty content' );
$this->run_stage( [
'content' => '',
'model' => 'model/test',
'prompt_tokens' => 100,
'completion_tokens' => 4000,
] );
}
/**
* Whitespace-only output counts as empty (the guard trims).
*/
public function test_whitespace_stage_output_throws(): void {
$this->expectException( \RuntimeException::class );
$this->expectExceptionMessage( 'draft' );
$this->run_stage( [
'content' => " \n\t ",
'model' => 'model/test',
'prompt_tokens' => 100,
'completion_tokens' => 4000,
] );
}
/**
* The stage + prompt_key metadata for the provider's empty-completion
* guard rides the LLM options (v0.18.1).
*/
public function test_stage_metadata_rides_the_llm_options(): void {
$captured = null;
$this->run_stage(
[
'content' => 'ok',
'model' => 'model/test',
'prompt_tokens' => 1,
'completion_tokens' => 1,
],
$captured
);
$this->assertSame( 'draft', $captured['stage'] );
$this->assertSame( 'content.single_pass', $captured['prompt_key'] );
$this->assertSame( 4000, $captured['max_tokens'] );
}
}

35
tests/unit/Core/PublisherTest.php
View File

@@ -36,6 +36,10 @@ class PublisherTest extends BaseTestCase {
Functions\when( 'sanitize_text_field' )->returnArg();
Functions\when( 'wp_kses_post' )->returnArg();
// v0.18.1 empty-content guard strips tags before the emptiness check.
Functions\when( 'wp_strip_all_tags' )->alias( function ( $text ) {
return trim( strip_tags( (string) $text ) );
} );
Functions\when( 'is_wp_error' )->justReturn( false );
Functions\when( 'apply_filters' )->returnArg( 2 );
Functions\when( 'absint' )->alias( function ( $val ) { return abs( (int) $val ); } );
@@ -295,4 +299,35 @@ class PublisherTest extends BaseTestCase {
$publisher = new \PRAutoBlogger_Publisher();
$publisher->publish( '<p>Content</p>', $this->idea, $this->review );
}
/**
* v0.18.1 belt-and-braces: save_as_draft() with empty content throws
* and never reaches wp_insert_post — no empty draft post can be
* created (the 2026-06-11 post-921 incident shape).
*/
public function test_save_as_draft_refuses_empty_content(): void {
Functions\expect( 'wp_insert_post' )->never();
$publisher = new \PRAutoBlogger_Publisher();
$this->expectException( \RuntimeException::class );
$this->expectExceptionMessage( 'generated content is empty' );
$publisher->save_as_draft( '', $this->idea, $this->review, 'run_empty_draft' );
}
/**
* v0.18.1 belt-and-braces: publish() refuses content that is empty
* once tags and whitespace are stripped (an empty HTML shell).
*/
public function test_publish_refuses_whitespace_only_html_content(): void {
Functions\expect( 'wp_insert_post' )->never();
$publisher = new \PRAutoBlogger_Publisher();
$this->expectException( \RuntimeException::class );
$this->expectExceptionMessage( 'generated content is empty' );
$publisher->publish( "<p> \n\t </p>", $this->idea, $this->review );
}
}

240
tests/unit/Providers/OpenRouterCompletionGuardTest.php Normal file
View File

@@ -0,0 +1,240 @@
<?php
/**
* Tests for PRAutoBlogger_OpenRouter_Completion_Guard (v0.18.1).
*
* Validates the empty-completion verdict matrix: healthy pass-through,
* the single reasoning-disabled retry on length/reasoning empties, the
* never-retry-twice rule, and the hard failure for unexplained empties.
*
* @package PRAutoBlogger\Tests\Providers
*/
namespace PRAutoBlogger\Tests\Providers;
use PRAutoBlogger\Tests\BaseTestCase;
use Brain\Monkey\Functions;
class OpenRouterCompletionGuardTest extends BaseTestCase {
protected function setUp(): void {
parent::setUp();
\PRAutoBlogger_Run_Context::clear();
\PRAutoBlogger_Audit_Writer::flush_cache();
$this->stub_get_option( [
'prautoblogger_log_level' => 'error',
] );
// Cost_Tracker::log_api_call() (failed-attempt booking) inserts a
// generation_log row and prices the burn via the pricing chain.
$wpdb = $this->create_mock_wpdb();
$wpdb->method( 'insert' )->willReturn( 1 );
$wpdb->insert_id = 7;
$GLOBALS['wpdb'] = $wpdb;
}
protected function tearDown(): void {
unset( $GLOBALS['wpdb'] );
\PRAutoBlogger_Run_Context::clear();
\PRAutoBlogger_Audit_Writer::flush_cache();
parent::tearDown();
}
/**
* Build a parsed-response fixture in the Response_Parser shape.
*
* @param string $content Visible content.
* @param string $finish finish_reason value.
* @return array
*/
private function parsed( string $content, string $finish ): array {
return [
'content' => $content,
'model' => 'deepseek/deepseek-v4-flash',
'prompt_tokens' => 1200,
'completion_tokens' => 4000,
'reasoning_tokens' => 4000,
'total_tokens' => 5200,
'finish_reason' => $finish,
'reasoning_content' => 'thinking…',
];
}
/**
* Healthy completion passes through untouched; the retry callable is
* never invoked.
*/
public function test_healthy_completion_passes_through(): void {
$parsed = $this->parsed( 'A full article.', 'stop' );
$invoked = 0;
$result = ( new \PRAutoBlogger_OpenRouter_Completion_Guard() )->finalize(
$parsed,
[ 'stage' => 'draft' ],
false,
function () use ( &$invoked ) {
++$invoked;
return [];
}
);
$this->assertSame( $parsed, $result );
$this->assertSame( 0, $invoked );
}
/**
* The incident shape — empty content + finish_reason=length — retries
* exactly once with reasoning disabled and the empty_retry marker set,
* and returns the retry's result.
*/
public function test_empty_plus_length_retries_once_with_reasoning_disabled(): void {
$recovered = $this->parsed( 'Recovered article.', 'stop' );
$captured_opts = null;
$result = ( new \PRAutoBlogger_OpenRouter_Completion_Guard() )->finalize(
$this->parsed( '', 'length' ),
[ 'stage' => 'draft', 'prompt_key' => 'content.single_pass' ],
true,
function ( array $retry_options ) use ( &$captured_opts, $recovered ) {
$captured_opts = $retry_options;
return $recovered;
}
);
$this->assertSame( $recovered, $result );
$this->assertSame( [ 'enabled' => false ], $captured_opts['reasoning'] );
$this->assertTrue( $captured_opts['empty_retry'] );
$this->assertSame( 'draft', $captured_opts['stage'] );
}
/**
* Whitespace-only content counts as empty (the guard trims).
*/
public function test_whitespace_only_content_counts_as_empty(): void {
$invoked = 0;
( new \PRAutoBlogger_OpenRouter_Completion_Guard() )->finalize(
$this->parsed( " \n\t ", 'length' ),
[ 'stage' => 'polish' ],
true,
function ( array $retry_options ) use ( &$invoked ) {
++$invoked;
return $this->parsed( 'ok', 'stop' );
}
);
$this->assertSame( 1, $invoked );
}
/**
* An attempt already marked empty_retry is never retried again — the
* guard fails the call instead (no infinite retry loops).
*/
public function test_already_retried_empty_fails_without_second_retry(): void {
$invoked = 0;
try {
( new \PRAutoBlogger_OpenRouter_Completion_Guard() )->finalize(
$this->parsed( '', 'length' ),
[ 'stage' => 'draft', 'empty_retry' => true ],
false,
function () use ( &$invoked ) {
++$invoked;
return [];
}
);
$this->fail( 'Expected RuntimeException for retried-and-still-empty completion.' );
} catch ( \RuntimeException $e ) {
$this->assertStringContainsString( 'empty content', $e->getMessage() );
$this->assertStringContainsString( 'after a reasoning-disabled retry', $e->getMessage() );
}
$this->assertSame( 0, $invoked );
}
/**
* Empty content with finish_reason=stop and no reasoning active: a
* reasoning-disabled retry cannot plausibly help — fail immediately.
*/
public function test_unexplained_empty_fails_without_retry(): void {
$invoked = 0;
try {
( new \PRAutoBlogger_OpenRouter_Completion_Guard() )->finalize(
$this->parsed( '', 'stop' ),
[ 'stage' => 'review' ],
false,
function () use ( &$invoked ) {
++$invoked;
return [];
}
);
$this->fail( 'Expected RuntimeException for unexplained empty completion.' );
} catch ( \RuntimeException $e ) {
$this->assertStringContainsString( 'stage "review"', $e->getMessage() );
$this->assertStringContainsString( 'finish_reason=stop', $e->getMessage() );
}
$this->assertSame( 0, $invoked );
}
/**
* Empty content with reasoning active retries even when finish_reason
* is not 'length' (reasoning can eat the budget without a length flag).
*/
public function test_empty_with_reasoning_active_retries_on_any_finish_reason(): void {
$invoked = 0;
( new \PRAutoBlogger_OpenRouter_Completion_Guard() )->finalize(
$this->parsed( '', 'stop' ),
[ 'stage' => 'draft' ],
true,
function ( array $retry_options ) use ( &$invoked ) {
++$invoked;
return $this->parsed( 'ok', 'stop' );
}
);
$this->assertSame( 1, $invoked );
}
/**
* The failed attempt is booked to generation_log with its REAL status:
* response_status='error', the stage, the model, and the token burn.
*/
public function test_failed_attempt_books_generation_log_row_with_error_status(): void {
$captured_row = null;
$wpdb = $this->create_mock_wpdb();
$wpdb->method( 'insert' )->willReturnCallback(
function ( $table, $row ) use ( &$captured_row ) {
if ( false !== strpos( (string) $table, 'generation_log' ) ) {
$captured_row = $row;
}
return 1;
}
);
$wpdb->insert_id = 9;
$GLOBALS['wpdb'] = $wpdb;
try {
( new \PRAutoBlogger_OpenRouter_Completion_Guard() )->finalize(
$this->parsed( '', 'stop' ),
[ 'stage' => 'draft' ],
false,
function () {
return [];
}
);
$this->fail( 'Expected RuntimeException.' );
} catch ( \RuntimeException $e ) {
$this->assertIsArray( $captured_row );
$this->assertSame( 'error', $captured_row['response_status'] );
$this->assertSame( 'draft', $captured_row['stage'] );
$this->assertSame( 'deepseek/deepseek-v4-flash', $captured_row['model'] );
$this->assertSame( 4000, $captured_row['completion_tokens'] );
$this->assertStringContainsString( 'Empty completion', $captured_row['error_message'] );
}
}
}

149
tests/unit/Providers/OpenRouterProviderTest.php
View File

@@ -166,4 +166,153 @@ class OpenRouterProviderTest extends BaseTestCase {
$this->expectException( \RuntimeException::class );
$provider->send_chat_completion( $messages, 'model/test', $options );
}
/**
* Wire the provider for a full send_chat_completion run: decryptable
* API key, reasoning settings, HTTP stubs that replay scripted
* responses, and a wpdb mock for the guard's failure bookkeeping.
*
* @param array $option_overrides Extra get_option values.
* @param array $responses Scripted JSON-decodable response bodies.
* @param array $requests (by ref) captures decoded request bodies.
*/
private function wire_full_provider( array $option_overrides, array $responses, array &$requests ): void {
\PRAutoBlogger_Run_Context::clear();
$encrypted_key = \PRAutoBlogger_Encryption::encrypt( 'sk-or-v1-unit-test-key-0000' );
$this->stub_get_option( array_merge( [
'prautoblogger_openrouter_api_key' => $encrypted_key,
'prautoblogger_log_level' => 'error',
], $option_overrides ) );
Functions\when( 'absint' )->alias( function ( $val ) {
return abs( (int) $val );
} );
Functions\when( 'wp_remote_post' )->alias(
function ( $url, $args ) use ( &$requests, $responses ) {
$requests[] = json_decode( $args['body'], true );
$body = $responses[ count( $requests ) - 1 ];
return [
'body' => $body,
'response' => [ 'code' => 200 ],
];
}
);
Functions\when( 'wp_remote_retrieve_response_code' )->alias(
function ( $response ) {
return $response['response']['code'];
}
);
Functions\when( 'wp_remote_retrieve_body' )->alias(
function ( $response ) {
return $response['body'];
}
);
// The guard books failed attempts through Cost_Tracker + Logger.
$wpdb = $this->create_mock_wpdb();
$wpdb->method( 'insert' )->willReturn( 1 );
$wpdb->insert_id = 11;
$GLOBALS['wpdb'] = $wpdb;
}
/**
* v0.18.1 incident shape, end to end: reasoning enabled (uncapped
* effort mode), the first response burns the whole completion budget
* on thinking and returns empty content with finish_reason=length —
* the provider retries exactly once with reasoning disabled and
* returns the recovered completion.
*/
public function test_empty_length_completion_retries_once_with_reasoning_disabled(): void {
$requests = [];
$empty_response = json_encode( [
'model' => 'deepseek/deepseek-v4-flash',
'choices' => [ [ 'message' => [ 'content' => '', 'reasoning' => 'endless thinking…' ], 'finish_reason' => 'length' ] ],
'usage' => [ 'prompt_tokens' => 1200, 'completion_tokens' => 4000, 'reasoning_tokens' => 4000, 'total_tokens' => 5200 ],
] );
$good_response = json_encode( [
'model' => 'deepseek/deepseek-v4-flash',
'choices' => [ [ 'message' => [ 'content' => 'Recovered article.' ], 'finish_reason' => 'stop' ] ],
'usage' => [ 'prompt_tokens' => 1200, 'completion_tokens' => 900, 'total_tokens' => 2100 ],
] );
$this->wire_full_provider(
[
'prautoblogger_reasoning_enabled' => '1',
'prautoblogger_reasoning_effort' => 'xhigh',
'prautoblogger_reasoning_max_tokens' => 0,
],
[ $empty_response, $good_response ],
$requests
);
try {
$provider = new \PRAutoBlogger_OpenRouter_Provider();
$result = $provider->send_chat_completion(
[ [ 'role' => 'user', 'content' => 'Write the article.' ] ],
'deepseek/deepseek-v4-flash',
[ 'temperature' => 0.7, 'max_tokens' => 4000, 'stage' => 'draft' ]
);
$this->assertSame( 'Recovered article.', $result['content'] );
$this->assertCount( 2, $requests );
// First attempt: global reasoning, uncapped effort mode.
$this->assertSame( [ 'enabled' => true, 'effort' => 'xhigh' ], $requests[0]['reasoning'] );
$this->assertSame( 4000, $requests[0]['max_tokens'] );
// Retry: reasoning disabled per call.
$this->assertSame( [ 'enabled' => false ], $requests[1]['reasoning'] );
// Caller metadata never reaches the HTTP body.
$this->assertArrayNotHasKey( 'stage', $requests[0] );
$this->assertArrayNotHasKey( 'empty_retry', $requests[1] );
} finally {
unset( $GLOBALS['wpdb'] );
\PRAutoBlogger_Run_Context::clear();
}
}
/**
* v0.18.1 reasoning budget: with the token cap active (default), the
* outgoing request caps thinking via reasoning.max_tokens and raises
* max_tokens by the cap — and a healthy completion needs no retry.
*/
public function test_reasoning_cap_shapes_the_outgoing_request(): void {
$requests = [];
$good_response = json_encode( [
'model' => 'deepseek/deepseek-v4-flash',
'choices' => [ [ 'message' => [ 'content' => 'Article text.' ], 'finish_reason' => 'stop' ] ],
'usage' => [ 'prompt_tokens' => 1200, 'completion_tokens' => 900, 'total_tokens' => 2100 ],
] );
$this->wire_full_provider(
[
'prautoblogger_reasoning_enabled' => '1',
'prautoblogger_reasoning_effort' => 'xhigh',
'prautoblogger_reasoning_max_tokens' => 2048,
],
[ $good_response ],
$requests
);
try {
$provider = new \PRAutoBlogger_OpenRouter_Provider();
$result = $provider->send_chat_completion(
[ [ 'role' => 'user', 'content' => 'Write the article.' ] ],
'deepseek/deepseek-v4-flash',
[ 'temperature' => 0.7, 'max_tokens' => 4000, 'stage' => 'draft' ]
);
$this->assertSame( 'Article text.', $result['content'] );
$this->assertCount( 1, $requests );
$this->assertSame( 2048, $requests[0]['reasoning']['max_tokens'] );
$this->assertArrayNotHasKey( 'effort', $requests[0]['reasoning'] );
$this->assertSame( 6048, $requests[0]['max_tokens'] );
} finally {
unset( $GLOBALS['wpdb'] );
\PRAutoBlogger_Run_Context::clear();
}
}
}

189
tests/unit/Providers/OpenRouterRequestBuilderTest.php Normal file
View File

@@ -0,0 +1,189 @@
<?php
/**
* Tests for PRAutoBlogger_OpenRouter_Request_Builder::build_body() (v0.18.1).
*
* Validates body assembly, the global-vs-per-call reasoning precedence, the
* reasoning token cap + completion headroom, and that caller-metadata keys
* never leak into the HTTP body.
*
* @package PRAutoBlogger\Tests\Providers
*/
namespace PRAutoBlogger\Tests\Providers;
use PRAutoBlogger\Tests\BaseTestCase;
use Brain\Monkey\Functions;
class OpenRouterRequestBuilderTest extends BaseTestCase {
private const MESSAGES = [
[ 'role' => 'system', 'content' => 'sys' ],
[ 'role' => 'user', 'content' => 'usr' ],
];
protected function setUp(): void {
parent::setUp();
Functions\when( 'absint' )->alias( function ( $val ) {
return abs( (int) $val );
} );
}
/**
* Reasoning enabled globally + cap set: thinking is capped via
* reasoning.max_tokens, effort is dropped, and the completion budget
* is raised by the cap so content keeps its full allowance.
*/
public function test_reasoning_cap_replaces_effort_and_raises_headroom(): void {
$this->stub_get_option( [
'prautoblogger_reasoning_enabled' => '1',
'prautoblogger_reasoning_effort' => 'xhigh',
'prautoblogger_reasoning_max_tokens' => 2048,
] );
$body = ( new \PRAutoBlogger_OpenRouter_Request_Builder() )
->build_body( self::MESSAGES, 'deepseek/deepseek-v4-flash', [ 'max_tokens' => 4000 ] );
$this->assertSame( 2048, $body['reasoning']['max_tokens'] );
$this->assertArrayNotHasKey( 'effort', $body['reasoning'] );
$this->assertSame( 6048, $body['max_tokens'] );
$this->assertTrue( $body['reasoning']['enabled'] );
}
/**
* Cap setting 0 = pure effort mode: the historical body shape is
* preserved exactly (effort kept, no reasoning.max_tokens, no headroom).
*/
public function test_cap_zero_keeps_pure_effort_mode(): void {
$this->stub_get_option( [
'prautoblogger_reasoning_enabled' => '1',
'prautoblogger_reasoning_effort' => 'xhigh',
'prautoblogger_reasoning_max_tokens' => 0,
] );
$body = ( new \PRAutoBlogger_OpenRouter_Request_Builder() )
->build_body( self::MESSAGES, 'deepseek/deepseek-v4-flash', [ 'max_tokens' => 4000 ] );
$this->assertSame( [ 'enabled' => true, 'effort' => 'xhigh' ], $body['reasoning'] );
$this->assertSame( 4000, $body['max_tokens'] );
}
/**
* Per-call reasoning override (the guard's retry) beats the global
* setting; a disabled override gets no cap and no headroom.
*/
public function test_per_call_disabled_override_beats_global_setting(): void {
$this->stub_get_option( [
'prautoblogger_reasoning_enabled' => '1',
'prautoblogger_reasoning_effort' => 'xhigh',
'prautoblogger_reasoning_max_tokens' => 2048,
] );
$body = ( new \PRAutoBlogger_OpenRouter_Request_Builder() )
->build_body(
self::MESSAGES,
'deepseek/deepseek-v4-flash',
[
'max_tokens' => 4000,
'reasoning' => [ 'enabled' => false ],
]
);
$this->assertSame( [ 'enabled' => false ], $body['reasoning'] );
$this->assertSame( 4000, $body['max_tokens'] );
}
/**
* Global reasoning off + no override: no reasoning key at all and no
* budget math (historical default path).
*/
public function test_reasoning_off_produces_no_reasoning_key(): void {
$this->stub_get_option( [
'prautoblogger_reasoning_enabled' => '0',
] );
$body = ( new \PRAutoBlogger_OpenRouter_Request_Builder() )
->build_body(
self::MESSAGES,
'google/gemini-2.5-flash-lite',
[
'temperature' => 0.7,
'max_tokens' => 4000,
'response_format' => [ 'type' => 'json_object' ],
]
);
$this->assertArrayNotHasKey( 'reasoning', $body );
$this->assertSame( 0.7, $body['temperature'] );
$this->assertSame( 4000, $body['max_tokens'] );
$this->assertSame( [ 'type' => 'json_object' ], $body['response_format'] );
}
/**
* An explicit caller reasoning config with its own max_tokens is
* respected (no cap overwrite) but still gets completion headroom.
*/
public function test_caller_reasoning_max_tokens_respected_with_headroom(): void {
$this->stub_get_option( [
'prautoblogger_reasoning_enabled' => '0',
'prautoblogger_reasoning_max_tokens' => 2048,
] );
$body = ( new \PRAutoBlogger_OpenRouter_Request_Builder() )
->build_body(
self::MESSAGES,
'deepseek/deepseek-v4-flash',
[
'max_tokens' => 1000,
'reasoning' => [ 'enabled' => true, 'max_tokens' => 512 ],
]
);
$this->assertSame( 512, $body['reasoning']['max_tokens'] );
$this->assertSame( 1512, $body['max_tokens'] );
}
/**
* Caller-metadata keys (stage, prompt_key, empty_retry) must never
* appear in the HTTP body.
*/
public function test_caller_metadata_keys_never_leak_into_body(): void {
$this->stub_get_option( [
'prautoblogger_reasoning_enabled' => '0',
] );
$body = ( new \PRAutoBlogger_OpenRouter_Request_Builder() )
->build_body(
self::MESSAGES,
'google/gemini-2.5-flash-lite',
[
'max_tokens' => 4000,
'stage' => 'draft',
'prompt_key' => 'content.single_pass',
'empty_retry' => true,
]
);
$this->assertArrayNotHasKey( 'stage', $body );
$this->assertArrayNotHasKey( 'prompt_key', $body );
$this->assertArrayNotHasKey( 'empty_retry', $body );
$this->assertSame( [ 'model', 'messages', 'max_tokens' ], array_keys( $body ) );
}
/**
* Reasoning enabled but the call has no completion ceiling: nothing to
* protect, body passes through with effort untouched.
*/
public function test_no_max_tokens_means_no_budget_math(): void {
$this->stub_get_option( [
'prautoblogger_reasoning_enabled' => '1',
'prautoblogger_reasoning_effort' => 'high',
'prautoblogger_reasoning_max_tokens' => 2048,
] );
$body = ( new \PRAutoBlogger_OpenRouter_Request_Builder() )
->build_body( self::MESSAGES, 'deepseek/deepseek-v4-flash', [] );
$this->assertSame( [ 'enabled' => true, 'effort' => 'high' ], $body['reasoning'] );
$this->assertArrayNotHasKey( 'max_tokens', $body );
}
}