peptide-repo/prautoblogger
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: empty-content guard + finish_reason=length handling + reasoning budget (v0.18.1)

Browse Source 
Reasoning models at high effort could spend the entire max_tokens budget
on thinking and return zero visible content with finish_reason=length;
the empty article flowed writer -> editor (reject) -> empty draft post,
with the cost booked as success (2026-06-11 incident: prod run acf24029,
draft post 921; diagnosis: convo pipeline-v2-phase1 seq 12).

- Empty-completion guard at the provider seam (new
  OpenRouter_Completion_Guard): empty/whitespace content is a failure;
  one retry with reasoning disabled (per-call override) when
  finish_reason=length or reasoning was active, then the stage fails.
- finish_reason=length always warning-logged (model, stage, tokens).
- Failed/retried attempts booked to generation_log with
  response_status=error + real token burn; run_decisions verdict rows
  (retry_reasoning_disabled / failed_empty_completion) for v0.18.0 runs.
- Writer-path guard in Content_Generator::execute_stage and
  belt-and-braces empty check in Publisher::create_post.
- Reasoning budget sanity: reasoning.max_tokens cap + completion
  headroom via new prautoblogger_reasoning_max_tokens setting (default
  constant 2048, 0 = legacy uncapped effort mode); cap replaces effort
  when active; governor reservations use the effective ceiling.
- Body assembly moved to Request_Builder::build_body (300-line cap).
- Tests: guard verdict matrix, builder budget shaping, writer/publisher
  guards, full provider retry path.

Agent-Session: prab-empty-draft-guard-20260611
This commit is contained in:
peptiderepo
2026-06-11 13:15:58 +00:00
parent f5658f1c97
commit 2bdc763dc0
19 changed files with 1196 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files

47
CHANGELOG.md
View File

@@ -5,6 +5,53 @@ All notable changes to PRAutoBlogger will be documented in this file.
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project uses [Semantic Versioning](https://semver.org/).
## [0.18.1] - 2026-06-11
Empty-draft hotfix (CTO diagnosis: convo thread `2026-06-pipeline-v2-phase1` seq 12; prod run
`acf24029`, draft post 921). A true reasoning model at a high reasoning effort could spend the
entire `max_tokens` completion budget on thinking, return zero visible content with
`finish_reason=length`, and the empty article flowed writer → editor (reject: "draft text is
missing") → empty draft post in the Review Queue, with the cost booked as `success`.
### Fixed
- **Empty-completion guard at the call seam** (`PRAutoBlogger_OpenRouter_Completion_Guard`):
a chat completion whose visible content is empty/whitespace is now a **failure**, never a
success. When the empty result co-occurs with `finish_reason=length` or an active reasoning
request, the call is retried **once** with reasoning disabled (per-call `reasoning`
override) before the stage fails. Failed/retried attempts land in `generation_log` with
`response_status='error'`, the failure note, and the real token burn; inside a v0.18.0 run a
`run_decisions` row records the verdict (`retry_reasoning_disabled` /
`failed_empty_completion`).
- **Writer-path guard**: a writing stage (outline/draft/polish/single-pass) whose output is
empty throws instead of passing an empty article to the editor (belt over the seam guard,
covers any future non-OpenRouter provider).
- **Publisher belt-and-braces**: `Publisher::create_post()` refuses to create any post —
publish or draft — whose content is empty after tag stripping. No empty draft post can ever
be created again.
- `finish_reason=length` is now always warning-logged with model, stage, and
prompt/completion/reasoning token counts (previously silent — the truncated `llm_research`
call in the incident run was invisible).
### Added
- **Reasoning token cap + completion headroom**: when a request enables reasoning, the
thinking budget is capped via OpenRouter's `reasoning.max_tokens` and the request's
`max_tokens` is **raised** by the same amount, so no reasoning effort (incl. `xhigh`) can
consume the visible-content budget. New setting `prautoblogger_reasoning_max_tokens`
(Settings → AI Models, default `PRAUTOBLOGGER_DEFAULT_REASONING_MAX_TOKENS` = 2048; 0
restores the previous uncapped effort mode). When the cap is active it replaces `effort`
(OpenRouter treats them as alternative budget controls); the `xhigh`/`high` effort options
stay available but are bounded by default. Cost-governor reservations use the effective
(raised) ceiling.
- LLM call sites now pass their pipeline stage to the provider (`stage` option, never sent
upstream) so guard warnings and failure rows are attributable: `analysis`, `review`,
`llm_research`, `image_prompt_rewrite`, `opik_eval_judge`, and the writer stages (which
also pass `prompt_key`).
### Notes
- `OpenRouter_Provider::send_chat_completion()` body assembly moved to
`OpenRouter_Request_Builder::build_body()` (300-line cap; behavior unchanged apart from the
reasoning budget above).
## [0.18.0] - 2026-06-12
Pipeline v2 — Phase 1 substrate (plan of record: CPO thread

12
includes/admin/class-settings-fields-extended.php
View File

@@ -78,7 +78,17 @@ class PRAutoBlogger_Settings_Fields_Extended {
'low' => __( 'Low — light reasoning, lower cost', 'prautoblogger' ),
'minimal' => __( 'Minimal — barely any reasoning', 'prautoblogger' ),
),
'description' => __( 'How much effort the model spends reasoning before answering. Higher effort = more reasoning tokens = higher cost. Only applies when reasoning is enabled above.', 'prautoblogger' ),
'description' => __( 'How much effort the model spends reasoning before answering. Higher effort = more reasoning tokens = higher cost. Only applies when reasoning is enabled above. When the Reasoning Token Cap below is set (default), the cap is sent instead of the effort level — set the cap to 0 to use effort only.', 'prautoblogger' ),
),
array(
'id' => 'prautoblogger_reasoning_max_tokens',
'label' => __( 'Reasoning Token Cap', 'prautoblogger' ),
'type' => 'number',
'section' => 'prautoblogger_models',
'default' => PRAUTOBLOGGER_DEFAULT_REASONING_MAX_TOKENS,
'min' => 0,
'max' => 32000,
'description' => __( 'Hard ceiling on thinking tokens per call when reasoning is enabled. The completion budget is raised by this amount so reasoning can never crowd out the article text — an uncapped high effort can spend the entire budget thinking and return an empty draft. 0 = no cap (uncapped effort mode, not recommended).', 'prautoblogger' ),
),
// ── Schedule & Budget ───────────────────────────────────────

1
includes/core/class-chief-editor.php
View File

@@ -64,6 +64,7 @@ class PRAutoBlogger_Chief_Editor {
'temperature' => 0.3,
'max_tokens' => 5000,
'response_format' => array( 'type' => 'json_object' ),
'stage' => 'review',
)
);

1
includes/core/class-content-analyzer.php
View File

@@ -78,6 +78,7 @@ class PRAutoBlogger_Content_Analyzer {
'temperature' => 0.5,
'max_tokens' => 8000,
'response_format' => array( 'type' => 'json_object' ),
'stage' => 'analysis',
)
);

21
includes/core/class-content-generator.php
View File

@@ -197,6 +197,12 @@ class PRAutoBlogger_Content_Generator {
PRAutoBlogger_Run_Stage_State::start( $this->run_id, $stage, '', $this->item_key );
}
// v0.18.1: stage metadata for the provider's empty-completion
// guard — warning logs and failed-attempt audit rows carry the
// real stage + registry key (never sent upstream).
$options['stage'] = $stage;
$options['prompt_key'] = $prompt_key;
$ctx = PRAutoBlogger_Opik_Trace_Context::current();
$span_id = $ctx->start_span(
@@ -223,6 +229,21 @@ class PRAutoBlogger_Content_Generator {
$options
);
// v0.18.1 writer-path guard (belt over the provider seam, which
// already throws for OpenRouter): no empty article may ever leave
// a writing stage — it would flow to the editor, be rejected as
// "draft text is missing", and land as an empty draft post.
if ( '' === trim( (string) $response['content'] ) ) {
throw new \RuntimeException(
sprintf(
/* translators: 1: writing stage name, 2: model identifier. */
__( 'Writing stage "%1$s" produced empty content (model %2$s) — failing the stage instead of passing an empty article downstream.', 'prautoblogger' ),
$stage,
(string) $response['model']
)
);
}
$ctx->end_span(
$span_id,
array(

1
includes/core/class-image-prompt-builder.php
View File

@@ -200,6 +200,7 @@ PROMPT;
array(
'temperature' => 0.7,
'max_tokens' => self::REWRITER_MAX_TOKENS,
'stage' => 'image_prompt_rewrite',
)
);

17
includes/core/class-publisher.php
View File

@@ -82,6 +82,23 @@ class PRAutoBlogger_Publisher {
?string $run_id = null,
?PRAutoBlogger_Cost_Tracker $cost_tracker = null
): int {
// v0.18.1 belt-and-braces: never create a post whose content is
// empty once tags and whitespace are stripped — an empty "draft
// for human review" (2026-06-11 incident, post 921) reviews
// nothing and hides the underlying generation failure. The
// provider/writer guards make this unreachable on OpenRouter
// paths; this protects every other path into post creation.
if ( '' === trim( wp_strip_all_tags( $content ) ) ) {
throw new \RuntimeException(
sprintf(
/* translators: 1: intended post status, 2: article title. */
__( 'Refusing to create %1$s post "%2$s": generated content is empty.', 'prautoblogger' ),
$post_status,
$idea->get_suggested_title()
)
);
}
// v0.18.0 idempotency: post creation is keyed by run + idea
// (check-before-insert) so a retried/resumed run cannot create a
// duplicate post. One run_id spans all N articles of a batch, so

1
includes/providers/class-llm-research-provider.php
View File

@@ -70,6 +70,7 @@ class PRAutoBlogger_LLM_Research_Provider implements PRAutoBlogger_Source_Provid
'temperature' => 0.7,
'max_tokens' => 8000,
'response_format' => array( 'type' => 'json_object' ),
'stage' => 'llm_research',
)
);

168
includes/providers/class-open-router-completion-guard.php Normal file
View File

@@ -0,0 +1,168 @@
<?php
declare(strict_types=1);
/**
* phpcs:ignore WordPress.Files.FileName.InvalidClassFileName -- class naming convention differs from WordPress standard
*
* Empty-completion guard for OpenRouter chat responses (v0.18.1).
*
* What: Treats a chat completion whose visible content is empty/whitespace
* as a FAILURE, never a success — per the LLM-call discipline:
* schema-validate every output, retry/fall back, never silently
* pass. The incident this prevents (2026-06-11, prod run acf24029,
* draft post 921): a true reasoning model at a high reasoning effort
* spent the entire max_tokens completion budget on thinking,
* finished with finish_reason='length', and emitted zero visible
* content — which flowed writer → editor (reject: "draft text is
* missing") → an empty draft post in the Review Queue, with the
* cost booked as 'success'.
*
* Behavior on every parsed response:
* - finish_reason='length' is warning-logged (model, stage, prompt/
* completion/reasoning tokens) whether or not content survived.
* - Empty content + (finish_reason='length' OR reasoning active on
* the request) → ONE retry with reasoning disabled for that call
* (per-call `reasoning` override), then fail the stage.
* - Every failed/retried attempt lands in generation_log with its
* REAL response_status ('error') and token burn, and — inside a
* v0.18.0 run — a run_decisions row records the verdict.
* Who triggers it: PRAutoBlogger_OpenRouter_Provider::send_chat_completion()
* on every successfully parsed response.
* Dependencies: Logger, Cost_Tracker (failure rows), Run_Context (run
* attribution), Audit_Writer (run_decisions verdicts).
*
* @see class-open-router-provider.php — Integration point; supplies the retry callable.
* @see class-open-router-response-parser.php — Produces the parsed array assessed here.
* @see class-open-router-request-builder.php — Reasoning token cap (first line of defense).
* @see core/class-audit-writer.php — run_decisions verdict rows.
*/
class PRAutoBlogger_OpenRouter_Completion_Guard {
/** Verdict recorded when an empty attempt is retried with reasoning disabled. */
public const VERDICT_RETRY = 'retry_reasoning_disabled';
/** Verdict recorded when a call finally fails for empty content. */
public const VERDICT_FAIL = 'failed_empty_completion';
/**
* Assess a parsed completion: pass it through, retry once, or fail loudly.
*
* @param array<string, mixed> $parsed Parsed response from Response_Parser::parse_success().
* @param array<string, mixed> $options Original call options. Reads the caller-metadata keys
* 'stage' / 'prompt_key' (audit attribution) and
* 'empty_retry' (marks the retry attempt — never retried twice).
* @param bool $reasoning_active Whether the dispatched request had reasoning enabled.
* @param callable $retry Re-dispatches the call with the options it is given.
* Invoked at most once, with reasoning disabled.
*
* @return array<string, mixed> A completion whose visible content is non-empty.
*
* @throws \RuntimeException When content is empty and the retry budget is spent or a retry
* cannot plausibly help.
*/
public function finalize( array $parsed, array $options, bool $reasoning_active, callable $retry ): array {
$stage = (string) ( $options['stage'] ?? 'unknown' );
$finish_reason = (string) ( $parsed['finish_reason'] ?? 'unknown' );
if ( 'length' === $finish_reason ) {
PRAutoBlogger_Logger::instance()->warning(
sprintf(
'Completion hit its max_tokens ceiling (finish_reason=length): model=%s, stage=%s, prompt_tokens=%d, completion_tokens=%d, reasoning_tokens=%d.',
(string) $parsed['model'],
$stage,
(int) $parsed['prompt_tokens'],
(int) $parsed['completion_tokens'],
(int) $parsed['reasoning_tokens']
),
'openrouter'
);
}
if ( '' !== trim( (string) ( $parsed['content'] ?? '' ) ) ) {
return $parsed; // Healthy completion — visible content present.
}
$already_retried = ! empty( $options['empty_retry'] );
// A reasoning-disabled retry can plausibly help when the budget was
// exhausted (length) or when reasoning was eating the budget.
$retry_can_help = 'length' === $finish_reason || $reasoning_active;
if ( $retry_can_help && ! $already_retried ) {
$note = sprintf(
'Empty completion (finish_reason=%s, completion_tokens=%d, reasoning_tokens=%d) — retrying once with reasoning disabled.',
$finish_reason,
(int) $parsed['completion_tokens'],
(int) $parsed['reasoning_tokens']
);
$this->record_attempt( $parsed, $options, $stage, self::VERDICT_RETRY, $note );
PRAutoBlogger_Logger::instance()->warning(
sprintf( 'Stage %s: %s (model %s)', $stage, $note, (string) $parsed['model'] ),
'openrouter'
);
$retry_options = $options;
$retry_options['reasoning'] = array( 'enabled' => false );
$retry_options['empty_retry'] = true;
return $retry( $retry_options );
}
$note = sprintf(
'Empty completion (finish_reason=%s, completion_tokens=%d, reasoning_tokens=%d)%s — failing the call.',
$finish_reason,
(int) $parsed['completion_tokens'],
(int) $parsed['reasoning_tokens'],
$already_retried ? ' after a reasoning-disabled retry' : ''
);
$this->record_attempt( $parsed, $options, $stage, self::VERDICT_FAIL, $note );
throw new \RuntimeException(
sprintf(
/* translators: 1: pipeline stage, 2: model identifier, 3: finish_reason value, 4: retry note. */
__( 'LLM call for stage "%1$s" returned empty content (model %2$s, finish_reason=%3$s)%4$s.', 'prautoblogger' ),
$stage,
(string) $parsed['model'],
$finish_reason,
$already_retried ? __( ' after a reasoning-disabled retry', 'prautoblogger' ) : ''
)
);
}
/**
* Book a failed/retried attempt: a generation_log row with the REAL
* response_status and token burn (the empty call still cost money),
* plus a run_decisions verdict for new-style runs. Both writes are
* self-healing no-ops on a half-migrated schema; neither can throw.
*
* @param array<string, mixed> $parsed Parsed response of the failed attempt.
* @param array<string, mixed> $options Call options ('prompt_key' attribution).
* @param string $stage Stage label for the log row.
* @param string $verdict VERDICT_RETRY or VERDICT_FAIL.
* @param string $note Human-readable failure note.
* @return void
*/
private function record_attempt( array $parsed, array $options, string $stage, string $verdict, string $note ): void {
$run_id = PRAutoBlogger_Run_Context::current_run_id();
$tracker = new PRAutoBlogger_Cost_Tracker();
if ( null !== $run_id && '' !== $run_id ) {
// Attribute the row to the surrounding run. Idempotent: the
// run context, ledger row, and prompt pins already exist.
$tracker->set_run_id( $run_id );
}
$tracker->log_api_call(
null,
$stage,
'OpenRouter',
(string) $parsed['model'],
(int) $parsed['prompt_tokens'],
(int) $parsed['completion_tokens'],
'error',
$note,
null,
isset( $options['prompt_key'] ) ? (string) $options['prompt_key'] : null
);
// run_decisions verdict — no-op outside a run or pre-1.2.0 schema.
PRAutoBlogger_Audit_Writer::record_decision( (string) ( $run_id ?? '' ), $stage, $verdict, $note );
}
}

76
includes/providers/class-open-router-provider.php
View File

@@ -19,8 +19,9 @@ declare(strict_types=1);
* @see class-open-router-config.php — Config helpers (base URL, cache TTL).
* @see class-open-router-pricing.php — Pricing and model list lookups.
* @see class-open-router-validator.php — Credential validation (delegated).
* @see class-open-router-request-builder.php — Request header building + cURL filter.
* @see class-open-router-request-builder.php — Request body/header building + cURL filter.
* @see class-open-router-response-parser.php — Response parsing + error classification.
* @see class-open-router-completion-guard.php — Empty-completion guard + reasoning-off retry (v0.18.1).
* @see class-cost-tracker.php — Called after every request to log token usage.
* @see ARCHITECTURE.md — Data flow diagram showing where this fits.
*/
@@ -40,8 +41,15 @@ class PRAutoBlogger_OpenRouter_Provider implements PRAutoBlogger_LLM_Provider_In
* temperature?: float,
* max_tokens?: int,
* response_format?: array{type: string},
* reasoning?: array{enabled: bool, effort?: string},
* } $options Optional parameters. Pass 'reasoning' to override the global setting.
* reasoning?: array{enabled: bool, effort?: string, max_tokens?: int},
* stage?: string,
* prompt_key?: string,
* empty_retry?: bool,
* } $options Optional parameters. Pass 'reasoning' to override the global
* setting. 'stage'/'prompt_key' are caller metadata for the
* empty-completion guard's logging + audit rows (never sent
* upstream); 'empty_retry' is set internally by the guard's
* single reasoning-disabled retry.
*
* @return array{
* content: string,
@@ -52,46 +60,39 @@ class PRAutoBlogger_OpenRouter_Provider implements PRAutoBlogger_LLM_Provider_In
* finish_reason: string,
* }
*
* @throws \RuntimeException On API error after retries exhausted.
* @throws \RuntimeException On API error after retries exhausted, or when the
* completion's visible content is empty/whitespace
* (v0.18.1 guard — after at most one reasoning-disabled retry).
* @throws PRAutoBlogger_Cost_Ceiling_Exception When the per-run cost governor blocks
* the call (run already halted).
*/
public function send_chat_completion( array $messages, string $model, array $options = array() ): array {
$builder = new PRAutoBlogger_OpenRouter_Request_Builder();
// Fetch + format-validate the key (moved to Request_Builder in
// v0.18.0 for the 300-line cap; behavior unchanged — throws the
// same messages on missing/corrupted keys).
$api_key = ( new PRAutoBlogger_OpenRouter_Request_Builder() )->resolve_api_key();
$api_key = $builder->resolve_api_key();
$body = array(
'model' => $model,
'messages' => $messages,
);
// Body assembly — incl. global/per-call reasoning and the v0.18.1
// reasoning token cap + completion headroom — lives in the builder.
$body = $builder->build_body( $messages, $model, $options );
if ( isset( $options['temperature'] ) ) {
$body['temperature'] = $options['temperature'];
}
if ( isset( $options['max_tokens'] ) ) {
$body['max_tokens'] = $options['max_tokens'];
}
if ( isset( $options['response_format'] ) ) {
$body['response_format'] = $options['response_format'];
}
// Reasoning mode: explicit caller override takes priority, then global setting.
if ( isset( $options['reasoning'] ) ) {
$body['reasoning'] = $options['reasoning'];
} elseif ( '1' === get_option( 'prautoblogger_reasoning_enabled', '0' ) ) {
$body['reasoning'] = array(
'enabled' => true,
'effort' => get_option( 'prautoblogger_reasoning_effort', 'medium' ),
);
}
// Whether the outgoing request has reasoning enabled — drives the
// empty-completion retry decision after parse.
$reasoning_active = isset( $body['reasoning'] ) && false !== ( $body['reasoning']['enabled'] ?? true );
// v0.18.0 — per-run cost governor: reserve the worst-case estimate
// BEFORE any dispatch. Throws (and the run is already halted) when
// the reservation would breach the run ceiling; returns null when
// this process is not inside a governed run (historical behavior).
$reservation = PRAutoBlogger_Cost_Governor::open_chat_reservation( $model, $messages, $options );
// v0.18.1: the estimate uses the EFFECTIVE completion ceiling from
// the built body (incl. reasoning headroom), not the raw option.
$govern_options = $options;
if ( isset( $body['max_tokens'] ) ) {
$govern_options['max_tokens'] = $body['max_tokens'];
}
$reservation = PRAutoBlogger_Cost_Governor::open_chat_reservation( $model, $messages, $govern_options );
$last_error = '';
@@ -101,7 +102,6 @@ class PRAutoBlogger_OpenRouter_Provider implements PRAutoBlogger_LLM_Provider_In
$cache_ttl = $config->get_cache_ttl_seconds();
$via_gateway = $config->is_via_gateway();
$builder = new PRAutoBlogger_OpenRouter_Request_Builder();
$request_headers = $builder->build_headers( $api_key, $via_gateway, $cache_ttl );
$curl_auth_filter = $builder->register_curl_auth_filter( $request_headers, $base_host );
@@ -190,7 +190,21 @@ class PRAutoBlogger_OpenRouter_Provider implements PRAutoBlogger_LLM_Provider_In
$this->estimate_cost( $model, (int) $parsed['prompt_tokens'], (int) $parsed['completion_tokens'] )
);
$reservation = null;
return $parsed;
// v0.18.1 — empty-completion guard: empty visible content
// is a FAILURE, never a success. Warns on finish_reason=
// length, retries ONCE with reasoning disabled when that
// can plausibly help, books failed attempts with their
// real status, and otherwise throws. The recursive retry
// manages its own reservation and cURL filter.
return ( new PRAutoBlogger_OpenRouter_Completion_Guard() )->finalize(
$parsed,
$options,
$reasoning_active,
function ( array $retry_options ) use ( $messages, $model ): array {
return $this->send_chat_completion( $messages, $model, $retry_options );
}
);
}
// All retries exhausted.

94
includes/providers/class-open-router-request-builder.php
View File

@@ -18,6 +18,100 @@ declare(strict_types=1);
*/
class PRAutoBlogger_OpenRouter_Request_Builder {
/**
* Build the JSON request body for a chat-completion call.
*
* Centralizes option handling (v0.18.1, moved from the provider for
* the 300-line cap): temperature, max_tokens, response_format, and
* the reasoning block — an explicit caller override takes priority
* over the global setting (that per-call override is how the
* empty-completion retry disables reasoning for its second attempt).
*
* Caller-metadata keys ('stage', 'prompt_key', 'empty_retry') are
* deliberately NOT copied into the HTTP body.
*
* @param array<int, array{role: string, content: string}> $messages Chat messages.
* @param string $model Model identifier.
* @param array<string, mixed> $options Call options.
*
* @return array<string, mixed> Request body ready for wp_json_encode().
*/
public function build_body( array $messages, string $model, array $options ): array {
$body = array(
'model' => $model,
'messages' => $messages,
);
if ( isset( $options['temperature'] ) ) {
$body['temperature'] = $options['temperature'];
}
if ( isset( $options['max_tokens'] ) ) {
$body['max_tokens'] = $options['max_tokens'];
}
if ( isset( $options['response_format'] ) ) {
$body['response_format'] = $options['response_format'];
}
// Reasoning mode: explicit caller override takes priority, then global setting.
if ( isset( $options['reasoning'] ) ) {
$body['reasoning'] = $options['reasoning'];
} elseif ( '1' === get_option( 'prautoblogger_reasoning_enabled', '0' ) ) {
$body['reasoning'] = array(
'enabled' => true,
'effort' => get_option( 'prautoblogger_reasoning_effort', 'medium' ),
);
}
return $this->apply_reasoning_budget( $body );
}
/**
* Reasoning budget sanity (v0.18.1): cap thinking, protect content.
*
* When the outgoing request enables reasoning and carries a completion
* ceiling, the thinking budget is capped via OpenRouter's
* `reasoning.max_tokens` and the completion ceiling is raised by the
* same amount — so no reasoning effort (incl. xhigh) can consume the
* entire completion budget and emit an empty message (the 2026-06-11
* empty-draft incident, prod run acf24029). The cap is SETTINGS-backed
* (`prautoblogger_reasoning_max_tokens`, default
* PRAUTOBLOGGER_DEFAULT_REASONING_MAX_TOKENS, 0 = pure effort mode:
* no cap, no headroom — the historical behavior). When the cap is
* active it replaces `effort` — OpenRouter treats the two as
* alternative budget controls and normalizes max_tokens per model.
*
* No-op when reasoning is absent/disabled on the request, when the
* request has no completion ceiling, or when the cap setting is 0.
*
* @param array<string, mixed> $body Assembled request body.
*
* @return array<string, mixed> Body with the reasoning budget applied.
*/
private function apply_reasoning_budget( array $body ): array {
if ( ! isset( $body['reasoning'] ) || false === ( $body['reasoning']['enabled'] ?? true ) ) {
return $body;
}
if ( ! isset( $body['max_tokens'] ) ) {
return $body; // No completion ceiling to protect.
}
if ( ! isset( $body['reasoning']['max_tokens'] ) ) {
$cap = absint( get_option( 'prautoblogger_reasoning_max_tokens', PRAUTOBLOGGER_DEFAULT_REASONING_MAX_TOKENS ) );
if ( $cap < 1 ) {
return $body; // Cap disabled — pure effort mode.
}
$body['reasoning']['max_tokens'] = $cap;
unset( $body['reasoning']['effort'] );
}
// Headroom: models that spend reasoning from the completion budget
// (e.g. deepseek/deepseek-v4-flash) must still have the caller's
// full max_tokens available for visible content.
$body['max_tokens'] = (int) $body['max_tokens'] + (int) $body['reasoning']['max_tokens'];
return $body;
}
/**
* Build request headers for OpenRouter API call.
*

1
includes/services/opik/class-opik-eval-scorer.php
View File

@@ -82,6 +82,7 @@ class PRAutoBlogger_Opik_Eval_Scorer {
array(
'temperature' => 0.1,
'max_tokens' => 200,
'stage' => 'opik_eval_judge',
)
);

10
prautoblogger.php
View File

@@ -9,7 +9,7 @@
* Plugin Name: PRAutoBlogger
* Plugin URI: https://peptiderepo.com/prautoblogger
* Description: Monitors social media for trending topics, generates SEO-friendly blog posts using AI, and publishes them on a daily schedule with full cost tracking and self-improvement metrics.
* Version: 0.18.0
* Version: 0.18.1
* Requires at least: 6.0
* Requires PHP: 7.4
* Author: PeptideRepo
@@ -35,7 +35,7 @@ if ( ! defined( 'ABSPATH' ) ) {
| Defined here so every file in the plugin can reference paths, versions,
| and limits without magic strings.
*/
define( 'PRAUTOBLOGGER_VERSION', '0.18.0' );
define( 'PRAUTOBLOGGER_VERSION', '0.18.1' );
define( 'PRAUTOBLOGGER_DB_VERSION', '1.2.0' );
define( 'PRAUTOBLOGGER_PLUGIN_FILE', __FILE__ );
define( 'PRAUTOBLOGGER_PLUGIN_DIR', plugin_dir_path( __FILE__ ) );
@@ -49,6 +49,12 @@ define( 'PRAUTOBLOGGER_MAX_RETRIES', 3 );
// the named get_option() fallbacks — never inline the literals.
define( 'PRAUTOBLOGGER_DEFAULT_RUN_CEILING_USD', 0.50 );
define( 'PRAUTOBLOGGER_DEFAULT_REQUEST_JSON_RETENTION_DAYS', 14 );
// v0.18.1: ceiling on "thinking" tokens per reasoning-enabled LLM call. The
// request's completion budget (max_tokens) is RAISED by this amount so
// reasoning can never consume the visible-content budget and return an empty
// message (the 2026-06-11 empty-draft incident). SETTINGS-backed
// (prautoblogger_reasoning_max_tokens); 0 disables the cap (pure effort mode).
define( 'PRAUTOBLOGGER_DEFAULT_REASONING_MAX_TOKENS', 2048 );
define( 'PRAUTOBLOGGER_RETRY_BASE_DELAY_SECONDS', 2 );
define( 'PRAUTOBLOGGER_API_TIMEOUT_SECONDS', 180 );
define( 'PRAUTOBLOGGER_CACHE_TTL_SECONDS', 3600 );

3
tests/bootstrap.php
View File

@@ -28,6 +28,9 @@ if ( ! defined( 'PRAUTOBLOGGER_DEFAULT_RUN_CEILING_USD' ) ) {
if ( ! defined( 'PRAUTOBLOGGER_DEFAULT_REQUEST_JSON_RETENTION_DAYS' ) ) {
define( 'PRAUTOBLOGGER_DEFAULT_REQUEST_JSON_RETENTION_DAYS', 14 );
}
if ( ! defined( 'PRAUTOBLOGGER_DEFAULT_REASONING_MAX_TOKENS' ) ) {
define( 'PRAUTOBLOGGER_DEFAULT_REASONING_MAX_TOKENS', 2048 );
}
// WordPress time constants used by CostTracker and ContentAnalyzer.
if ( ! defined( 'DAY_IN_SECONDS' ) ) {

164
tests/unit/Core/ContentGeneratorTest.php Normal file
View File

@@ -0,0 +1,164 @@
<?php
/**
* Tests for PRAutoBlogger_Content_Generator (v0.18.1 writer-path guard).
*
* Validates that a writing stage whose LLM output is empty/whitespace
* throws instead of passing an empty article downstream, that healthy
* output passes through, and that the stage metadata for the provider's
* empty-completion guard rides the options array.
*
* execute_stage() is exercised via reflection: generate() defines the
* PRAUTOBLOGGER_EVAL_MODE constant, which cannot be redefined across
* tests in one PHP process.
*
* @package PRAutoBlogger\Tests\Core
*/
namespace PRAutoBlogger\Tests\Core;
use PRAutoBlogger\Tests\BaseTestCase;
use Brain\Monkey\Functions;
class ContentGeneratorTest extends BaseTestCase {
private \PRAutoBlogger_Content_Request $request;
protected function setUp(): void {
parent::setUp();
\PRAutoBlogger_Prompt_Registry::flush_cache();
$this->stub_get_option( [
'prautoblogger_log_level' => 'error',
] );
// Content_Prompts::build_system() lists recent posts for the
// linking rules — none exist in the unit environment.
Functions\when( 'get_posts' )->justReturn( [] );
$idea = new \PRAutoBlogger_Article_Idea( $this->get_article_idea_fixture() );
$this->request = new \PRAutoBlogger_Content_Request(
$idea,
'single_pass',
'informational',
800,
2000,
'',
[],
''
);
}
protected function tearDown(): void {
\PRAutoBlogger_Prompt_Registry::flush_cache();
parent::tearDown();
}
/**
* Invoke the private execute_stage() on a generator wired to the
* given LLM response.
*
* @param array $llm_response Response the mock provider returns.
* @param array|null $captured Receives the options passed to the LLM.
* @return string Stage output.
*/
private function run_stage( array $llm_response, ?array &$captured = null ): string {
$llm = $this->createMock( \PRAutoBlogger_LLM_Provider_Interface::class );
$llm->method( 'send_chat_completion' )->willReturnCallback(
function ( array $messages, string $model, array $options ) use ( $llm_response, &$captured ) {
$captured = $options;
return $llm_response;
}
);
$llm->method( 'get_provider_name' )->willReturn( 'OpenRouter' );
$llm->method( 'estimate_cost' )->willReturn( 0.001 );
$cost_tracker = $this->createMock( \PRAutoBlogger_Cost_Tracker::class );
$generator = new \PRAutoBlogger_Content_Generator( $llm, $cost_tracker );
$method = new \ReflectionMethod( \PRAutoBlogger_Content_Generator::class, 'execute_stage' );
$method->setAccessible( true );
return $method->invoke(
$generator,
'draft',
'draft_generation',
'Write the article.',
$this->request,
'model/test',
[
'temperature' => 0.7,
'max_tokens' => 4000,
],
'content.single_pass'
);
}
/**
* Healthy stage output passes through unchanged.
*/
public function test_non_empty_stage_output_passes_through(): void {
$output = $this->run_stage( [
'content' => '<p>A real article.</p>',
'model' => 'model/test',
'prompt_tokens' => 100,
'completion_tokens' => 200,
] );
$this->assertSame( '<p>A real article.</p>', $output );
}
/**
* Empty stage output throws — no empty article may leave the writer.
*/
public function test_empty_stage_output_throws(): void {
$this->expectException( \RuntimeException::class );
$this->expectExceptionMessage( 'produced empty content' );
$this->run_stage( [
'content' => '',
'model' => 'model/test',
'prompt_tokens' => 100,
'completion_tokens' => 4000,
] );
}
/**
* Whitespace-only output counts as empty (the guard trims).
*/
public function test_whitespace_stage_output_throws(): void {
$this->expectException( \RuntimeException::class );
$this->expectExceptionMessage( 'draft' );
$this->run_stage( [
'content' => " \n\t ",
'model' => 'model/test',
'prompt_tokens' => 100,
'completion_tokens' => 4000,
] );
}
/**
* The stage + prompt_key metadata for the provider's empty-completion
* guard rides the LLM options (v0.18.1).
*/
public function test_stage_metadata_rides_the_llm_options(): void {
$captured = null;
$this->run_stage(
[
'content' => 'ok',
'model' => 'model/test',
'prompt_tokens' => 1,
'completion_tokens' => 1,
],
$captured
);
$this->assertSame( 'draft', $captured['stage'] );
$this->assertSame( 'content.single_pass', $captured['prompt_key'] );
$this->assertSame( 4000, $captured['max_tokens'] );
}
}

35
tests/unit/Core/PublisherTest.php
View File

@@ -36,6 +36,10 @@ class PublisherTest extends BaseTestCase {
Functions\when( 'sanitize_text_field' )->returnArg();
Functions\when( 'wp_kses_post' )->returnArg();
// v0.18.1 empty-content guard strips tags before the emptiness check.
Functions\when( 'wp_strip_all_tags' )->alias( function ( $text ) {
return trim( strip_tags( (string) $text ) );
} );
Functions\when( 'is_wp_error' )->justReturn( false );
Functions\when( 'apply_filters' )->returnArg( 2 );
Functions\when( 'absint' )->alias( function ( $val ) { return abs( (int) $val ); } );
@@ -295,4 +299,35 @@ class PublisherTest extends BaseTestCase {
$publisher = new \PRAutoBlogger_Publisher();
$publisher->publish( '<p>Content</p>', $this->idea, $this->review );
}
/**
* v0.18.1 belt-and-braces: save_as_draft() with empty content throws
* and never reaches wp_insert_post — no empty draft post can be
* created (the 2026-06-11 post-921 incident shape).
*/
public function test_save_as_draft_refuses_empty_content(): void {
Functions\expect( 'wp_insert_post' )->never();
$publisher = new \PRAutoBlogger_Publisher();
$this->expectException( \RuntimeException::class );
$this->expectExceptionMessage( 'generated content is empty' );
$publisher->save_as_draft( '', $this->idea, $this->review, 'run_empty_draft' );
}
/**
* v0.18.1 belt-and-braces: publish() refuses content that is empty
* once tags and whitespace are stripped (an empty HTML shell).
*/
public function test_publish_refuses_whitespace_only_html_content(): void {
Functions\expect( 'wp_insert_post' )->never();
$publisher = new \PRAutoBlogger_Publisher();
$this->expectException( \RuntimeException::class );
$this->expectExceptionMessage( 'generated content is empty' );
$publisher->publish( "<p> \n\t </p>", $this->idea, $this->review );
}
}

240
tests/unit/Providers/OpenRouterCompletionGuardTest.php Normal file
View File

@@ -0,0 +1,240 @@
<?php
/**
* Tests for PRAutoBlogger_OpenRouter_Completion_Guard (v0.18.1).
*
* Validates the empty-completion verdict matrix: healthy pass-through,
* the single reasoning-disabled retry on length/reasoning empties, the
* never-retry-twice rule, and the hard failure for unexplained empties.
*
* @package PRAutoBlogger\Tests\Providers
*/
namespace PRAutoBlogger\Tests\Providers;
use PRAutoBlogger\Tests\BaseTestCase;
use Brain\Monkey\Functions;
class OpenRouterCompletionGuardTest extends BaseTestCase {
protected function setUp(): void {
parent::setUp();
\PRAutoBlogger_Run_Context::clear();
\PRAutoBlogger_Audit_Writer::flush_cache();
$this->stub_get_option( [
'prautoblogger_log_level' => 'error',
] );
// Cost_Tracker::log_api_call() (failed-attempt booking) inserts a
// generation_log row and prices the burn via the pricing chain.
$wpdb = $this->create_mock_wpdb();
$wpdb->method( 'insert' )->willReturn( 1 );
$wpdb->insert_id = 7;
$GLOBALS['wpdb'] = $wpdb;
}
protected function tearDown(): void {
unset( $GLOBALS['wpdb'] );
\PRAutoBlogger_Run_Context::clear();
\PRAutoBlogger_Audit_Writer::flush_cache();
parent::tearDown();
}
/**
* Build a parsed-response fixture in the Response_Parser shape.
*
* @param string $content Visible content.
* @param string $finish finish_reason value.
* @return array
*/
private function parsed( string $content, string $finish ): array {
return [
'content' => $content,
'model' => 'deepseek/deepseek-v4-flash',
'prompt_tokens' => 1200,
'completion_tokens' => 4000,
'reasoning_tokens' => 4000,
'total_tokens' => 5200,
'finish_reason' => $finish,
'reasoning_content' => 'thinking…',
];
}
/**
* Healthy completion passes through untouched; the retry callable is
* never invoked.
*/
public function test_healthy_completion_passes_through(): void {
$parsed = $this->parsed( 'A full article.', 'stop' );
$invoked = 0;
$result = ( new \PRAutoBlogger_OpenRouter_Completion_Guard() )->finalize(
$parsed,
[ 'stage' => 'draft' ],
false,
function () use ( &$invoked ) {
++$invoked;
return [];
}
);
$this->assertSame( $parsed, $result );
$this->assertSame( 0, $invoked );
}
/**
* The incident shape — empty content + finish_reason=length — retries
* exactly once with reasoning disabled and the empty_retry marker set,
* and returns the retry's result.
*/
public function test_empty_plus_length_retries_once_with_reasoning_disabled(): void {
$recovered = $this->parsed( 'Recovered article.', 'stop' );
$captured_opts = null;
$result = ( new \PRAutoBlogger_OpenRouter_Completion_Guard() )->finalize(
$this->parsed( '', 'length' ),
[ 'stage' => 'draft', 'prompt_key' => 'content.single_pass' ],
true,
function ( array $retry_options ) use ( &$captured_opts, $recovered ) {
$captured_opts = $retry_options;
return $recovered;
}
);
$this->assertSame( $recovered, $result );
$this->assertSame( [ 'enabled' => false ], $captured_opts['reasoning'] );
$this->assertTrue( $captured_opts['empty_retry'] );
$this->assertSame( 'draft', $captured_opts['stage'] );
}
/**
* Whitespace-only content counts as empty (the guard trims).
*/
public function test_whitespace_only_content_counts_as_empty(): void {
$invoked = 0;
( new \PRAutoBlogger_OpenRouter_Completion_Guard() )->finalize(
$this->parsed( " \n\t ", 'length' ),
[ 'stage' => 'polish' ],
true,
function ( array $retry_options ) use ( &$invoked ) {
++$invoked;
return $this->parsed( 'ok', 'stop' );
}
);
$this->assertSame( 1, $invoked );
}
/**
* An attempt already marked empty_retry is never retried again — the
* guard fails the call instead (no infinite retry loops).
*/
public function test_already_retried_empty_fails_without_second_retry(): void {
$invoked = 0;
try {
( new \PRAutoBlogger_OpenRouter_Completion_Guard() )->finalize(
$this->parsed( '', 'length' ),
[ 'stage' => 'draft', 'empty_retry' => true ],
false,
function () use ( &$invoked ) {
++$invoked;
return [];
}
);
$this->fail( 'Expected RuntimeException for retried-and-still-empty completion.' );
} catch ( \RuntimeException $e ) {
$this->assertStringContainsString( 'empty content', $e->getMessage() );
$this->assertStringContainsString( 'after a reasoning-disabled retry', $e->getMessage() );
}
$this->assertSame( 0, $invoked );
}
/**
* Empty content with finish_reason=stop and no reasoning active: a
* reasoning-disabled retry cannot plausibly help — fail immediately.
*/
public function test_unexplained_empty_fails_without_retry(): void {
$invoked = 0;
try {
( new \PRAutoBlogger_OpenRouter_Completion_Guard() )->finalize(
$this->parsed( '', 'stop' ),
[ 'stage' => 'review' ],
false,
function () use ( &$invoked ) {
++$invoked;
return [];
}
);
$this->fail( 'Expected RuntimeException for unexplained empty completion.' );
} catch ( \RuntimeException $e ) {
$this->assertStringContainsString( 'stage "review"', $e->getMessage() );
$this->assertStringContainsString( 'finish_reason=stop', $e->getMessage() );
}
$this->assertSame( 0, $invoked );
}
/**
* Empty content with reasoning active retries even when finish_reason
* is not 'length' (reasoning can eat the budget without a length flag).
*/
public function test_empty_with_reasoning_active_retries_on_any_finish_reason(): void {
$invoked = 0;
( new \PRAutoBlogger_OpenRouter_Completion_Guard() )->finalize(
$this->parsed( '', 'stop' ),
[ 'stage' => 'draft' ],
true,
function ( array $retry_options ) use ( &$invoked ) {
++$invoked;
return $this->parsed( 'ok', 'stop' );
}
);
$this->assertSame( 1, $invoked );
}
/**
* The failed attempt is booked to generation_log with its REAL status:
* response_status='error', the stage, the model, and the token burn.
*/
public function test_failed_attempt_books_generation_log_row_with_error_status(): void {
$captured_row = null;
$wpdb = $this->create_mock_wpdb();
$wpdb->method( 'insert' )->willReturnCallback(
function ( $table, $row ) use ( &$captured_row ) {
if ( false !== strpos( (string) $table, 'generation_log' ) ) {
$captured_row = $row;
}
return 1;
}
);
$wpdb->insert_id = 9;
$GLOBALS['wpdb'] = $wpdb;
try {
( new \PRAutoBlogger_OpenRouter_Completion_Guard() )->finalize(
$this->parsed( '', 'stop' ),
[ 'stage' => 'draft' ],
false,
function () {
return [];
}
);
$this->fail( 'Expected RuntimeException.' );
} catch ( \RuntimeException $e ) {
$this->assertIsArray( $captured_row );
$this->assertSame( 'error', $captured_row['response_status'] );
$this->assertSame( 'draft', $captured_row['stage'] );
$this->assertSame( 'deepseek/deepseek-v4-flash', $captured_row['model'] );
$this->assertSame( 4000, $captured_row['completion_tokens'] );
$this->assertStringContainsString( 'Empty completion', $captured_row['error_message'] );
}
}
}

149
tests/unit/Providers/OpenRouterProviderTest.php
View File

@@ -166,4 +166,153 @@ class OpenRouterProviderTest extends BaseTestCase {
$this->expectException( \RuntimeException::class );
$provider->send_chat_completion( $messages, 'model/test', $options );
}
/**
* Wire the provider for a full send_chat_completion run: decryptable
* API key, reasoning settings, HTTP stubs that replay scripted
* responses, and a wpdb mock for the guard's failure bookkeeping.
*
* @param array $option_overrides Extra get_option values.
* @param array $responses Scripted JSON-decodable response bodies.
* @param array $requests (by ref) captures decoded request bodies.
*/
private function wire_full_provider( array $option_overrides, array $responses, array &$requests ): void {
\PRAutoBlogger_Run_Context::clear();
$encrypted_key = \PRAutoBlogger_Encryption::encrypt( 'sk-or-v1-unit-test-key-0000' );
$this->stub_get_option( array_merge( [
'prautoblogger_openrouter_api_key' => $encrypted_key,
'prautoblogger_log_level' => 'error',
], $option_overrides ) );
Functions\when( 'absint' )->alias( function ( $val ) {
return abs( (int) $val );
} );
Functions\when( 'wp_remote_post' )->alias(
function ( $url, $args ) use ( &$requests, $responses ) {
$requests[] = json_decode( $args['body'], true );
$body = $responses[ count( $requests ) - 1 ];
return [
'body' => $body,
'response' => [ 'code' => 200 ],
];
}
);
Functions\when( 'wp_remote_retrieve_response_code' )->alias(
function ( $response ) {
return $response['response']['code'];
}
);
Functions\when( 'wp_remote_retrieve_body' )->alias(
function ( $response ) {
return $response['body'];
}
);
// The guard books failed attempts through Cost_Tracker + Logger.
$wpdb = $this->create_mock_wpdb();
$wpdb->method( 'insert' )->willReturn( 1 );
$wpdb->insert_id = 11;
$GLOBALS['wpdb'] = $wpdb;
}
/**
* v0.18.1 incident shape, end to end: reasoning enabled (uncapped
* effort mode), the first response burns the whole completion budget
* on thinking and returns empty content with finish_reason=length —
* the provider retries exactly once with reasoning disabled and
* returns the recovered completion.
*/
public function test_empty_length_completion_retries_once_with_reasoning_disabled(): void {
$requests = [];
$empty_response = json_encode( [
'model' => 'deepseek/deepseek-v4-flash',
'choices' => [ [ 'message' => [ 'content' => '', 'reasoning' => 'endless thinking…' ], 'finish_reason' => 'length' ] ],
'usage' => [ 'prompt_tokens' => 1200, 'completion_tokens' => 4000, 'reasoning_tokens' => 4000, 'total_tokens' => 5200 ],
] );
$good_response = json_encode( [
'model' => 'deepseek/deepseek-v4-flash',
'choices' => [ [ 'message' => [ 'content' => 'Recovered article.' ], 'finish_reason' => 'stop' ] ],
'usage' => [ 'prompt_tokens' => 1200, 'completion_tokens' => 900, 'total_tokens' => 2100 ],
] );
$this->wire_full_provider(
[
'prautoblogger_reasoning_enabled' => '1',
'prautoblogger_reasoning_effort' => 'xhigh',
'prautoblogger_reasoning_max_tokens' => 0,
],
[ $empty_response, $good_response ],
$requests
);
try {
$provider = new \PRAutoBlogger_OpenRouter_Provider();
$result = $provider->send_chat_completion(
[ [ 'role' => 'user', 'content' => 'Write the article.' ] ],
'deepseek/deepseek-v4-flash',
[ 'temperature' => 0.7, 'max_tokens' => 4000, 'stage' => 'draft' ]
);
$this->assertSame( 'Recovered article.', $result['content'] );
$this->assertCount( 2, $requests );
// First attempt: global reasoning, uncapped effort mode.
$this->assertSame( [ 'enabled' => true, 'effort' => 'xhigh' ], $requests[0]['reasoning'] );
$this->assertSame( 4000, $requests[0]['max_tokens'] );
// Retry: reasoning disabled per call.
$this->assertSame( [ 'enabled' => false ], $requests[1]['reasoning'] );
// Caller metadata never reaches the HTTP body.
$this->assertArrayNotHasKey( 'stage', $requests[0] );
$this->assertArrayNotHasKey( 'empty_retry', $requests[1] );
} finally {
unset( $GLOBALS['wpdb'] );
\PRAutoBlogger_Run_Context::clear();
}
}
/**
* v0.18.1 reasoning budget: with the token cap active (default), the
* outgoing request caps thinking via reasoning.max_tokens and raises
* max_tokens by the cap — and a healthy completion needs no retry.
*/
public function test_reasoning_cap_shapes_the_outgoing_request(): void {
$requests = [];
$good_response = json_encode( [
'model' => 'deepseek/deepseek-v4-flash',
'choices' => [ [ 'message' => [ 'content' => 'Article text.' ], 'finish_reason' => 'stop' ] ],
'usage' => [ 'prompt_tokens' => 1200, 'completion_tokens' => 900, 'total_tokens' => 2100 ],
] );
$this->wire_full_provider(
[
'prautoblogger_reasoning_enabled' => '1',
'prautoblogger_reasoning_effort' => 'xhigh',
'prautoblogger_reasoning_max_tokens' => 2048,
],
[ $good_response ],
$requests
);
try {
$provider = new \PRAutoBlogger_OpenRouter_Provider();
$result = $provider->send_chat_completion(
[ [ 'role' => 'user', 'content' => 'Write the article.' ] ],
'deepseek/deepseek-v4-flash',
[ 'temperature' => 0.7, 'max_tokens' => 4000, 'stage' => 'draft' ]
);
$this->assertSame( 'Article text.', $result['content'] );
$this->assertCount( 1, $requests );
$this->assertSame( 2048, $requests[0]['reasoning']['max_tokens'] );
$this->assertArrayNotHasKey( 'effort', $requests[0]['reasoning'] );
$this->assertSame( 6048, $requests[0]['max_tokens'] );
} finally {
unset( $GLOBALS['wpdb'] );
\PRAutoBlogger_Run_Context::clear();
}
}
}

189
tests/unit/Providers/OpenRouterRequestBuilderTest.php Normal file
View File

@@ -0,0 +1,189 @@
<?php
/**
* Tests for PRAutoBlogger_OpenRouter_Request_Builder::build_body() (v0.18.1).
*
* Validates body assembly, the global-vs-per-call reasoning precedence, the
* reasoning token cap + completion headroom, and that caller-metadata keys
* never leak into the HTTP body.
*
* @package PRAutoBlogger\Tests\Providers
*/
namespace PRAutoBlogger\Tests\Providers;
use PRAutoBlogger\Tests\BaseTestCase;
use Brain\Monkey\Functions;
class OpenRouterRequestBuilderTest extends BaseTestCase {
private const MESSAGES = [
[ 'role' => 'system', 'content' => 'sys' ],
[ 'role' => 'user', 'content' => 'usr' ],
];
protected function setUp(): void {
parent::setUp();
Functions\when( 'absint' )->alias( function ( $val ) {
return abs( (int) $val );
} );
}
/**
* Reasoning enabled globally + cap set: thinking is capped via
* reasoning.max_tokens, effort is dropped, and the completion budget
* is raised by the cap so content keeps its full allowance.
*/
public function test_reasoning_cap_replaces_effort_and_raises_headroom(): void {
$this->stub_get_option( [
'prautoblogger_reasoning_enabled' => '1',
'prautoblogger_reasoning_effort' => 'xhigh',
'prautoblogger_reasoning_max_tokens' => 2048,
] );
$body = ( new \PRAutoBlogger_OpenRouter_Request_Builder() )
->build_body( self::MESSAGES, 'deepseek/deepseek-v4-flash', [ 'max_tokens' => 4000 ] );
$this->assertSame( 2048, $body['reasoning']['max_tokens'] );
$this->assertArrayNotHasKey( 'effort', $body['reasoning'] );
$this->assertSame( 6048, $body['max_tokens'] );
$this->assertTrue( $body['reasoning']['enabled'] );
}
/**
* Cap setting 0 = pure effort mode: the historical body shape is
* preserved exactly (effort kept, no reasoning.max_tokens, no headroom).
*/
public function test_cap_zero_keeps_pure_effort_mode(): void {
$this->stub_get_option( [
'prautoblogger_reasoning_enabled' => '1',
'prautoblogger_reasoning_effort' => 'xhigh',
'prautoblogger_reasoning_max_tokens' => 0,
] );
$body = ( new \PRAutoBlogger_OpenRouter_Request_Builder() )
->build_body( self::MESSAGES, 'deepseek/deepseek-v4-flash', [ 'max_tokens' => 4000 ] );
$this->assertSame( [ 'enabled' => true, 'effort' => 'xhigh' ], $body['reasoning'] );
$this->assertSame( 4000, $body['max_tokens'] );
}
/**
* Per-call reasoning override (the guard's retry) beats the global
* setting; a disabled override gets no cap and no headroom.
*/
public function test_per_call_disabled_override_beats_global_setting(): void {
$this->stub_get_option( [
'prautoblogger_reasoning_enabled' => '1',
'prautoblogger_reasoning_effort' => 'xhigh',
'prautoblogger_reasoning_max_tokens' => 2048,
] );
$body = ( new \PRAutoBlogger_OpenRouter_Request_Builder() )
->build_body(
self::MESSAGES,
'deepseek/deepseek-v4-flash',
[
'max_tokens' => 4000,
'reasoning' => [ 'enabled' => false ],
]
);
$this->assertSame( [ 'enabled' => false ], $body['reasoning'] );
$this->assertSame( 4000, $body['max_tokens'] );
}
/**
* Global reasoning off + no override: no reasoning key at all and no
* budget math (historical default path).
*/
public function test_reasoning_off_produces_no_reasoning_key(): void {
$this->stub_get_option( [
'prautoblogger_reasoning_enabled' => '0',
] );
$body = ( new \PRAutoBlogger_OpenRouter_Request_Builder() )
->build_body(
self::MESSAGES,
'google/gemini-2.5-flash-lite',
[
'temperature' => 0.7,
'max_tokens' => 4000,
'response_format' => [ 'type' => 'json_object' ],
]
);
$this->assertArrayNotHasKey( 'reasoning', $body );
$this->assertSame( 0.7, $body['temperature'] );
$this->assertSame( 4000, $body['max_tokens'] );
$this->assertSame( [ 'type' => 'json_object' ], $body['response_format'] );
}
/**
* An explicit caller reasoning config with its own max_tokens is
* respected (no cap overwrite) but still gets completion headroom.
*/
public function test_caller_reasoning_max_tokens_respected_with_headroom(): void {
$this->stub_get_option( [
'prautoblogger_reasoning_enabled' => '0',
'prautoblogger_reasoning_max_tokens' => 2048,
] );
$body = ( new \PRAutoBlogger_OpenRouter_Request_Builder() )
->build_body(
self::MESSAGES,
'deepseek/deepseek-v4-flash',
[
'max_tokens' => 1000,
'reasoning' => [ 'enabled' => true, 'max_tokens' => 512 ],
]
);
$this->assertSame( 512, $body['reasoning']['max_tokens'] );
$this->assertSame( 1512, $body['max_tokens'] );
}
/**
* Caller-metadata keys (stage, prompt_key, empty_retry) must never
* appear in the HTTP body.
*/
public function test_caller_metadata_keys_never_leak_into_body(): void {
$this->stub_get_option( [
'prautoblogger_reasoning_enabled' => '0',
] );
$body = ( new \PRAutoBlogger_OpenRouter_Request_Builder() )
->build_body(
self::MESSAGES,
'google/gemini-2.5-flash-lite',
[
'max_tokens' => 4000,
'stage' => 'draft',
'prompt_key' => 'content.single_pass',
'empty_retry' => true,
]
);
$this->assertArrayNotHasKey( 'stage', $body );
$this->assertArrayNotHasKey( 'prompt_key', $body );
$this->assertArrayNotHasKey( 'empty_retry', $body );
$this->assertSame( [ 'model', 'messages', 'max_tokens' ], array_keys( $body ) );
}
/**
* Reasoning enabled but the call has no completion ceiling: nothing to
* protect, body passes through with effort untouched.
*/
public function test_no_max_tokens_means_no_budget_math(): void {
$this->stub_get_option( [
'prautoblogger_reasoning_enabled' => '1',
'prautoblogger_reasoning_effort' => 'high',
'prautoblogger_reasoning_max_tokens' => 2048,
] );
$body = ( new \PRAutoBlogger_OpenRouter_Request_Builder() )
->build_body( self::MESSAGES, 'deepseek/deepseek-v4-flash', [] );
$this->assertSame( [ 'enabled' => true, 'effort' => 'high' ], $body['reasoning'] );
$this->assertArrayNotHasKey( 'max_tokens', $body );
}
}