fix(cost): resolve snapshot-suffixed model IDs to base pricing before conservative estimate

OpenRouter sometimes returns a dated snapshot ID as the response model
(request deepseek/deepseek-v4-flash -> response model
deepseek/deepseek-v4-flash-20260423). The pricing chain missed the
registry on those IDs and booked the conservative $10/$30 per-M
estimate: today's prod single-pass run logged $0.582 against a true
flash-tier cost of roughly 1-2 cents. Inflated actuals eat the
Cost_Governor per-run ceiling (default $0.50; today's run would have
halted) and the monthly $5 budget gate (~9 days to self-block).

Fix at the single pricing seam (OpenRouter_Pricing::estimate_cost):
on a lookup miss, strip a trailing 8-digit 20-prefixed -YYYYMMDD
snapshot suffix and retry the hardcoded table + cached models list
once with the base ID. The conservative fallback is unchanged when
the base ID is also unknown. Governor reserve/settle and
generation-log booking all price through this seam, so every path
survives snapshot IDs.

Tests: snapshot ID resolves to the base price when only the base ID
is registered; an unknown ID without a suffix still gets the
conservative estimate; a non-date trailing digit run (foo-202612345)
is never stripped.

Agent-Session: pipeline-v2-phase1-pricing-fix-20260611
This commit is contained in:
peptiderepo
2026-06-11 05:31:57 +00:00
parent 1062af2815
commit e6e3f85542
3 changed files with 171 additions and 13 deletions

View File

@@ -99,6 +99,12 @@ behavior change** to the Economy (single-pass) and multi-step publish paths.
columns when the columns are missing (a cron run right after deploy no longer loses cost columns when the columns are missing (a cron run right after deploy no longer loses cost
rows), and the db-version self-heal now also runs on cron requests (`init` + rows), and the db-version self-heal now also runs on cron requests (`init` +
`wp_doing_cron()`), not only on `admin_init`. `wp_doing_cron()`), not only on `admin_init`.
- OpenRouter pricing now resolves dated snapshot model IDs returned by the API
(e.g. response model `deepseek/deepseek-v4-flash-20260423` for a
`deepseek/deepseek-v4-flash` request) to the base model's price before the
conservative-estimate fallback applies. Such responses were previously booked
at the conservative $10/$30 per-M rate (~30x real cost on flash-tier models) —
phantom spend that ate the per-run cost ceiling and the monthly budget gate.
## [0.17.0] - 2026-06-11 ## [0.17.0] - 2026-06-11

View File

@@ -132,7 +132,9 @@ class PRAutoBlogger_OpenRouter_Pricing {
/** /**
* Estimate the cost of an API call in USD. * Estimate the cost of an API call in USD.
* *
* Uses hardcoded pricing first, falls back to cached model data. * Uses hardcoded pricing first, falls back to cached model data. IDs
* carrying a date-snapshot suffix that miss both sources are retried
* once as their base ID (see resolve_pricing()).
* *
* @param string $model Model identifier. * @param string $model Model identifier.
* @param int $prompt_tokens Input tokens. * @param int $prompt_tokens Input tokens.
@@ -146,18 +148,7 @@ class PRAutoBlogger_OpenRouter_Pricing {
return 0.0; return 0.0;
} }
$pricing = self::MODEL_PRICING[ $model ] ?? null; $pricing = $this->resolve_pricing( $model );
// Fall back to cached model list pricing.
if ( null === $pricing ) {
$models = $this->get_available_models();
foreach ( $models as $m ) {
if ( $m['id'] === $model && isset( $m['pricing'] ) ) {
$pricing = $m['pricing'];
break;
}
}
}
if ( null === $pricing ) { if ( null === $pricing ) {
// Unknown model — use a conservative estimate. // Unknown model — use a conservative estimate.
@@ -172,6 +163,82 @@ class PRAutoBlogger_OpenRouter_Pricing {
+ ( $completion_tokens * $pricing['completion'] / 1000000 ); + ( $completion_tokens * $pricing['completion'] / 1000000 );
} }
/**
* Resolve pricing for a model ID, normalizing dated snapshot IDs.
*
* Tries the exact ID first. On a miss, strips a trailing date-snapshot
* suffix and retries once with the base ID: OpenRouter responses can
* name a dated snapshot of the requested model (e.g. request
* 'deepseek/deepseek-v4-flash', response model
* 'deepseek/deepseek-v4-flash-20260423') while pricing sources only
* list the base ID. Every cost path (governor reserve/settle and
* generation-log booking) prices through this seam, so all of them
* survive snapshot IDs instead of booking the conservative estimate.
*
* Side effects: may fetch the models list over HTTP on cache miss
* (via get_available_models()).
*
* @param string $model Model ID as requested, or as returned by the API.
*
* @return array{prompt: float, completion: float}|null Per-million USD pricing, or null when unknown.
*/
private function resolve_pricing( string $model ): ?array {
$pricing = $this->lookup_pricing( $model );
if ( null !== $pricing ) {
return $pricing;
}
$base = $this->strip_snapshot_suffix( $model );
if ( $base !== $model ) {
return $this->lookup_pricing( $base );
}
return null;
}
/**
* Look up one exact model ID: hardcoded table first, then cached model list.
*
* Side effects: may fetch the models list over HTTP on cache miss
* (via get_available_models()).
*
* @param string $model Model ID (exact match, no normalization).
*
* @return array{prompt: float, completion: float}|null Per-million USD pricing, or null on miss.
*/
private function lookup_pricing( string $model ): ?array {
$pricing = self::MODEL_PRICING[ $model ] ?? null;
if ( null !== $pricing ) {
return $pricing;
}
foreach ( $this->get_available_models() as $m ) {
if ( $m['id'] === $model && isset( $m['pricing'] ) ) {
return $m['pricing'];
}
}
return null;
}
/**
* Strip a trailing date-snapshot suffix (e.g. '-20260423') from a model ID.
*
* Only a dash followed by an 8-digit, 20-prefixed date at the very end
* is treated as a snapshot suffix. Any other trailing digit run (e.g.
* 'foo-202612345') is part of the model name and left untouched.
*
* Side effects: none — pure string transformation.
*
* @param string $model Model ID.
*
* @return string Base model ID, or the input unchanged when no snapshot suffix is present.
*/
private function strip_snapshot_suffix( string $model ): string {
$base = preg_replace( '/-20\d{6}$/', '', $model );
return is_string( $base ) ? $base : $model;
}
/** /**
* Get the decrypted API key from options. * Get the decrypted API key from options.
* *

View File

@@ -177,4 +177,89 @@ class OpenRouterPricingTest extends BaseTestCase {
// At least some models should be present. // At least some models should be present.
$this->assertGreaterThan( 0, count( $models ) ); $this->assertGreaterThan( 0, count( $models ) );
} }
/**
* Test a snapshot-suffixed response model ID resolves to the base model's price.
*
* OpenRouter sometimes returns a dated snapshot ID as the response model
* (e.g. 'deepseek/deepseek-v4-flash-20260423' for a
* 'deepseek/deepseek-v4-flash' request). Pricing must strip the date
* suffix and bill at the base model's rate, not the conservative fallback.
*/
public function test_estimate_cost_snapshot_suffix_resolves_to_base_model_price(): void {
Functions\when( 'get_transient' )->alias(
function ( string $key ) {
if ( 'prautoblogger_openrouter_models' === $key ) {
return [
[
'id' => 'deepseek/deepseek-v4-flash',
'name' => 'DeepSeek V4 Flash',
'context_length' => 131072,
'pricing' => [ 'prompt' => 0.05, 'completion' => 0.20 ],
],
];
}
return false;
}
);
$pricing = new \PRAutoBlogger_OpenRouter_Pricing();
$base_cost = $pricing->estimate_cost( 'deepseek/deepseek-v4-flash', 1000, 500 );
$snapshot_cost = $pricing->estimate_cost( 'deepseek/deepseek-v4-flash-20260423', 1000, 500 );
// Billed exactly like the base model: (1000 * 0.05 + 500 * 0.20) / 1M.
$this->assertSame( $base_cost, $snapshot_cost );
$this->assertEqualsWithDelta( 0.00015, $snapshot_cost, 1e-12 );
// And not at the conservative fallback rate (10/30 per M = 0.025 here).
$this->assertNotEquals( 0.025, $snapshot_cost );
}
/**
* Test an unknown model with no date suffix still gets the conservative estimate.
*/
public function test_estimate_cost_unknown_model_without_suffix_uses_conservative_estimate(): void {
$pricing = new \PRAutoBlogger_OpenRouter_Pricing();
$cost = $pricing->estimate_cost( 'unknown/fake-model', 1000, 500 );
// Conservative fallback: (1000 * $10 + 500 * $30) / 1M.
$this->assertEqualsWithDelta( 0.025, $cost, 1e-12 );
}
/**
* Test a trailing digit run that is not an 8-digit date is never stripped.
*
* 'foo-202612345' has nine trailing digits — even with 'foo' and 'foo-2'
* registered, no stripping may occur and the conservative estimate applies.
*/
public function test_estimate_cost_non_date_suffix_is_not_stripped(): void {
Functions\when( 'get_transient' )->alias(
function ( string $key ) {
if ( 'prautoblogger_openrouter_models' === $key ) {
return [
[
'id' => 'foo',
'name' => 'Foo',
'context_length' => 8192,
'pricing' => [ 'prompt' => 0.01, 'completion' => 0.02 ],
],
[
'id' => 'foo-2',
'name' => 'Foo 2',
'context_length' => 8192,
'pricing' => [ 'prompt' => 0.01, 'completion' => 0.02 ],
],
];
}
return false;
}
);
$pricing = new \PRAutoBlogger_OpenRouter_Pricing();
$cost = $pricing->estimate_cost( 'foo-202612345', 1000, 500 );
// Nine trailing digits is not a snapshot date — conservative estimate applies.
$this->assertEqualsWithDelta( 0.025, $cost, 1e-12 );
}
} }