peptide-repo/pr-vision
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

PR Vision v0.3.0: per-call cost accountability + LLM call audit trail

Browse Source 
P0 security:
- PRV_Gateway_Client: remove response body from RuntimeException (HTTP status only)
- PRV_Capture_Writer: allowlist-only write (prompt/response text, never headers/auth)
- Mandatory capture order: can_afford → probe → persist_result → update_row_cost → write_meta → capture_io
- capture_io last and wrapped in swallowing try/catch

New tables (SCHEMA_VERSION 3):
- prv_call_meta: cost/metadata per call, kept indefinitely
- prv_call_io: rendered prompt + raw response, pruned on retention window (default 90d)

New classes:
- PRV_Call_Meta_Table, PRV_Call_Io_Table: table schema + dbDelta
- PRV_Capture_Writer: metadata + I/O write; security allowlist enforced
- PRV_Prune_Cron: daily prv_daily_prune cron deletes aged prv_call_io rows only
- PRV_Cost_Rollup_Query: MTD summary, model breakdown, drill-down, projection
- PRV_Call_Log_Query: paginated + filtered call-log query + per-call I/O fetch
- PRV_Costs_Page + PRV_Costs_Renderer: Costs admin sub-page (cap progress, bento, model table, drill-down)
- PRV_Call_Log_Page + PRV_Call_Log_Table_Renderer: Call Log admin sub-page (filter bar, paginated table, WCAG AA pagination)
- PRV_Call_Drawer_Renderer: non-modal complementary drawer (5 states: Normal/Pruned/Legacy/Failed/Loading)
- PRV_Call_Drawer_Script: inline JS for drawer (300-line split from renderer)
- PRV_Call_Detail_Ajax: wp_ajax_prv_call_detail AJAX handler
- PRV_Admin_Page_Css: CSS provider (300-line split from PRV_Admin_Page)
- PRV_Probe_Run_Executor: inner probe loop (300-line split from PRV_Probe_Runner)

Updated:
- PRV_Probe_Result: tokens_in/tokens_out params + getters
- Both providers: report tokens_in/tokens_out from usage.prompt_tokens/completion_tokens
- PRV_Probe_Runner: slimmed to lock+UUID+delegation; injects PRV_Capture_Writer
- PRV_Config: get_io_retention_days(); PRV_IO_RETENTION_DEFAULT_DAYS constant
- PRV_Upgrader/Activator/Deactivator/Plugin: register new tables, cron, pages, AJAX
- Settings: retention days field + save in controller
- uninstall.php: DROP both new tables, clear prv_daily_prune cron
- Subnav 4-tab strip (Dashboard | Costs | Call Log | Settings) on all pages
- ARCHITECTURE.md, CHANGELOG.md, CONTEXT.md, CONVENTIONS.md updated

Tests:
- test-capture-writer.php: write_meta, write_io, P0 security (401 record has no key/Bearer/sk-or-)
- test-cost-audit.php: MTD reconciliation, projection, prune, forced capture_io exception, table drops

Agent-Session: peptiderepo-geomonitor-v0.3.0-cost-audit
This commit is contained in:
peptiderepo
2026-06-15 09:09:29 +00:00
parent 2f4c1b2c72
commit 4eee9f5f64
37 changed files with 3137 additions and 328 deletions
Download Patch File Download Diff File Expand all files Collapse all files

299
includes/core/class-prv-probe-run-executor.php Normal file
View File

@@ -0,0 +1,299 @@
<?php
/**
* Executes the inner loop of a probe run (v0.3.0 extraction).
*
* @package PrVision
*/
declare(strict_types=1);
/**
* Executes a single probe run's inner loop: peptides × intents × models.
*
* Split from PRV_Probe_Runner to keep each class under 300 lines.
* Called only by PRV_Probe_Runner::execute_run() (inside the run-lock).
*
* Capture ordering (P0 mandatory):
* can_afford → probe → persist_result → update_row_cost → write_meta → capture_io
* capture_io is wrapped in a swallowing try/catch — failure cannot throw.
*
* Who triggers: PRV_Probe_Runner::execute_run().
* Dependencies: PRV_Cost_Ledger, PRV_Capture_Writer, PRV_Table_Manager,
* PRV_Config, PRV_Config_Version, PRV_Model_Registry, providers.
*
* @see class-prv-probe-runner.php — Acquires lock + calls run().
* @see class-prv-capture-writer.php — Per-call metadata + I/O writer (P0).
* @see ARCHITECTURE.md — Section Probe run flow v0.3.0.
* @package PrVision
*/
class PRV_Probe_Run_Executor {
/**
* Budget ledger for cost-cap enforcement.
*
* @var PRV_Cost_Ledger
*/
private PRV_Cost_Ledger $ledger;
/**
* Per-call capture writer (best-effort; never in critical path).
*
* @var PRV_Capture_Writer
*/
private PRV_Capture_Writer $capture;
/**
* Constructor.
*
* @param PRV_Cost_Ledger $ledger Budget ledger.
* @param PRV_Capture_Writer $capture Capture writer.
*/
public function __construct( PRV_Cost_Ledger $ledger, PRV_Capture_Writer $capture ) {
$this->ledger = $ledger;
$this->capture = $capture;
}
/**
* Run the full peptide × intent × model loop.
*
* Side effects: HTTP calls to LLM APIs, database writes, option writes.
*
* @param string $run_id UUID for this run.
* @param int $config_ver Config version at run time.
*
* @return array{probed: int, skipped_budget: int, skipped_error: int, truncated: bool, run_id: string}
*/
public function execute(
string $run_id,
int $config_ver
): array {
$peptides = PRV_Config::get_peptides();
$intents = PRV_Config::get_prompt_intents();
$models = PRV_Config::get_models();
$model_outcomes = array();
foreach ( $models as $slug ) {
$model_outcomes[ $slug ] = array( 'probed' => 0, 'errors' => 0 );
}
$counts = array(
'probed' => 0,
'skipped_budget' => 0,
'skipped_error' => 0,
'truncated' => false,
'run_id' => $run_id,
);
$budget_hit = false;
foreach ( $peptides as $peptide ) {
foreach ( $intents as $intent_tpl ) {
$query = str_replace( '{peptide}', $peptide['label'], $intent_tpl );
foreach ( $models as $model ) {
$this->probe_one( $run_id, $peptide, $model, $intent_tpl, $query, $config_ver, $counts, $model_outcomes, $budget_hit );
}
}
}
if ( $budget_hit && $counts['probed'] > 0 ) {
$counts['truncated'] = true;
}
update_option( 'prv_last_run_truncated', $counts['truncated'] ? 1 : 0 );
if ( $counts['truncated'] ) {
update_option( 'prv_last_run_truncated_at', current_time( 'mysql', true ) );
}
update_option( 'prv_last_run_at', current_time( 'mysql', true ) );
update_option( 'prv_last_run_counts', $counts );
PRV_Model_Registry::update_health( $run_id, $model_outcomes );
$this->settle_api_key_status( $counts, $model_outcomes );
return $counts;
}
/**
* Execute one probe combination in mandatory P0 order.
*
* @param string $run_id Run UUID.
* @param array<string,string> $peptide Peptide config.
* @param string $model Model slug.
* @param string $intent_tpl Intent template.
* @param string $query Rendered query.
* @param int $config_ver Config version.
* @param array<string,mixed> &$counts Counts (modified).
* @param array<string,array{probed:int,errors:int}> &$model_outcomes Outcomes (modified).
* @param bool &$budget_hit Budget flag (modified).
*
* @return void
*/
private function probe_one(
string $run_id,
array $peptide,
string $model,
string $intent_tpl,
string $query,
int $config_ver,
array &$counts,
array &$model_outcomes,
bool &$budget_hit
): void {
$provider = $this->resolve_provider( $model );
if ( null === $provider || ! $provider->is_configured() ) {
++$counts['skipped_error'];
if ( isset( $model_outcomes[ $model ] ) ) {
++$model_outcomes[ $model ]['errors'];
}
return;
}
$estimated = $this->get_estimated_cost( $model );
if ( ! $this->ledger->can_afford( $estimated ) ) {
++$counts['skipped_budget'];
$budget_hit = true;
return;
}
$start_ms = (int) round( microtime( true ) * 1000 );
try {
$result = $provider->probe( $query );
$latency_ms = (int) round( microtime( true ) * 1000 ) - $start_ms;
$http_status = 200;
} catch ( \Exception $e ) {
$latency_ms = (int) round( microtime( true ) * 1000 ) - $start_ms;
$http_status = $this->extract_http_status( $e );
// phpcs:ignore WordPress.PHP.DevelopmentFunctions.error_log_error_log
error_log( sprintf( 'PRV: probe failed [%s / %s]: %s', $model, $peptide['slug'], $e->getMessage() ) );
++$counts['skipped_error'];
if ( isset( $model_outcomes[ $model ] ) ) {
++$model_outcomes[ $model ]['errors'];
}
$call_id = $this->capture->write_meta(
array(
'visibility_row' => null, 'run_id' => $run_id,
'peptide_slug' => $peptide['slug'], 'model' => $model,
'intent_label' => $intent_tpl, 'tokens_in' => null,
'tokens_out' => null, 'cost_usd' => 0.0,
'latency_ms' => $latency_ms, 'cited' => null,
'http_status' => $http_status, 'config_version' => $config_ver,
)
);
if ( $call_id > 0 ) {
try { $this->capture->write_io( $call_id, $query, '' ); } catch ( \Throwable $io_err ) {}
}
return;
}
$row_id = $this->persist_result( $run_id, $peptide, $model, $intent_tpl, $result, $config_ver );
if ( $row_id > 0 ) {
$this->ledger->update_row_cost( $row_id, $result->get_cost_usd() );
}
$call_id = $this->capture->write_meta(
array(
'visibility_row' => $row_id > 0 ? $row_id : null, 'run_id' => $run_id,
'peptide_slug' => $peptide['slug'], 'model' => $model,
'intent_label' => $intent_tpl, 'tokens_in' => $result->get_tokens_in(),
'tokens_out' => $result->get_tokens_out(), 'cost_usd' => $result->get_cost_usd(),
'latency_ms' => $latency_ms, 'cited' => $result->is_cited() ? 1 : 0,
'http_status' => $http_status, 'config_version' => $config_ver,
)
);
if ( $call_id > 0 ) {
try { $this->capture->write_io( $call_id, $query, $result->get_raw_excerpt() ); } catch ( \Throwable $io_err ) {}
}
++$counts['probed'];
if ( isset( $model_outcomes[ $model ] ) ) {
++$model_outcomes[ $model ]['probed'];
}
}
/**
* Persist one probe result to prv_ai_visibility.
*
* @param string $run_id UUID. @param array<string,string> $peptide Peptide config.
* @param string $model Model slug. @param string $intent_tpl Intent template.
* @param PRV_Probe_Result $result Probe result. @param int $config_ver Config version.
*
* @return int Row ID, or 0 on failure.
*/
private function persist_result( string $run_id, array $peptide, string $model, string $intent_tpl, PRV_Probe_Result $result, int $config_ver ): int {
global $wpdb;
$table = PRV_Table_Manager::get_table_name();
// phpcs:ignore WordPress.DB.DirectDatabaseQuery.DirectQuery
$wpdb->insert(
$table,
array(
'run_id' => $run_id, 'captured_at' => current_time( 'mysql', true ),
'peptide_slug' => $peptide['slug'], 'peptide_label' => $peptide['label'],
'model' => $model, 'prompt_intent' => $intent_tpl,
'cited' => $result->is_cited() ? 1 : 0, 'our_position' => $result->get_our_position(),
'source_domains' => wp_json_encode( $result->get_source_domains() ),
'raw_excerpt' => $result->get_raw_excerpt(),
'cost_usd' => number_format( $result->get_cost_usd(), 8, '.', '' ),
'config_version' => $config_ver,
),
array( '%s', '%s', '%s', '%s', '%s', '%s', '%d', '%d', '%s', '%s', '%s', '%d' )
);
return (int) $wpdb->insert_id;
}
/**
* Extract HTTP status from a gateway exception message.
*
* @param \Exception $e Caught exception.
*
* @return int HTTP status, or 0 if unknown.
*/
private function extract_http_status( \Exception $e ): int {
if ( preg_match( '/PR Vision gateway HTTP (\d+)/', $e->getMessage(), $m ) ) {
return (int) $m[1];
}
return 0;
}
/**
* Resolve provider instance for a model slug.
*
* @param string $model Model slug. @return PRV_Probe_Provider|null
*/
private function resolve_provider( string $model ): ?PRV_Probe_Provider {
if ( 'perplexity/sonar' === $model ) {
return new PRV_Perplexity_Provider();
}
return new PRV_OpenRouter_Provider( $model );
}
/**
* Conservative per-call cost estimate for budget pre-check.
*
* @param string $model Model slug. @return float Estimated USD.
*/
private function get_estimated_cost( string $model ): float {
if ( 'perplexity/sonar' === $model ) {
return PRV_Perplexity_Provider::ESTIMATED_COST_PER_PROBE;
}
return PRV_OpenRouter_Provider::ESTIMATED_COST_PER_PROBE;
}
/**
* Write the API-key status option based on run outcomes.
*
* @param array<string,mixed> $counts Run counts.
* @param array<string,array{probed:int,errors:int}> $model_outcomes Model outcomes.
*
* @return void
*/
private function settle_api_key_status( array $counts, array $model_outcomes ): void {
if ( $counts['probed'] > 0 ) {
update_option( 'prv_api_key_status', 'ok' );
} elseif ( $counts['skipped_error'] > 0 && 0 === $counts['probed'] ) {
update_option( 'prv_api_key_status', 'failed' );
}
update_option( 'prv_api_key_last_check', current_time( 'mysql', true ) );
}
}