Files
pr-vision/includes/core/class-prv-probe-run-executor.php
peptiderepo 39dbe8fcc1 Adopt reusable CI + PHPUnit + fold onto shared deploy (#9)
* v0.3.2 — adopt estate CI: PHPUnit suite + reusable pipeline + governance workflows

- Convert 17 hand-rolled tests to PHPUnit TestCases (tests: stubs pattern)
  P0 security tests preserved: key-leak/encryption (KeyStoreTest),
  capture-allowlist/executor-path sentinel (CaptureWriterTest)
- Add phpunit.xml.dist + update tests/bootstrap.php (Composer autoloader first)
- Add phpunit/phpunit ^9.6 + yoast/phpunit-polyfills ^2.0 to require-dev
- composer test -> vendor/bin/phpunit --configuration phpunit.xml.dist
- ci.yml: thin caller of peptide-e2e reusable CI (tests: stubs, has_js: true,
  workflow_call trigger, permissions: contents: write)
- deploy.yml: estate pattern caller of peptide-e2e deploy-app.yml@main
  (target_path: wp-content/plugins/pr-vision, gated on reusable CI)
  Pull-deploy on Hostinger stays as break-glass
- Add cto-review.yml (Haiku-only, A/B panel dropped 2026-06-12)
- Add main-push-audit.yml (direct-push tripwire)

Agent-Session: claude-adopt-ci-20260616

* style: auto-fix PHPCS violations via phpcbf

* fix: isolate sentinel-constant test + disable js-lint

Two CI failures fixed:

1. PHPUnit constant-scoping bleed: test_p0_executor_path_sentinel_key_does_not_
   leak defines PRV_OPENROUTER_API_KEY as a process-scoped PHP constant.
   Running after CaptureWriter (alphabetically earlier), KeyStoreTest's
   resolver tests saw the canary value instead of empty/option.
   Fix: @runInSeparateProcess + @preserveGlobalState disabled so the
   constant is confined to its own forked PHP process.

2. JS lint job failure: has_js: true caused the reusable CI's lint-js job
   to attempt npm ci, but pr-vision has no package.json.
   Fixed: has_js: false.

Agent-Session: claude/adopt-ci-20260616

* fix: resolve all PHPUnit constant-scoping + crypto + collector test failures

Root causes identified and fixed:

1. Missing salts in bootstrap: derive_encryption_key() returned '' (no 32-byte
   key) because AUTH_KEY/SECURE_AUTH_KEY were undefined. All crypto group tests
   in KeyStoreTest failed as a result. Fix: define AUTH_KEY + SECURE_AUTH_KEY
   in tests/bootstrap.php with deterministic 64-char test salts.

2. PHP constant mid-run redefinition: ConfigVersionTest.setUp() defined
   PRV_OPENROUTER_API_KEY='sk-or-test-key', which bled into KeyStoreTest
   (runs after ConfigVersionTest alphabetically). Same constant was defined in
   5 test classes (ConfigVersion, ProbeRunner, ProjectedCost, RunLock, Uninstall).
   Fix: define PRV_OPENROUTER_API_KEY once in bootstrap; remove per-class defines.
   Update KeyStoreTest expectations to reflect constant-is-always-set reality:
   get_source() => SOURCE_CONSTANT, get_key() => 'sk-or-test-key'.

3. Remaining @runInSeparateProcess complexity: CaptureWriterTest sentinel test
   was using @runInSeparateProcess to avoid constant bleed, but that caused
   error_log() calls inside the executor to surface as PHPUnit\Framework\Exception
   in the child process output. Now that constant is defined in bootstrap, the
   sentinel test runs in the main process and just asserts 'sk-or-test-key'
   doesn't appear in stored rows (equivalent P0 security coverage).

4. DashboardPanelTest collector calls get_results() twice (trendline + standings).
   The wpdb stub returned the same array for both calls; the second call expected
   peptide_slug keys but got trendline keys, causing 'Undefined array key' error.
   Fix: add wpdb_results_queue support to the stub (shifts one result set per
   call when queue is non-empty); update collector tests to use the queue.

Agent-Session: claude/adopt-ci-20260616

* fix: update KeyStoreTest Group 2 for always-defined constant

Group 2 tests (test_get_source_none_when_no_key, test_store_key_writes_encrypted_option,
test_clear_key_removes_option) were still expecting SOURCE_NONE and SOURCE_OPTION.
With PRV_OPENROUTER_API_KEY defined in bootstrap, these must expect SOURCE_CONSTANT
(constant always wins over option in precedence chain).

- Renamed test_get_source_none_when_no_key -> test_get_source_constant_when_constant_defined
- Updated test_store_key_writes_encrypted_option: SOURCE_OPTION -> SOURCE_CONSTANT assertion
- Updated test_clear_key_removes_option: SOURCE_NONE -> SOURCE_CONSTANT assertion

Agent-Session: claude/adopt-ci-20260616

---------

Co-authored-by: peptiderepo <peptiderepo@users.noreply.github.com>
2026-06-17 01:16:37 +07:00

301 lines
9.9 KiB
PHP
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<?php
/**
* Inner probe loop executor split from PRV_Probe_Runner (v0.3.0).
*
* @package PrVision
*/
declare(strict_types=1);
/**
* Executes one probe run's inner loop: peptides × intents × models.
* P0: can_afford → probe → persist → update_cost → write_meta → [swallowed] capture_io.
*
* @see class-prv-capture-writer.php — Allowlist-only capture writer (P0).
* @package PrVision
*/
class PRV_Probe_Run_Executor {
/**
* Budget ledger for cost-cap enforcement.
*
* @var PRV_Cost_Ledger
*/
private PRV_Cost_Ledger $ledger;
/**
* Per-call capture writer (best-effort; never in critical path).
*
* @var PRV_Capture_Writer
*/
private PRV_Capture_Writer $capture;
/**
* Constructor.
*
* @param PRV_Cost_Ledger $ledger Budget ledger.
* @param PRV_Capture_Writer $capture Capture writer.
*/
public function __construct( PRV_Cost_Ledger $ledger, PRV_Capture_Writer $capture ) {
$this->ledger = $ledger;
$this->capture = $capture;
}
/**
* Run the full peptide × intent × model loop.
*
* @param string $run_id UUID for this run.
* @param int $config_ver Config version.
* @return array{probed: int, skipped_budget: int, skipped_error: int, truncated: bool, run_id: string}
*/
public function execute(
string $run_id,
int $config_ver
): array {
$peptides = PRV_Config::get_peptides();
$intents = PRV_Config::get_prompt_intents();
$models = PRV_Config::get_models();
$model_outcomes = array();
foreach ( $models as $slug ) {
// phpcs:ignore WordPress.Arrays.ArrayDeclarationSpacing.AssociativeArrayFound
$model_outcomes[ $slug ] = array( 'probed' => 0, 'errors' => 0 );
}
$counts = array(
'probed' => 0,
'skipped_budget' => 0,
'skipped_error' => 0,
'truncated' => false,
'run_id' => $run_id,
);
$budget_hit = false;
foreach ( $peptides as $peptide ) {
foreach ( $intents as $intent_tpl ) {
$query = str_replace( '{peptide}', $peptide['label'], $intent_tpl );
foreach ( $models as $model ) {
$this->probe_one( $run_id, $peptide, $model, $intent_tpl, $query, $config_ver, $counts, $model_outcomes, $budget_hit );
}
}
}
if ( $budget_hit && $counts['probed'] > 0 ) {
$counts['truncated'] = true;
}
update_option( 'prv_last_run_truncated', $counts['truncated'] ? 1 : 0 );
if ( $counts['truncated'] ) {
update_option( 'prv_last_run_truncated_at', current_time( 'mysql', true ) );
}
update_option( 'prv_last_run_at', current_time( 'mysql', true ) );
update_option( 'prv_last_run_counts', $counts );
PRV_Model_Registry::update_health( $run_id, $model_outcomes );
$this->settle_api_key_status( $counts, $model_outcomes );
return $counts;
}
/**
* Execute one probe combination: can_afford → probe → persist → capture (P0 order).
*
* @param string $run_id Run UUID.
* @param array<string,string> $peptide Peptide config.
* @param string $model Model slug.
* @param string $intent_tpl Intent template.
* @param string $query Rendered query string.
* @param int $config_ver Config version.
* @param array<string,mixed> &$counts Counts accumulator.
* @param array<string,array> &$model_outcomes Per-model outcome map.
* @param bool &$budget_hit Budget exhaustion flag.
* @return void
*/
private function probe_one(
string $run_id,
array $peptide,
string $model,
string $intent_tpl,
string $query,
int $config_ver,
array &$counts,
array &$model_outcomes,
bool &$budget_hit
): void {
$provider = $this->resolve_provider( $model );
if ( null === $provider || ! $provider->is_configured() ) {
++$counts['skipped_error'];
if ( isset( $model_outcomes[ $model ] ) ) {
++$model_outcomes[ $model ]['errors'];
}
return;
}
$estimated = $this->get_estimated_cost( $model );
if ( ! $this->ledger->can_afford( $estimated ) ) {
++$counts['skipped_budget'];
$budget_hit = true;
return;
}
$start_ms = (int) round( microtime( true ) * 1000 );
try {
$result = $provider->probe( $query );
$latency_ms = (int) round( microtime( true ) * 1000 ) - $start_ms;
$http_status = 200;
} catch ( \Exception $e ) {
$latency_ms = (int) round( microtime( true ) * 1000 ) - $start_ms;
$http_status = $this->extract_http_status( $e );
// phpcs:ignore WordPress.PHP.DevelopmentFunctions.error_log_error_log
error_log( sprintf( 'PRV: probe failed [%s / %s]: %s', $model, $peptide['slug'], $e->getMessage() ) );
++$counts['skipped_error'];
if ( isset( $model_outcomes[ $model ] ) ) {
++$model_outcomes[ $model ]['errors'];
}
// phpcs:disable WordPress.Arrays.ArrayDeclarationSpacing.ArrayItemNoNewLine, Universal.WhiteSpace.CommaSpacing.TooMuchSpaceAfter
$cid = $this->capture->write_meta(
array(
'visibility_row' => null, 'run_id' => $run_id,
'peptide_slug' => $peptide['slug'], 'model' => $model,
'intent_label' => $intent_tpl, 'tokens_in' => null,
'tokens_out' => null, 'cost_usd' => 0.0,
'latency_ms' => $latency_ms, 'cited' => null,
'http_status' => $http_status, 'config_version' => $config_ver,
)
); // phpcs:enable
if ( $cid > 0 ) {
try {
$this->capture->write_io( $cid, $query, '' );
} catch ( \Throwable $e2 ) {
// Best-effort: I/O capture failure must not propagate (P0-4).
unset( $e2 );
}
}
return;
}
$row_id = $this->persist_result( $run_id, $peptide, $model, $intent_tpl, $result, $config_ver );
if ( $row_id > 0 ) {
$this->ledger->update_row_cost( $row_id, $result->get_cost_usd() );
}
// phpcs:disable WordPress.Arrays.ArrayDeclarationSpacing.ArrayItemNoNewLine, Universal.WhiteSpace.CommaSpacing.TooMuchSpaceAfter
$cid = $this->capture->write_meta(
array(
'visibility_row' => $row_id > 0 ? $row_id : null, 'run_id' => $run_id,
'peptide_slug' => $peptide['slug'], 'model' => $model,
'intent_label' => $intent_tpl, 'tokens_in' => $result->get_tokens_in(),
'tokens_out' => $result->get_tokens_out(), 'cost_usd' => $result->get_cost_usd(),
'latency_ms' => $latency_ms, 'cited' => $result->is_cited() ? 1 : 0,
'http_status' => $http_status, 'config_version' => $config_ver,
)
); // phpcs:enable
if ( $cid > 0 ) {
try {
$this->capture->write_io( $cid, $query, $result->get_raw_excerpt() );
} catch ( \Throwable $e2 ) {
// Best-effort: I/O capture failure must not propagate (P0-4).
unset( $e2 );
}
}
++$counts['probed'];
if ( isset( $model_outcomes[ $model ] ) ) {
++$model_outcomes[ $model ]['probed'];
}
}
/**
* Persist one probe result to prv_ai_visibility.
*
* @param string $run_id Run UUID.
* @param array<string,string> $peptide Peptide config.
* @param string $model Model slug.
* @param string $intent_tpl Intent template.
* @param PRV_Probe_Result $result Probe result.
* @param int $config_ver Config version.
* @return int Row ID, or 0 on failure.
*/
private function persist_result( string $run_id, array $peptide, string $model, string $intent_tpl, PRV_Probe_Result $result, int $config_ver ): int {
global $wpdb;
$table = PRV_Table_Manager::get_table_name();
// phpcs:ignore WordPress.DB.DirectDatabaseQuery.DirectQuery
$wpdb->insert(
$table,
array(
'run_id' => $run_id,
'captured_at' => current_time( 'mysql', true ),
'peptide_slug' => $peptide['slug'],
'peptide_label' => $peptide['label'],
'model' => $model,
'prompt_intent' => $intent_tpl,
'cited' => $result->is_cited() ? 1 : 0,
'our_position' => $result->get_our_position(),
'source_domains' => wp_json_encode( $result->get_source_domains() ),
'raw_excerpt' => $result->get_raw_excerpt(),
'cost_usd' => number_format( $result->get_cost_usd(), 8, '.', '' ),
'config_version' => $config_ver,
),
array( '%s', '%s', '%s', '%s', '%s', '%s', '%d', '%d', '%s', '%s', '%s', '%d' )
);
return (int) $wpdb->insert_id;
}
/**
* Extract HTTP status from a gateway exception message.
*
* @param \Exception $e Caught exception.
* @return int HTTP status, or 0 if unknown.
*/
private function extract_http_status( \Exception $e ): int {
if ( preg_match( '/PR Vision gateway HTTP (\d+)/', $e->getMessage(), $m ) ) {
return (int) $m[1];
}
return 0;
}
/**
* Resolve provider instance for a model slug.
*
* @param string $model Model slug.
* @return PRV_Probe_Provider|null Provider, or null if not configured.
*/
private function resolve_provider( string $model ): ?PRV_Probe_Provider {
if ( 'perplexity/sonar' === $model ) {
return new PRV_Perplexity_Provider();
}
return new PRV_OpenRouter_Provider( $model );
}
/**
* Conservative per-call cost estimate for budget pre-check.
*
* @param string $model Model slug.
* @return float Estimated USD per call.
*/
private function get_estimated_cost( string $model ): float {
if ( 'perplexity/sonar' === $model ) {
return PRV_Perplexity_Provider::ESTIMATED_COST_PER_PROBE;
}
return PRV_OpenRouter_Provider::ESTIMATED_COST_PER_PROBE;
}
/**
* Write the API-key status option based on run outcomes.
*
* @param array<string,mixed> $counts Run counts.
* @param array<string,array{probed:int,errors:int}> $model_outcomes Model outcomes.
* @return void
*/
private function settle_api_key_status( array $counts, array $model_outcomes ): void {
if ( $counts['probed'] > 0 ) {
update_option( 'prv_api_key_status', 'ok' );
} elseif ( $counts['skipped_error'] > 0 && 0 === $counts['probed'] ) {
update_option( 'prv_api_key_status', 'failed' );
}
update_option( 'prv_api_key_last_check', current_time( 'mysql', true ) );
}
}