Files
pr-vision/includes/collector/class-prv-ai-visibility-collector.php
peptiderepo 39dbe8fcc1 Adopt reusable CI + PHPUnit + fold onto shared deploy (#9)
* v0.3.2 — adopt estate CI: PHPUnit suite + reusable pipeline + governance workflows

- Convert 17 hand-rolled tests to PHPUnit TestCases (tests: stubs pattern)
  P0 security tests preserved: key-leak/encryption (KeyStoreTest),
  capture-allowlist/executor-path sentinel (CaptureWriterTest)
- Add phpunit.xml.dist + update tests/bootstrap.php (Composer autoloader first)
- Add phpunit/phpunit ^9.6 + yoast/phpunit-polyfills ^2.0 to require-dev
- composer test -> vendor/bin/phpunit --configuration phpunit.xml.dist
- ci.yml: thin caller of peptide-e2e reusable CI (tests: stubs, has_js: true,
  workflow_call trigger, permissions: contents: write)
- deploy.yml: estate pattern caller of peptide-e2e deploy-app.yml@main
  (target_path: wp-content/plugins/pr-vision, gated on reusable CI)
  Pull-deploy on Hostinger stays as break-glass
- Add cto-review.yml (Haiku-only, A/B panel dropped 2026-06-12)
- Add main-push-audit.yml (direct-push tripwire)

Agent-Session: claude-adopt-ci-20260616

* style: auto-fix PHPCS violations via phpcbf

* fix: isolate sentinel-constant test + disable js-lint

Two CI failures fixed:

1. PHPUnit constant-scoping bleed: test_p0_executor_path_sentinel_key_does_not_
   leak defines PRV_OPENROUTER_API_KEY as a process-scoped PHP constant.
   Running after CaptureWriter (alphabetically earlier), KeyStoreTest's
   resolver tests saw the canary value instead of empty/option.
   Fix: @runInSeparateProcess + @preserveGlobalState disabled so the
   constant is confined to its own forked PHP process.

2. JS lint job failure: has_js: true caused the reusable CI's lint-js job
   to attempt npm ci, but pr-vision has no package.json.
   Fixed: has_js: false.

Agent-Session: claude/adopt-ci-20260616

* fix: resolve all PHPUnit constant-scoping + crypto + collector test failures

Root causes identified and fixed:

1. Missing salts in bootstrap: derive_encryption_key() returned '' (no 32-byte
   key) because AUTH_KEY/SECURE_AUTH_KEY were undefined. All crypto group tests
   in KeyStoreTest failed as a result. Fix: define AUTH_KEY + SECURE_AUTH_KEY
   in tests/bootstrap.php with deterministic 64-char test salts.

2. PHP constant mid-run redefinition: ConfigVersionTest.setUp() defined
   PRV_OPENROUTER_API_KEY='sk-or-test-key', which bled into KeyStoreTest
   (runs after ConfigVersionTest alphabetically). Same constant was defined in
   5 test classes (ConfigVersion, ProbeRunner, ProjectedCost, RunLock, Uninstall).
   Fix: define PRV_OPENROUTER_API_KEY once in bootstrap; remove per-class defines.
   Update KeyStoreTest expectations to reflect constant-is-always-set reality:
   get_source() => SOURCE_CONSTANT, get_key() => 'sk-or-test-key'.

3. Remaining @runInSeparateProcess complexity: CaptureWriterTest sentinel test
   was using @runInSeparateProcess to avoid constant bleed, but that caused
   error_log() calls inside the executor to surface as PHPUnit\Framework\Exception
   in the child process output. Now that constant is defined in bootstrap, the
   sentinel test runs in the main process and just asserts 'sk-or-test-key'
   doesn't appear in stored rows (equivalent P0 security coverage).

4. DashboardPanelTest collector calls get_results() twice (trendline + standings).
   The wpdb stub returned the same array for both calls; the second call expected
   peptide_slug keys but got trendline keys, causing 'Undefined array key' error.
   Fix: add wpdb_results_queue support to the stub (shifts one result set per
   call when queue is non-empty); update collector tests to use the queue.

Agent-Session: claude/adopt-ci-20260616

* fix: update KeyStoreTest Group 2 for always-defined constant

Group 2 tests (test_get_source_none_when_no_key, test_store_key_writes_encrypted_option,
test_clear_key_removes_option) were still expecting SOURCE_NONE and SOURCE_OPTION.
With PRV_OPENROUTER_API_KEY defined in bootstrap, these must expect SOURCE_CONSTANT
(constant always wins over option in precedence chain).

- Renamed test_get_source_none_when_no_key -> test_get_source_constant_when_constant_defined
- Updated test_store_key_writes_encrypted_option: SOURCE_OPTION -> SOURCE_CONSTANT assertion
- Updated test_clear_key_removes_option: SOURCE_NONE -> SOURCE_CONSTANT assertion

Agent-Session: claude/adopt-ci-20260616

---------

Co-authored-by: peptiderepo <peptiderepo@users.noreply.github.com>
2026-06-17 01:16:37 +07:00

232 lines
8.9 KiB
PHP
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<?php
/**
* AI-visibility data collector for the PR Vision dashboard.
*
* @package PrVision
*/
declare(strict_types=1);
/**
* AI-visibility data collector — v1 implementation of PRV_Data_Collector.
*
* Reads the prv_ai_visibility table and returns structured data for:
* - Trendline: per-run visibility score (% cited, position-weighted) with config_version stamp.
* - Standings: per-peptide latest cited status, our position, competing domains.
* - Run metadata: last run time, MTD cost vs cap.
* - Per-model run-health from PRV_Model_Registry (last_run_counts).
* - All config-version records for trendline break annotations (config_versions).
*
* Score formula (documented in CONTEXT.md):
* base_score = cited_probes / total_probes (range 0–1)
* position_bonus = Σ (1 / our_position) for cited probes / total_probes
* visibility_score = round((base_score + position_bonus) / 2, 4)
*
* Who triggers: PRV_Admin_Page calls the registered collector via the registry.
* Dependencies: $wpdb, PRV_Table_Manager, PRV_Cost_Ledger, PRV_Config,
* PRV_Model_Registry, PRV_Config_Version.
*
* @see interface-prv-data-collector.php — Interface this implements.
* @see class-prv-ai-visibility-panel.php — Renders the data returned here.
* @see class-prv-model-registry.php — Source of per-model health data.
* @see class-prv-config-version.php — Source of config-version records.
* @see CONTEXT.md — Score formula + glossary.
* @see ARCHITECTURE.md — §Storage, §Score.
* @package PrVision
*/
class PRV_Ai_Visibility_Collector implements PRV_Data_Collector {
/**
* Budget ledger for MTD cost lookup.
*
* @var PRV_Cost_Ledger
*/
private PRV_Cost_Ledger $ledger;
/**
* Constructor.
*
* @param PRV_Cost_Ledger|null $ledger Injected for testing.
*/
public function __construct( ?PRV_Cost_Ledger $ledger = null ) {
$this->ledger = $ledger ?? new PRV_Cost_Ledger();
}
/**
* {@inheritDoc}
*
* @return array{
* trendline: array<int, array{run_id: string, captured_at: string, score: float, config_version: int|null}>,
* standings: array<string, array{label: string, cited: bool, our_position: int|null, top_domains: string[], model_count: int}>,
* last_run_at: string|null,
* mtd_cost_usd: float,
* monthly_cap_usd: float,
* last_run_counts: array<string, array{health_status: string, health_probed: int, health_errors: int}>,
* config_versions: array<int, array<string, mixed>>,
* }
*/
public function collect(): array {
global $wpdb;
$table = PRV_Table_Manager::get_table_name();
// ── Trendline: one score per run_id, with config_version ───────
// phpcs:disable WordPress.DB.PreparedSQL.InterpolatedNotPrepared, WordPress.DB.DirectDatabaseQuery.DirectQuery, WordPress.DB.DirectDatabaseQuery.NoCaching
$run_rows = $wpdb->get_results(
"SELECT run_id, MIN(captured_at) AS captured_at,
SUM(cited) AS cited_count,
COUNT(*) AS total_count,
SUM(CASE WHEN our_position IS NOT NULL THEN 1.0/our_position ELSE 0 END) AS position_sum,
MIN(config_version) AS config_version
FROM {$table}
GROUP BY run_id
ORDER BY MIN(captured_at) ASC
LIMIT 52",
ARRAY_A
);
// phpcs:enable WordPress.DB.PreparedSQL.InterpolatedNotPrepared, WordPress.DB.DirectDatabaseQuery.DirectQuery, WordPress.DB.DirectDatabaseQuery.NoCaching
$trendline = array();
foreach ( (array) $run_rows as $row ) {
$trendline[] = array(
'run_id' => (string) $row['run_id'],
'captured_at' => (string) $row['captured_at'],
'score' => $this->compute_score(
(int) $row['cited_count'],
(int) $row['total_count'],
(float) $row['position_sum']
),
'config_version' => isset( $row['config_version'] ) ? (int) $row['config_version'] : null,
);
}
// ── Standings: latest result per peptide ────────────────────────
// phpcs:disable WordPress.DB.PreparedSQL.InterpolatedNotPrepared, WordPress.DB.DirectDatabaseQuery.DirectQuery, WordPress.DB.DirectDatabaseQuery.NoCaching
$standing_rows = $wpdb->get_results(
"SELECT peptide_slug, peptide_label,
MAX(cited) AS cited,
MIN(CASE WHEN cited=1 THEN our_position ELSE NULL END) AS our_position,
GROUP_CONCAT(DISTINCT source_domains ORDER BY captured_at DESC SEPARATOR '|||') AS domains_json_list,
COUNT(DISTINCT model) AS model_count
FROM {$table}
WHERE run_id = (SELECT run_id FROM {$table} ORDER BY captured_at DESC LIMIT 1)
GROUP BY peptide_slug, peptide_label",
ARRAY_A
);
// phpcs:enable WordPress.DB.PreparedSQL.InterpolatedNotPrepared, WordPress.DB.DirectDatabaseQuery.DirectQuery, WordPress.DB.DirectDatabaseQuery.NoCaching
$standings = array();
foreach ( (array) $standing_rows as $row ) {
$top_domains = $this->extract_top_domains( (string) ( $row['domains_json_list'] ?? '' ) );
$standings[ (string) $row['peptide_slug'] ] = array(
'label' => (string) $row['peptide_label'],
'cited' => (bool) (int) $row['cited'],
'our_position' => null !== $row['our_position'] ? (int) $row['our_position'] : null,
'top_domains' => $top_domains,
'model_count' => (int) $row['model_count'],
);
}
// ── Run metadata ────────────────────────────────────────────────
// phpcs:disable WordPress.DB.PreparedSQL.InterpolatedNotPrepared, WordPress.DB.DirectDatabaseQuery.DirectQuery, WordPress.DB.DirectDatabaseQuery.NoCaching
$last_run = $wpdb->get_var( "SELECT MAX(captured_at) FROM {$table}" );
// phpcs:enable WordPress.DB.PreparedSQL.InterpolatedNotPrepared, WordPress.DB.DirectDatabaseQuery.DirectQuery, WordPress.DB.DirectDatabaseQuery.NoCaching
// ── Per-model health from PRV_Model_Registry ────────────────────
$last_run_counts = $this->collect_model_health();
// ── Config-version records for trendline break annotations ──────
$config_versions = PRV_Config_Version::get_all_versions();
return array(
'trendline' => $trendline,
'standings' => $standings,
'last_run_at' => $last_run ? (string) $last_run : null,
'mtd_cost_usd' => $this->ledger->get_month_to_date_usd(),
'monthly_cap_usd' => PRV_Config::get_monthly_budget_usd(),
'last_run_counts' => $last_run_counts,
'config_versions' => $config_versions,
);
}
/**
* {@inheritDoc}
*/
public function get_key(): string {
return 'ai_visibility';
}
/**
* Compute the visibility score from a run's aggregate stats.
*
* See CONTEXT.md for the full formula explanation.
*
* @param int $cited_count Number of probes where we were cited.
* @param int $total_count Total probes in the run.
* @param float $position_sum Sum of (1/our_position) for cited probes.
*
* @return float Visibility score in range [0, 1].
*/
public function compute_score( int $cited_count, int $total_count, float $position_sum ): float {
if ( 0 === $total_count ) {
return 0.0;
}
$base_score = $cited_count / $total_count;
$position_bonus = $position_sum / $total_count;
return round( ( $base_score + $position_bonus ) / 2.0, 4 );
}
/**
* Collect per-model health data from PRV_Model_Registry for the dashboard pill.
*
* Returns a keyed map of model slug -> health fields. Models with
* health_status='disabled' are included so the pill reflects all registered
* models, not just the active ones.
*
* @return array<string, array{health_status: string, health_probed: int, health_errors: int}>
*/
private function collect_model_health(): array {
$models = PRV_Model_Registry::get_all();
$out = array();
foreach ( $models as $m ) {
$slug = (string) ( $m['slug'] ?? '' );
if ( '' === $slug ) {
continue;
}
$out[ $slug ] = array(
'health_status' => (string) ( $m['health_status'] ?? 'unknown' ),
'health_probed' => (int) ( $m['health_probed'] ?? 0 ),
'health_errors' => (int) ( $m['health_errors'] ?? 0 ),
);
}
return $out;
}
/**
* Extract and deduplicate the top domain names from a GROUP_CONCAT result.
*
* @param string $domains_json_list Pipe-separated JSON arrays from GROUP_CONCAT.
*
* @return string[] Up to 5 top domains.
*/
private function extract_top_domains( string $domains_json_list ): array {
if ( '' === $domains_json_list ) {
return array();
}
$all = array();
$chunks = explode( '|||', $domains_json_list );
foreach ( $chunks as $chunk ) {
$decoded = json_decode( $chunk, true );
if ( is_array( $decoded ) ) {
$all = array_merge( $all, $decoded );
}
}
$counts = array_count_values( $all );
arsort( $counts );
$top = array_slice( array_keys( $counts ), 0, 5 );
return array_values( array_filter( $top, fn( $d ) => PRV_TARGET_DOMAIN !== $d ) );
}
}