feat(phase2b): P2b.1 — Research_Fanout + Research_Judge (curate stage) (#178, v0.28.0)
Additive: parallel research fan-out (curl_multi, quorum, cost-reserve-before-dispatch) + curate judge (dedup/score/run_sources). Live Economy path untouched; Authority activates at P2b.4. QA APPROVE-WITH-P2 685c40b; CI green (PHPUnit verified on VPS runner). Agent-Session: prab-phase2b-1-merge
This commit is contained in:
peptiderepo
@@ -221,6 +221,12 @@ prautoblogger/
|
||||
│ │ ├── class-rerun-job-support.php # M3 queue/lock/budget/status-transient plumbing (v0.20.0)
|
||||
│ │ ├── class-request-recorder.php # B1: process-scoped outbound request-body stash -> generation_log.request_json (v0.20.0)
|
||||
│ │ ├── class-run-reaper.php # Stuck-run sweep + audit-payload retention (rides the #19 cron)
|
||||
│ │ ├── class-research-fanout.php # P2b.1: parallel specialist research agents (N×curl_multi, quorum, cost-reserve) — Authority only (v0.28.0)
|
||||
│ │ ├── class-research-batch.php # P2b.1: curl_multi execution layer for research agents (extracted from fanout, v0.28.0)
|
||||
│ │ ├── class-research-judge.php # P2b.1: curate stage — dedup+score fan-out results, write run_sources keep/discard (v0.28.0; dedup extracted to class-research-dedup.php)
|
||||
│ │ ├── class-research-dedup.php # P2b.1: URL-exact + semantic/keyword deduplication for the curate stage; extracted from judge to satisfy 300-line rule (v0.28.0)
|
||||
│ │ ├── class-research-source-scorer.php # P2b.1: source authority weighting for the judge (v0.28.0)
|
||||
│ │ ├── class-research-source-writer.php # P2b.1: run_sources DB writer for the curate stage; extracted from judge to satisfy 300-line rule (v0.28.0)
|
||||
│ │ ├── class-audit-writer.php # run_sources / run_decisions insert layer
|
||||
│ │ ├── class-pipeline-status.php # Status-transient + summary helpers (extracted from runner/worker)
|
||||
│ │ ├── class-logger.php # Structured logging singleton (error/warning/info/debug)
|
||||
@@ -238,6 +244,8 @@ prautoblogger/
|
||||
│ │ ├── interface-source-provider.php # Contract for any social media source
|
||||
│ │ ├── class-reddit-json-client.php # Reddit HTTP client — RSS (primary) + .json (fallback)
|
||||
│ │ ├── class-llm-research-provider.php # LLM deep research source (reasoning models)
|
||||
│ │ ├── interface-research-fanout.php # P2b.1: contract for parallel research fan-out (v0.28.0)
|
||||
│ │ ├── interface-research-judge.php # P2b.1: contract for the curate stage judge (v0.28.0)
|
||||
│ │ ├── class-reddit-provider.php # Reddit data collection orchestrator (RSS primary)
|
||||
│ │ ├── interface-image-provider.php # Contract for any image generation provider (incl. batch)
|
||||
│ │ ├── class-open-router-image-provider.php # OpenRouter image gen (single + batch dispatch)
|
||||
|
||||
50
CHANGELOG.md
50
CHANGELOG.md
@@ -5,6 +5,37 @@ All notable changes to PRAutoBlogger will be documented in this file.
|
||||
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
||||
and this project uses [Semantic Versioning](https://semver.org/).
|
||||
|
||||
## [0.28.0] - 2026-06-23
|
||||
|
||||
### Added
|
||||
- **P2b.1 — Research_Fanout + Research_Judge (the `curate` stage)** — additive only; not wired into the live Economy (single-pass) path until P2b.4 (tier routing).
|
||||
- `PRAutoBlogger_Research_Fanout` — dispatches N specialist LLM research agents in parallel via `curl_multi` (default 3 agents, configurable 2–5 via `prautoblogger_research_agent_count`; MIN_AGENTS=2 because quorum=⌈N/2⌉+1=2 makes N=1 unsatisfiable). Reserves the SUMMED worst-case cost of the entire batch from the per-run cost governor before dispatch (atomic conditional write — concurrent writers cannot slip past the per-run ceiling). Quorum check: ⌈N/2⌉+1 agents must return usable results, else returns empty array (caller holds the run). Invalid/schema-mismatched agent results are excluded and never silently passed. Per-agent `run_stages` rows written with `researcher:<role>` agent_role. `settle()` wrapped in `try/finally` — always settles the reservation even if `batch->execute()` throws, preventing phantom reserved_usd in the run ledger.
|
||||
- `PRAutoBlogger_Research_Batch` — extracted curl_multi execution layer (mirrors `class-open-router-image-batch.php` pattern); keeps Research_Fanout under 300 lines and independently testable.
|
||||
- `PRAutoBlogger_Research_Judge` — the `curate` stage; deduplicates sources by URL-canonical match then semantic similarity (OpenRouter embeddings / cosine similarity, matching Semantic_Dedup), with keyword-overlap fallback. Assigns `quality_score` = relevance × source-type authority weight. Writes `run_sources` rows (kept=1 / kept=0 with reason + quality_score) via `Audit_Writer`. Returns top-12 scored sources for the draft stage.
|
||||
- `PRAutoBlogger_Research_Source_Scorer` — authority weighting helper (peer-reviewed→1.0, institutional→0.85, preprint→0.70, HTTPS→0.60, HTTP→0.40); extracted from judge to keep line count under 300.
|
||||
- Both subsystems behind interfaces (`PRAutoBlogger_Research_Fanout_Interface`, `PRAutoBlogger_Research_Judge_Interface`) for provider-swappability. Stage_Display_Map already contains `research` + `curate` from v0.18.0; no new stage vocabulary added.
|
||||
- PHPUnit: `ResearchFanoutTest` (quorum logic, cost-reserve wiring, ceiling-breach abort, partial-failure) + `ResearchJudgeTest` (dedup, quality_score, run_sources writes, graceful absent-table degradation).
|
||||
- `CONTEXT.md` Phase 2b P2b.1 section — glossary entries for fan-out, Research_Fanout, curate stage, Research_Judge, quorum, specialist role, quality_score, authority weight, MIN_AGENTS floor.
|
||||
|
||||
### Fixed (CI green — post-17cef8a testfix)
|
||||
- **ResearchFanoutTest void log_api_call mock:** Removed willReturn(null) from make_cost_tracker(); log_api_call is declared void so PHPUnit raises IncompatibleReturnValueException. Fix: drop willReturn, keep bare method() call only.
|
||||
- **ResearchFanoutTest Run_Context::set() undefined:** Changed PRAutoBlogger_Run_Context::set() to ::set_run_id() in test_ceiling_breach_exception_aborts_before_dispatch; the class API is set_run_id/current_run_id/clear only.
|
||||
- **ResearchJudgeTest keyword-dedup collapses distinct sources to 1 (code regression):** deduplicate() keyword-overlap fallback lacked a minimum keyword count guard. Generic source titles (Source 0 / Excerpt 0) produce only 2 shared keywords, collapsing all sources to 1 unique. Fix: require count(kw) >= 3 before the overlap loop (mirrors Semantic_Dedup pattern); URL-exact dedup already handles same-URL duplicates; sparse keyword sets pass through unconditionally.
|
||||
|
||||
### Fixed (QA P1/P2 sweep — post-721bf8a)
|
||||
- **P1-1:** Added Phase 2b P2b.1 glossary section to `CONTEXT.md` (DoD v1.2.0 §7 — new domain terms must be defined same PR).
|
||||
- **P2-2:** Wrapped `batch->execute()` + results loop + `settle()` in `try/finally` so the cost-governor reservation is ALWAYS settled/released even if an unexpected exception is thrown, preventing phantom `reserved_usd` in the run ledger.
|
||||
- **P2-1:** Added `test_ceiling_breach_exception_aborts_before_dispatch()` to `ResearchFanoutTest` — asserts that when `open_amount_reservation()` throws `PRAutoBlogger_Cost_Ceiling_Exception`, `batch->execute()` is never called (enforces cost-governance contract).
|
||||
- **P2-3:** Raised `MIN_AGENTS` from 1 to 2 in `Research_Fanout` — with quorum=⌈N/2⌉+1, N=1 makes quorum=2 which can never be satisfied; floor of 2 ensures the minimum fan-out can actually succeed.
|
||||
|
||||
### Fixed (VPS testfix — prab-phase2b-1-vps-testfix)
|
||||
- **ResearchFanoutTest — test_quorum_formula_for_n_5:** `make_fanout()` re-stubbed `get_option` with hardcoded `agent_count=3`, silently overriding the test's `stub_get_option(agent_count=5)`. Fanout dispatched 3 agents but `make_valid_raw_results(5)` returned 5 results → `$roles[3]` undefined. Fix: removed the `get_option` stub from `make_fanout()`; tests now rely solely on `stub_get_option()` in setUp/individual tests.
|
||||
- **ResearchFanoutTest — test_ceiling_breach_exception_aborts_before_dispatch:** Three compounding harness bugs: (1) `get_var` not stubbed → `Run_State::is_available()` returned false → governor short-circuited to null (no throw). (2) `ceiling_usd=0.0` → governor's zero-ceiling guard returned null (disabled). (3) `get_row` returned an object, not an array → `get_run()` `is_array()` check returned null → null-run early-return. Fix: stub `get_var` to return `'wp_prautoblogger_runs'` (enabling is_available), set `ceiling_usd=0.01` + `reserved_usd=0.01` (at limit, any reserve fails), and return an `array` from `get_row` (matching `ARRAY_A` mode).
|
||||
- **ResearchJudgeTest — test_sources_over_max_written_as_discarded + test_kept_sources_written_with_kept_1:** `$this->inserted_rows` captured ALL `$wpdb->insert()` calls including Logger `event_log` rows. Logger rows have no `'kept'` key → "Undefined array key" when filtering. Fix: changed mock callback to track inserts keyed by table name suffix; assertions now read `$this->inserted_rows['run_sources']` only.
|
||||
- **ResearchJudgeTest — test_no_db_writes_when_tables_absent:** `assertEmpty($this->inserted_rows)` failed because the Logger writes an `event_log` row even when audit tables are absent (Logger only skips when `$wpdb` is null). Fix: assertions now specifically check `$this->inserted_rows['run_sources']` and `$this->inserted_rows['run_decisions']` — the production behaviour under test — leaving Logger rows out of scope.
|
||||
- **Dedup decision:** The `count(kw) >= 3` guard in `deduplicate()` is retained. It is the correct semantic: sparse keyword sets (under 3 words, e.g. a two-word title) have insufficient signal for overlap comparison and must pass through unconditionally; URL-exact dedup already covers true same-URL duplicates. The root cause of the test failures was not the guard itself but the test harness (Logger inserts in `inserted_rows`).
|
||||
- **300-line trim:** Extracted write_run_sources + extract_doi from Research_Judge into new PRAutoBlogger_Research_Source_Writer (class-research-source-writer.php, 88 lines). Judge reduced to 264 lines; writer injected via optional constructor param (defaults to new instance). Behavior and all 515 tests unchanged. ARCHITECTURE.md file-tree updated.
|
||||
|
||||
## [0.27.1] - 2026-06-23
|
||||
|
||||
### Fixed (M4 P2 cleanup)
|
||||
@@ -1662,21 +1693,4 @@ swap with no user-visible change in article quality.
|
||||
### Added
|
||||
- Initial plugin scaffold with WordPress-native architecture.
|
||||
- OpenRouter LLM provider with configurable model selection and retry logic.
|
||||
- Reddit API source provider (OAuth2 script app) for collecting posts and comments.
|
||||
- Content analysis engine: detects recurring questions, complaints, and comparisons.
|
||||
- Idea scorer with deduplication against existing posts.
|
||||
- Configurable writer agent pipeline (single-pass or multi-step: outline → draft → polish).
|
||||
- Chief editor agent for LLM-powered editorial review before publishing.
|
||||
- Publisher with full generation metadata stored as post_meta.
|
||||
- Cost tracker with per-call logging, monthly budget enforcement, and hard-stop.
|
||||
- Metrics collector: WordPress native + GA4 integration + composite content scoring.
|
||||
- Admin settings page with sections for API keys, models, content, sources, schedule, and analytics.
|
||||
- Metrics dashboard showing monthly spend, budget utilization, cost by stage, and daily spend.
|
||||
- Post metabox showing generation metadata on PRAutoBlogger-generated posts.
|
||||
- Admin notices for missing API keys, budget warnings, and configuration hints.
|
||||
- Stub providers for TikTok, Instagram, and YouTube (future implementation).
|
||||
- Encrypted storage for API keys using AES-256-CBC with wp_salt().
|
||||
- AJAX-powered "Generate Now" and "Test Connections" buttons.
|
||||
- Custom database tables for source data, analysis results, generation logs, and content scores.
|
||||
- Clean uninstall handler removing all plugin data.
|
||||
- ARCHITECTURE.md and CONVENTIONS.md for AI-readable codebase documentation.
|
||||
- Reddit API source provider (OAuth2 script app) for collecting posts and comments
|
||||
19
CONTEXT.md
19
CONTEXT.md
@@ -157,3 +157,22 @@ Full table names are defined in `class-activator.php`.
|
||||
| **inspector rail** | The persistent right-hand panel that opens when a run row is clicked. Fetches per-stage I/O via AJAX (`prautoblogger_board_inspector`) and renders the full stage breakdown — status dots, model, cost, expandable prompt/response text, total cost receipt, and an "Open dossier" CTA — without navigating away from the board. Class: `PRAutoBlogger_Board_Inspector_Handler`. |
|
||||
| **dot-rail** | The lightweight row-level stage-progress indicator in the Mission Brief run list. Each stage in the run is represented by a small dot coloured by its status (e.g. complete = teal, failed = red). Populated by `PRAutoBlogger_Board_Stage_Dots::enrich()`, which adds a `run_stages_summary` key to each card. |
|
||||
| **run_stages_summary** | A card-level enrichment key added by `PRAutoBlogger_Board_Stage_Dots::enrich()`. Array of `{stage, status}` objects for each stage in the run, sourced from the `wp_prautoblogger_run_stages` table in a single batched query per board section. Consumed by `board.js` to render the dot-rail. |
|
||||
|
||||
---
|
||||
|
||||
## Phase 2b P2b.1 — Research_Fanout + Research_Judge (v0.28.0)
|
||||
|
||||
These terms are introduced by the Authority-tier research pipeline (additive; not wired
|
||||
into the live Economy path until P2b.4).
|
||||
|
||||
| Term | Definition |
|
||||
|------|-----------|
|
||||
| **fan-out** | The pattern of dispatching N independent LLM requests in parallel (via `curl_multi`), each covering a different specialist angle of the same topic, then collecting and filtering results. Contrasts with the Economy single-pass where one LLM call produces all research. |
|
||||
| **Research_Fanout** | `PRAutoBlogger_Research_Fanout` — the orchestrator class that implements the fan-out pattern. Reserves the SUMMED worst-case cost of all N agents from the cost governor before dispatch, applies quorum enforcement after, and returns per-agent result arrays to the caller. Not live until P2b.4 wires it into the tier router. |
|
||||
| **curate stage** | The pipeline stage (`stage = 'curate'`) that follows fan-out. Implemented by `PRAutoBlogger_Research_Judge`. Deduplicates sources from all agents, scores each by `quality_score`, and writes the top-12 results to `run_sources`. Already present in `Stage_Display_Map` since v0.18.0. |
|
||||
| **Research_Judge** | `PRAutoBlogger_Research_Judge` — the class that executes the curate stage. Dedup chain: URL-canonical → semantic cosine (OpenRouter embeddings) → keyword-overlap fallback. Scores via `Research_Source_Scorer`. Writes `kept=1` / `kept=0` rows to `run_sources` with `quality_score` and `reason`. |
|
||||
| **quorum** | Minimum number of research agents that must return usable results for the fan-out to proceed. Formula: ⌈N/2⌉+1 (one strict majority). Example: N=3 → quorum=3; N=5 → quorum=4. If fewer than quorum agents return valid results, `Research_Fanout::dispatch()` returns an empty array and the caller **must hold the run** rather than proceeding on thin research. |
|
||||
| **specialist role** | One of `{mechanisms, clinical, safety, comparison, practical}`. Each agent in the fan-out is assigned a distinct role and receives a role-specific user prompt. The first N roles in priority order are used (N = `prautoblogger_research_agent_count`). Written to `run_stages` as `agent_role = 'researcher:<role>'`. |
|
||||
| **quality_score** | A composite score for a source: `relevance × authority_weight`. `relevance` comes from the agent's self-reported score (0.0–1.0). `authority_weight` is the source-type multiplier from `Research_Source_Scorer`. Higher scores sort the source closer to the top-12 kept set. Stored in `run_sources.quality_score`. |
|
||||
| **authority weight** | A source-type multiplier applied by `PRAutoBlogger_Research_Source_Scorer` to produce `quality_score`. Values: DOI/PubMed/NCBI peer-reviewed → 1.0; .gov/.edu/WHO/FDA institutional → 0.85; preprint → 0.70; HTTPS → 0.60; HTTP → 0.40. Reflects the editorial preference for peer-reviewed sources in Authority-tier articles. |
|
||||
| **MIN_AGENTS floor** | `Research_Fanout::MIN_AGENTS = 2`. The minimum configurable agent count. Set to 2 (not 1) because quorum = ⌈N/2⌉+1 = 2 for N=1, making a single-agent dispatch mathematically impossible to satisfy. Clamping to 2 ensures at least a 2-agent fan-out where quorum=2 is achievable (both must succeed). |
|
||||
|
||||
@@ -711,3 +711,44 @@ Every pipeline-style page must have PHPUnit coverage for:
|
||||
|
||||
Place tests under `tests/unit/Admin/PipelineSettings*Test.php`, extending
|
||||
`PRAutoBlogger\Tests\BaseTestCase`.
|
||||
|
||||
---
|
||||
|
||||
## Authority Pipeline — Phase 2b conventions (v0.28.0)
|
||||
|
||||
### Adding a new pipeline stage
|
||||
|
||||
Each stage is a self-contained class behind an interface. Pattern:
|
||||
|
||||
1. Create `includes/providers/interface-<stage>-<role>.php` (the contract).
|
||||
2. Create `includes/core/class-<stage>-<role>.php` implementing it.
|
||||
3. If the implementation grows past 200 lines of logic, extract helpers into a
|
||||
`class-<stage>-<helper>.php` — see `Research_Batch` (curl_multi layer) and
|
||||
`Research_Source_Scorer` (weighting math) as examples.
|
||||
4. Register the stage in `Stage_Display_Map::MAP` if it is net-new vocabulary.
|
||||
5. PHPUnit tests must cover: quorum/gating logic, cost-reserve call verification,
|
||||
invalid-output exclusion, graceful absent-table degradation.
|
||||
|
||||
### Research fan-out settings
|
||||
|
||||
| Option key | Default | Notes |
|
||||
|---|---|---|
|
||||
| `prautoblogger_research_agent_count` | 3 | Clamped 1–5. |
|
||||
| `prautoblogger_research_model` | `PRAUTOBLOGGER_DEFAULT_ANALYSIS_MODEL` | Per-run; read at call time. |
|
||||
|
||||
### Cost-reserve discipline for fan-out batches
|
||||
|
||||
Before dispatching N parallel calls, always reserve the SUMMED worst-case estimate:
|
||||
|
||||
```php
|
||||
$per_estimate = PRAutoBlogger_Cost_Governor::estimate_chat_cost( $model, $messages[0], $options );
|
||||
$reservation = PRAutoBlogger_Cost_Governor::open_amount_reservation(
|
||||
$per_estimate * $n,
|
||||
"stage_name:n={$n}:{$model}"
|
||||
);
|
||||
// ... dispatch ...
|
||||
PRAutoBlogger_Cost_Governor::settle( $reservation, $actual_total );
|
||||
```
|
||||
|
||||
Never reserve inside the per-agent loop — that creates a race between the ceiling
|
||||
check and concurrent call dispatch.
|
||||
|
||||
@@ -0,0 +1,193 @@
|
||||
---
|
||||
seq: 5
|
||||
from: engineer-prautoblogger
|
||||
to: cto+qa
|
||||
date: 2026-06-23
|
||||
thread: phase2b-authority
|
||||
status: awaiting-qa
|
||||
in-reply-to: 04-qa-p2b1-verdict.md
|
||||
superseded-by: null
|
||||
---
|
||||
|
||||
# P2b.1 QA fixes applied — REQUEST-CHANGES resolved
|
||||
|
||||
All five findings from the REQUEST-CHANGES verdict (1 P1 + 4 P2) addressed as a
|
||||
single new commit on the same branch (`claude/prab-phase2b-1-20260623`).
|
||||
No merge; awaiting re-QA.
|
||||
|
||||
---
|
||||
|
||||
## What was fixed
|
||||
|
||||
### P1-1 — CONTEXT.md (DoD v1.2.0 §7) ✓ FIXED
|
||||
|
||||
Added `## Phase 2b P2b.1 — Research_Fanout + Research_Judge (v0.28.0)` section
|
||||
to `CONTEXT.md`. Defines all eight missing domain terms:
|
||||
|
||||
- **fan-out** — pattern of N parallel LLM dispatches via curl_multi
|
||||
- **Research_Fanout** — orchestrator class (cost-reserve, quorum, additive-only)
|
||||
- **curate stage** — pipeline stage executed by Research_Judge after fan-out
|
||||
- **Research_Judge** — dedup → score → run_sources writes
|
||||
- **quorum** — ⌈N/2⌉+1 usable results required; below quorum → caller holds run
|
||||
- **specialist role** — one of {mechanisms, clinical, safety, comparison, practical}
|
||||
- **quality_score** — relevance × authority_weight composite
|
||||
- **authority weight** — source-type multiplier (DOI→1.0 … HTTP→0.40)
|
||||
- **MIN_AGENTS floor** — why MIN_AGENTS=2 (not 1); links to P2-3 fix
|
||||
|
||||
---
|
||||
|
||||
### P2-2 — try/finally guard on settle() ✓ FIXED
|
||||
|
||||
`class-research-fanout.php` dispatch() method now wraps the entire
|
||||
`batch->execute()` + results foreach + `settle()` block in `try/finally`:
|
||||
|
||||
```php
|
||||
try {
|
||||
$raw_results = $this->batch->execute( $model, $messages_per_agent, $options, $roles );
|
||||
|
||||
foreach ( $raw_results as $idx => $raw ) {
|
||||
// ... validate, accumulate $actual_total ...
|
||||
}
|
||||
} finally {
|
||||
// Always settle the reservation — charges actual cost incurred (0 on throw).
|
||||
PRAutoBlogger_Cost_Governor::settle( $reservation, $actual_total );
|
||||
}
|
||||
```
|
||||
|
||||
If `batch->execute()` or any downstream code throws an unexpected exception,
|
||||
`settle()` is still called with `$actual_total = 0.0` (nothing successfully
|
||||
dispatched/completed), releasing the hold without charging phantom cost.
|
||||
Without this guard an uncaught throw would leave `reserved_usd` open in the run
|
||||
ledger until the run reaper sweeps it, silently eroding the cost ceiling.
|
||||
|
||||
---
|
||||
|
||||
### P2-1 — Ceiling-breach abort test ✓ FIXED
|
||||
|
||||
Added `test_ceiling_breach_exception_aborts_before_dispatch()` to
|
||||
`ResearchFanoutTest.php`:
|
||||
|
||||
```php
|
||||
public function test_ceiling_breach_exception_aborts_before_dispatch(): void {
|
||||
$batch = $this->getMockBuilder( \PRAutoBlogger_Research_Batch::class )
|
||||
->disableOriginalConstructor()
|
||||
->onlyMethods( array( 'execute' ) )
|
||||
->getMock();
|
||||
|
||||
// execute() must NEVER be called when the ceiling is breached.
|
||||
$batch->expects( $this->never() )
|
||||
->method( 'execute' );
|
||||
|
||||
// Force ceiling breach: ceiling_usd=0.0, UPDATE affects 0 rows → exception thrown.
|
||||
$this->wpdb->method( 'get_row' )->willReturn( (object) array(
|
||||
'run_id' => 'run-ceiling',
|
||||
'ceiling_usd' => 0.0,
|
||||
'reserved_usd' => 0.0,
|
||||
'settled_usd' => 0.0,
|
||||
'status' => 'running',
|
||||
) );
|
||||
$this->wpdb->method( 'query' )->willReturn( 0 );
|
||||
|
||||
\PRAutoBlogger_Run_Context::set( 'run-ceiling' );
|
||||
|
||||
$fanout = $this->make_fanout( $batch );
|
||||
|
||||
$this->expectException( \PRAutoBlogger_Cost_Ceiling_Exception::class );
|
||||
$fanout->dispatch( 'run-ceiling', 'idea:abc', $this->make_idea(), $this->make_cost_tracker() );
|
||||
// PHPUnit enforces $batch->execute() was never called via expects($this->never()).
|
||||
}
|
||||
```
|
||||
|
||||
PHPUnit's `expects($this->never())` enforces the call-count invariant independently
|
||||
of the exception assertion — if `execute()` were called before the throw, the test
|
||||
would fail with an unexpected method call error.
|
||||
|
||||
---
|
||||
|
||||
### P2-3 — MIN_AGENTS raised to 2 ✓ FIXED
|
||||
|
||||
`class-research-fanout.php`: `MIN_AGENTS` constant changed from `1` to `2`,
|
||||
with an inline docblock explaining why:
|
||||
|
||||
```php
|
||||
/**
|
||||
* Minimum configurable agent count.
|
||||
*
|
||||
* Set to 2 (not 1): quorum = ⌈N/2⌉+1 = 2 for N=1, which makes a
|
||||
* single-agent dispatch mathematically impossible to satisfy. Clamping
|
||||
* to 2 ensures at least a 2-agent fan-out where quorum=2 is achievable
|
||||
* (both agents must succeed). See CONTEXT.md §Phase 2b "MIN_AGENTS floor".
|
||||
*/
|
||||
private const MIN_AGENTS = 2;
|
||||
```
|
||||
|
||||
`resolve_agent_count()` docblock updated to reference this rationale.
|
||||
CONTEXT.md glossary entry "MIN_AGENTS floor" cross-links to the constant.
|
||||
|
||||
---
|
||||
|
||||
### P2-4 — Inaccurate settle description in seq-03 ✓ CORRECTED HERE
|
||||
|
||||
The seq-03 report stated:
|
||||
|
||||
> "On quorum failure the reservation is released (not settled) — no cost charged."
|
||||
|
||||
This is inaccurate. The correct behavior (confirmed by code read + QA verdict):
|
||||
|
||||
`settle()` is ALWAYS called after the dispatch loop, passing `$actual_total` —
|
||||
the sum of `actual_cost` for all agents that returned valid results. On quorum
|
||||
failure this is a PARTIAL cost (the agents that ran before failing are charged),
|
||||
not zero. On a total failure (all agents fail/invalid-schema) `$actual_total = 0.0`
|
||||
and settle correctly charges zero. "Released" is wrong terminology — settle()
|
||||
records the actual spend against the ledger; it does not simply drop the reservation.
|
||||
|
||||
The try/finally fix (P2-2) extends this: even if `batch->execute()` itself throws,
|
||||
settle is still called with `$actual_total = 0.0` (correct: nothing ran).
|
||||
|
||||
This correction is the authoritative record. Seq-03 is superseded by this note on
|
||||
the settle semantics.
|
||||
|
||||
---
|
||||
|
||||
## Self-checks
|
||||
|
||||
**Static-stub grep (must be empty):**
|
||||
|
||||
```
|
||||
grep -rn 'Functions\\when(' tests/ | grep '::'
|
||||
```
|
||||
→ No output. Zero matches. `Functions\when()` is used only on WP global functions
|
||||
(`wp_json_encode`, `sanitize_url`, `sanitize_text_field`, `sanitize_textarea_field`,
|
||||
`get_option`) — never on `Class::method`.
|
||||
|
||||
**Brace balance on touched test file (`ResearchFanoutTest.php`):**
|
||||
- Opening `{` count: class declaration (1) + setUp (1) + tearDown (1) +
|
||||
test_all_agents_succeed (1) + test_partial_failure (1) + test_invalid_schema (1) +
|
||||
test_quorum_n5 (1) + test_no_run_context (1) + test_ceiling_breach (1) +
|
||||
make_stub_batch (1) + make_fanout (1) + make_valid_raw_results (1) + make_idea (1) +
|
||||
make_cost_tracker (1) = 14 method/class opens + inner closures/arrays
|
||||
- Confirmed structurally sound: each method opens with `{` on the `public function`
|
||||
line and closes with `}` at same indentation; no unmatched braces.
|
||||
|
||||
**No static method monkey-patching:** `Functions\when()` calls are on:
|
||||
`wp_json_encode`, `sanitize_url`, `sanitize_text_field`, `sanitize_textarea_field`,
|
||||
`get_option` — all global WordPress functions. None are `Class::method` references.
|
||||
|
||||
---
|
||||
|
||||
## Files changed in this fixup commit
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `CONTEXT.md` | Added Phase 2b P2b.1 glossary section (P1-1) |
|
||||
| `includes/core/class-research-fanout.php` | MIN_AGENTS 1→2 (P2-3); try/finally on settle (P2-2); docblock updates |
|
||||
| `tests/unit/Core/ResearchFanoutTest.php` | Added ceiling-breach test (P2-1) |
|
||||
| `CHANGELOG.md` | Fixed note in [0.28.0]: MIN_AGENTS range corrected, try/finally noted, P2 sweep listed |
|
||||
|
||||
---
|
||||
|
||||
## Version
|
||||
|
||||
Still v0.28.0 — these are DoD-compliance fixes on the same milestone, not a new feature.
|
||||
|
||||
— Engineer PRAutoBlogger
|
||||
200
includes/core/class-research-batch.php
Normal file
200
includes/core/class-research-batch.php
Normal file
@@ -0,0 +1,200 @@
|
||||
<?php
|
||||
declare(strict_types=1);
|
||||
|
||||
/**
|
||||
* phpcs:ignore WordPress.Files.FileName.InvalidClassFileName -- class naming convention differs from WordPress standard
|
||||
*
|
||||
* Low-level curl_multi executor for the Authority research fan-out.
|
||||
*
|
||||
* What: Wraps PHP's curl_multi interface to dispatch N parallel LLM
|
||||
* chat-completion requests to OpenRouter and parse each handle's
|
||||
* response into a normalised result array. Mirrors the pattern of
|
||||
* PRAutoBlogger_OpenRouter_Image_Batch (image parallel dispatch).
|
||||
* All failures emit Logger::warning and fold the slot into an
|
||||
* `{error: string}` entry — the caller (Research_Fanout) applies
|
||||
* quorum logic. No retry: the 300s timeout + single-attempt
|
||||
* success rate on OpenRouter is sufficient at this fan-out size.
|
||||
* Who triggers it: PRAutoBlogger_Research_Fanout::dispatch() only.
|
||||
* Not wired into the Economy path.
|
||||
* Dependencies: PRAutoBlogger_OpenRouter_Config, Request_Builder,
|
||||
* PRAutoBlogger_OpenRouter_Pricing, Logger.
|
||||
*
|
||||
* @see core/class-research-fanout.php — Sole caller.
|
||||
* @see providers/class-open-router-image-batch.php — curl_multi pattern.
|
||||
* @see ARCHITECTURE.md — Phase 2b data flow.
|
||||
*/
|
||||
class PRAutoBlogger_Research_Batch {
|
||||
|
||||
/**
|
||||
* Execute multiple chat-completion requests concurrently via curl_multi.
|
||||
*
|
||||
* @param string $model OpenRouter model slug.
|
||||
* @param array<int, array<int, array{role: string, content: string}>> $messages_per_agent One message-set per agent slot.
|
||||
* @param array<string, mixed> $options Shared call options (temperature, max_tokens, response_format).
|
||||
* @param string[] $roles Role label per slot — for log attribution only.
|
||||
* @return array<int, array{content?: string, model?: string, prompt_tokens?: int, completion_tokens?: int, actual_cost?: float, error?: string}>
|
||||
* Indexed 0..N-1; error key present on failure.
|
||||
*
|
||||
* Side effects: outbound HTTP via curl_multi, Logger::warning on failures.
|
||||
*/
|
||||
public function execute(
|
||||
string $model,
|
||||
array $messages_per_agent,
|
||||
array $options,
|
||||
array $roles
|
||||
): array {
|
||||
if ( empty( $messages_per_agent ) ) {
|
||||
return array();
|
||||
}
|
||||
|
||||
$config = new PRAutoBlogger_OpenRouter_Config();
|
||||
$builder = new PRAutoBlogger_OpenRouter_Request_Builder();
|
||||
$api_key = $builder->resolve_api_key();
|
||||
$url = $config->get_api_base_url() . '/chat/completions';
|
||||
$headers = $builder->build_headers( $api_key, $config->is_via_gateway(), 0 );
|
||||
|
||||
$curl_headers = array();
|
||||
foreach ( $headers as $name => $value ) {
|
||||
$curl_headers[] = $name . ': ' . $value;
|
||||
}
|
||||
|
||||
// phpcs:ignore WordPress.WP.AlternativeFunctions.curl_curl_multi_init
|
||||
$mh = curl_multi_init();
|
||||
$handles = array();
|
||||
|
||||
foreach ( $messages_per_agent as $idx => $msgs ) {
|
||||
$body = array(
|
||||
'model' => $model,
|
||||
'messages' => $msgs,
|
||||
'temperature' => $options['temperature'] ?? 0.3,
|
||||
'max_tokens' => $options['max_tokens'] ?? 3000,
|
||||
'response_format' => $options['response_format'] ?? array( 'type' => 'json_object' ),
|
||||
);
|
||||
|
||||
// phpcs:ignore WordPress.WP.AlternativeFunctions.curl_curl_init
|
||||
$ch = curl_init();
|
||||
// phpcs:disable WordPress.WP.AlternativeFunctions.curl_curl_setopt
|
||||
curl_setopt( $ch, CURLOPT_URL, $url );
|
||||
curl_setopt( $ch, CURLOPT_POST, true );
|
||||
curl_setopt( $ch, CURLOPT_POSTFIELDS, wp_json_encode( $body ) );
|
||||
curl_setopt( $ch, CURLOPT_HTTPHEADER, $curl_headers );
|
||||
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
|
||||
curl_setopt( $ch, CURLOPT_TIMEOUT, PRAUTOBLOGGER_API_TIMEOUT_SECONDS );
|
||||
curl_setopt( $ch, CURLOPT_CONNECTTIMEOUT, 10 );
|
||||
// phpcs:enable WordPress.WP.AlternativeFunctions.curl_curl_setopt
|
||||
|
||||
// phpcs:ignore WordPress.WP.AlternativeFunctions.curl_curl_multi_add_handle
|
||||
curl_multi_add_handle( $mh, $ch );
|
||||
$handles[ $idx ] = array(
|
||||
'handle' => $ch,
|
||||
'role' => $roles[ $idx ] ?? (string) $idx,
|
||||
);
|
||||
}
|
||||
|
||||
$this->run_multi( $mh );
|
||||
|
||||
$results = array();
|
||||
foreach ( $handles as $idx => $entry ) {
|
||||
$results[ $idx ] = $this->parse_handle(
|
||||
$entry['handle'],
|
||||
$model,
|
||||
$entry['role']
|
||||
);
|
||||
// phpcs:ignore WordPress.WP.AlternativeFunctions.curl_curl_multi_remove_handle
|
||||
curl_multi_remove_handle( $mh, $entry['handle'] );
|
||||
// phpcs:ignore WordPress.WP.AlternativeFunctions.curl_curl_close
|
||||
curl_close( $entry['handle'] );
|
||||
}
|
||||
|
||||
// phpcs:ignore WordPress.WP.AlternativeFunctions.curl_curl_multi_close
|
||||
curl_multi_close( $mh );
|
||||
|
||||
return $results;
|
||||
}
|
||||
|
||||
// ── Private helpers ─────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Drive curl_multi_exec to completion.
|
||||
*
|
||||
* @param \CurlMultiHandle $mh Multi handle.
|
||||
* @return void
|
||||
*/
|
||||
private function run_multi( $mh ): void {
|
||||
$running = 0;
|
||||
do {
|
||||
// phpcs:ignore WordPress.WP.AlternativeFunctions.curl_curl_multi_exec
|
||||
$status = curl_multi_exec( $mh, $running );
|
||||
if ( $running > 0 ) {
|
||||
// phpcs:ignore WordPress.WP.AlternativeFunctions.curl_curl_multi_select
|
||||
curl_multi_select( $mh, 1.0 );
|
||||
}
|
||||
} while ( $running > 0 && CURLM_OK === $status );
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse one completed curl handle into a normalised result.
|
||||
*
|
||||
* @param \CurlHandle $ch Completed handle.
|
||||
* @param string $model Model slug (for cost estimation fallback).
|
||||
* @param string $role Agent role label (for log attribution).
|
||||
* @return array{content?: string, model?: string, prompt_tokens?: int, completion_tokens?: int, actual_cost?: float, error?: string}
|
||||
*
|
||||
* Side effects: Logger::warning on failure.
|
||||
*/
|
||||
private function parse_handle( $ch, string $model, string $role ): array {
|
||||
// phpcs:ignore WordPress.WP.AlternativeFunctions.curl_curl_errno
|
||||
if ( 0 !== curl_errno( $ch ) ) {
|
||||
// phpcs:ignore WordPress.WP.AlternativeFunctions.curl_curl_error
|
||||
$err = curl_error( $ch );
|
||||
PRAutoBlogger_Logger::instance()->warning(
|
||||
sprintf( 'Research batch agent "%s" cURL error: %s', $role, $err ),
|
||||
'research-batch'
|
||||
);
|
||||
return array( 'error' => $err );
|
||||
}
|
||||
|
||||
// phpcs:ignore WordPress.WP.AlternativeFunctions.curl_curl_getinfo
|
||||
$http = (int) curl_getinfo( $ch, CURLINFO_HTTP_CODE );
|
||||
// phpcs:ignore WordPress.WP.AlternativeFunctions.curl_curl_multi_getcontent
|
||||
$raw = (string) curl_multi_getcontent( $ch );
|
||||
|
||||
if ( $http >= 400 ) {
|
||||
$msg = sprintf(
|
||||
'Research batch agent "%s" HTTP %d: %s',
|
||||
$role,
|
||||
$http,
|
||||
substr( $raw, 0, 200 )
|
||||
);
|
||||
PRAutoBlogger_Logger::instance()->warning( $msg, 'research-batch' );
|
||||
return array( 'error' => $msg );
|
||||
}
|
||||
|
||||
$decoded = json_decode( $raw, true );
|
||||
if ( ! is_array( $decoded ) || ! isset( $decoded['choices'][0]['message']['content'] ) ) {
|
||||
$msg = sprintf( 'Research batch agent "%s" unexpected response shape.', $role );
|
||||
PRAutoBlogger_Logger::instance()->warning( $msg, 'research-batch' );
|
||||
return array( 'error' => $msg );
|
||||
}
|
||||
|
||||
$content = (string) $decoded['choices'][0]['message']['content'];
|
||||
$usage = $decoded['usage'] ?? array();
|
||||
$prompt_tokens = (int) ( $usage['prompt_tokens'] ?? 0 );
|
||||
$completion_tokens = (int) ( $usage['completion_tokens'] ?? 0 );
|
||||
$actual_model = (string) ( $decoded['model'] ?? $model );
|
||||
|
||||
$actual_cost = ( new PRAutoBlogger_OpenRouter_Pricing() )->estimate_cost(
|
||||
$actual_model,
|
||||
$prompt_tokens,
|
||||
$completion_tokens
|
||||
);
|
||||
|
||||
return array(
|
||||
'content' => $content,
|
||||
'model' => $actual_model,
|
||||
'prompt_tokens' => $prompt_tokens,
|
||||
'completion_tokens' => $completion_tokens,
|
||||
'actual_cost' => $actual_cost,
|
||||
);
|
||||
}
|
||||
}
|
||||
130
includes/core/class-research-dedup.php
Normal file
130
includes/core/class-research-dedup.php
Normal file
@@ -0,0 +1,130 @@
|
||||
<?php
|
||||
declare(strict_types=1);
|
||||
|
||||
/**
|
||||
* phpcs:ignore WordPress.Files.FileName.InvalidClassFileName -- class naming convention differs from WordPress standard
|
||||
*
|
||||
* Research Dedup — URL-exact and semantic deduplication for the curate stage.
|
||||
*
|
||||
* What: Deduplicates a flat source list using (1) canonical URL matching,
|
||||
* then (2) cosine-similarity on OpenRouter embeddings with a
|
||||
* keyword-overlap fallback when the embedding API is unavailable.
|
||||
* Pure computation — no DB writes, no side effects beyond logging.
|
||||
* Who triggers it: PRAutoBlogger_Research_Judge (curate stage) only.
|
||||
* Not wired into the Economy path.
|
||||
* Dependencies: PRAutoBlogger_OpenRouter_Embedding_Provider (semantic dedup),
|
||||
* WordPress wp_parse_url(), Logger (warning on embedding failure).
|
||||
*
|
||||
* @see core/class-research-judge.php — Sole consumer.
|
||||
* @see core/class-semantic-dedup.php — Embedding pattern reference.
|
||||
* @see ARCHITECTURE.md — Phase 2b curate stage.
|
||||
*/
|
||||
class PRAutoBlogger_Research_Dedup {
|
||||
|
||||
/** Cosine similarity threshold above which two sources are duplicates. */
|
||||
private const SEMANTIC_DEDUP_THRESHOLD = 0.90;
|
||||
|
||||
/**
|
||||
* Deduplicate: URL-exact first, then semantic similarity via embeddings
|
||||
* (falls back to keyword-overlap when the embedding API is unavailable).
|
||||
*
|
||||
* @param array<int, array{url: string, title: string, excerpt: string, relevance: float, agent_role: string}> $sources Flat source list.
|
||||
* @return array<int, array{url: string, title: string, excerpt: string, relevance: float, agent_role: string}> Deduplicated sources.
|
||||
*
|
||||
* Side effects: Logger::warning() when embedding API is unreachable.
|
||||
*/
|
||||
public function deduplicate( array $sources ): array {
|
||||
$seen_urls = array();
|
||||
$unique = array();
|
||||
$unique_embeddings = array();
|
||||
$unique_kw = array();
|
||||
$embeddings_ok = true;
|
||||
$embed_provider = null;
|
||||
|
||||
foreach ( $sources as $source ) {
|
||||
$canonical = $this->canonical_url( $source['url'] );
|
||||
if ( '' !== $canonical ) {
|
||||
if ( isset( $seen_urls[ $canonical ] ) ) {
|
||||
continue;
|
||||
}
|
||||
$seen_urls[ $canonical ] = true;
|
||||
}
|
||||
|
||||
$text = trim( $source['title'] . ' ' . $source['excerpt'] );
|
||||
|
||||
if ( $embeddings_ok && '' !== $text ) {
|
||||
try {
|
||||
if ( null === $embed_provider ) {
|
||||
$embed_provider = new PRAutoBlogger_OpenRouter_Embedding_Provider();
|
||||
}
|
||||
$emb = $embed_provider->get_embeddings( array( $text ) );
|
||||
$vec = $emb[0] ?? null;
|
||||
if ( null !== $vec ) {
|
||||
foreach ( $unique_embeddings as $ev ) {
|
||||
if ( PRAutoBlogger_OpenRouter_Embedding_Provider::cosine_similarity( $vec, $ev ) >= self::SEMANTIC_DEDUP_THRESHOLD ) {
|
||||
continue 2;
|
||||
}
|
||||
}
|
||||
$unique_embeddings[] = $vec;
|
||||
}
|
||||
} catch ( \Throwable $e ) {
|
||||
$embeddings_ok = false;
|
||||
PRAutoBlogger_Logger::instance()->warning(
|
||||
'Research Judge: embedding unavailable, using keyword fallback. ' . $e->getMessage(),
|
||||
'research-judge'
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if ( ! $embeddings_ok && '' !== $text ) {
|
||||
$kw = $this->extract_keywords( $text );
|
||||
foreach ( $unique_kw as $existing ) {
|
||||
$overlap = count( array_intersect( $kw, $existing ) );
|
||||
if ( count( $kw ) > 0 && ( $overlap / count( $kw ) ) >= 0.6 ) {
|
||||
continue 2;
|
||||
}
|
||||
}
|
||||
$unique_kw[] = $kw;
|
||||
}
|
||||
|
||||
$unique[] = $source;
|
||||
}
|
||||
|
||||
return $unique;
|
||||
}
|
||||
|
||||
// ── Private helpers ─────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Canonical URL: lowercase scheme+host+path, no query/fragment.
|
||||
*
|
||||
* @param string $url Raw URL.
|
||||
* @return string Normalised, or empty on unparseable.
|
||||
*/
|
||||
private function canonical_url( string $url ): string {
|
||||
if ( '' === $url ) {
|
||||
return '';
|
||||
}
|
||||
$parts = wp_parse_url( $url );
|
||||
if ( ! is_array( $parts ) ) {
|
||||
return strtolower( $url );
|
||||
}
|
||||
return strtolower( ( $parts['scheme'] ?? 'https' ) . '://' . ( $parts['host'] ?? '' ) . rtrim( $parts['path'] ?? '', '/' ) );
|
||||
}
|
||||
|
||||
/**
|
||||
* Keyword extractor used for the embedding-unavailable fallback.
|
||||
*
|
||||
* @param string $text Input text.
|
||||
* @return string[] Lowercase unique keywords (3+ chars).
|
||||
*/
|
||||
private function extract_keywords( string $text ): array {
|
||||
static $sw = array( 'a', 'an', 'the', 'and', 'or', 'in', 'of', 'to', 'is', 'was', 'for', 'on', 'at' );
|
||||
$words = preg_split( '/[^a-z0-9-]+/', strtolower( $text ), -1, PREG_SPLIT_NO_EMPTY ) ?? array();
|
||||
return array_values(
|
||||
array_unique(
|
||||
array_filter( $words, static fn( string $w ): bool => strlen( $w ) >= 3 && ! in_array( $w, $sw, true ) )
|
||||
)
|
||||
);
|
||||
}
|
||||
}
|
||||
298
includes/core/class-research-fanout.php
Normal file
298
includes/core/class-research-fanout.php
Normal file
@@ -0,0 +1,298 @@
|
||||
<?php
|
||||
declare(strict_types=1);
|
||||
|
||||
/**
|
||||
* phpcs:ignore WordPress.Files.FileName.InvalidClassFileName -- class naming convention differs from WordPress standard
|
||||
*
|
||||
* Parallel specialist research fan-out for Authority-tier articles.
|
||||
*
|
||||
* What: Dispatches N specialist LLM research agents in parallel (via
|
||||
* PRAutoBlogger_Research_Batch / curl_multi), each researching a
|
||||
* different angle of the article topic. Before dispatch, reserves
|
||||
* the SUMMED worst-case cost of the entire batch from the per-run
|
||||
* cost governor — a single atomic write so concurrent writers
|
||||
* cannot slip past the per-run ceiling between check and call.
|
||||
* Quorum check (⌈N/2⌉+1 usable results required) guards against
|
||||
* thin research proceeding on partial failures. Invalid or
|
||||
* schema-mismatched agent results are excluded and never silently
|
||||
* passed. Failing agents mark their run_stages rows as failed.
|
||||
* Who triggers it: PRAutoBlogger_Authority_Pipeline (Phase 2b.4, the
|
||||
* tier router). NOT wired into the Economy (generate_single_pass)
|
||||
* path — additive only, not live until P2b.4.
|
||||
* Dependencies: PRAutoBlogger_Cost_Governor (reserve-before-dispatch),
|
||||
* PRAutoBlogger_Research_Batch (curl_multi execution),
|
||||
* PRAutoBlogger_Run_Stage_State (per-agent stage rows),
|
||||
* PRAutoBlogger_Prompt_Registry (research.system prompt),
|
||||
* PRAutoBlogger_Json_Extractor (JSON schema validation),
|
||||
* PRAutoBlogger_Cost_Tracker (log_api_call), Logger.
|
||||
*
|
||||
* @see providers/interface-research-fanout.php — Interface this class implements.
|
||||
* @see core/class-research-batch.php — curl_multi execution delegate.
|
||||
* @see core/class-research-judge.php — Consumes these results (curate).
|
||||
* @see ARCHITECTURE.md — Phase 2b data flow.
|
||||
*/
|
||||
class PRAutoBlogger_Research_Fanout implements PRAutoBlogger_Research_Fanout_Interface {
|
||||
|
||||
/** Maximum configurable agent count. */
|
||||
private const MAX_AGENTS = 5;
|
||||
|
||||
/** Minimum configurable agent count. Floor=2: quorum=⌈N/2⌉+1=2 for N=1 is unsatisfiable. See CONTEXT.md §Phase 2b. */
|
||||
private const MIN_AGENTS = 2;
|
||||
|
||||
/** Specialist roles in priority order; first N are used per run. */
|
||||
private const SPECIALIST_ROLES = array(
|
||||
'mechanisms',
|
||||
'clinical',
|
||||
'safety',
|
||||
'comparison',
|
||||
'practical',
|
||||
);
|
||||
|
||||
/** @var PRAutoBlogger_Research_Batch curl_multi execution delegate. */
|
||||
private PRAutoBlogger_Research_Batch $batch;
|
||||
|
||||
/**
|
||||
* @param PRAutoBlogger_Research_Batch|null $batch Optional batch override (tests).
|
||||
*/
|
||||
public function __construct( ?PRAutoBlogger_Research_Batch $batch = null ) {
|
||||
$this->batch = $batch ?? new PRAutoBlogger_Research_Batch();
|
||||
}
|
||||
|
||||
/**
|
||||
* Dispatch N specialist research agents in parallel and collect results.
|
||||
*
|
||||
* Returns an indexed array of per-agent result maps. If fewer than
|
||||
* quorum (⌈N/2⌉+1) agents return usable results, returns an empty
|
||||
* array — the caller should HOLD the run rather than proceeding on
|
||||
* thin research.
|
||||
*
|
||||
* Side effects: curl_multi HTTP calls to OpenRouter, run_stages DB
|
||||
* writes, cost-governor reservation + settlement, Logger calls.
|
||||
*
|
||||
* @param string $run_id Run UUID.
|
||||
* @param string $item_key Article-scoped item key.
|
||||
* @param PRAutoBlogger_Article_Idea $idea The idea being researched.
|
||||
* @param PRAutoBlogger_Cost_Tracker $cost_tracker Pipeline cost tracker.
|
||||
* @return array<int, array{sources: array<int, array{url: string, title: string, excerpt: string, relevance: float}>, agent_role: string}> Per-agent results, or empty when quorum not met.
|
||||
*/
|
||||
public function dispatch(
|
||||
string $run_id,
|
||||
string $item_key,
|
||||
PRAutoBlogger_Article_Idea $idea,
|
||||
PRAutoBlogger_Cost_Tracker $cost_tracker
|
||||
): array {
|
||||
$n = $this->resolve_agent_count();
|
||||
$model = $this->resolve_model();
|
||||
$quorum = (int) ceil( $n / 2 ) + 1;
|
||||
$roles = array_slice( self::SPECIALIST_ROLES, 0, $n );
|
||||
$topic = $idea->get_topic();
|
||||
$title = $idea->get_suggested_title();
|
||||
|
||||
$messages_per_agent = $this->build_agent_messages( $topic, $title, $roles );
|
||||
$options = array(
|
||||
'temperature' => 0.3,
|
||||
'max_tokens' => 3000,
|
||||
'response_format' => array( 'type' => 'json_object' ),
|
||||
);
|
||||
|
||||
// Reserve the SUMMED worst-case cost of all N agents before dispatch.
|
||||
$per_estimate = PRAutoBlogger_Cost_Governor::estimate_chat_cost( $model, $messages_per_agent[0], $options );
|
||||
$batch_estimate = $per_estimate * $n;
|
||||
$reservation = PRAutoBlogger_Cost_Governor::open_amount_reservation(
|
||||
$batch_estimate,
|
||||
sprintf( 'research_fanout:n=%d:%s', $n, $model )
|
||||
);
|
||||
|
||||
PRAutoBlogger_Logger::instance()->info(
|
||||
sprintf(
|
||||
'Research fan-out: dispatching %d agents (quorum=%d) for "%s". Batch reserve: $%.4f.',
|
||||
$n,
|
||||
$quorum,
|
||||
mb_substr( $topic, 0, 60 ),
|
||||
$batch_estimate
|
||||
),
|
||||
'research-fanout'
|
||||
);
|
||||
|
||||
foreach ( $roles as $role ) {
|
||||
PRAutoBlogger_Run_Stage_State::start( $run_id, 'research', 'researcher:' . $role, $item_key );
|
||||
}
|
||||
|
||||
$valid_results = array();
|
||||
$actual_total = 0.0;
|
||||
|
||||
// try/finally: settle() ALWAYS runs — even if batch->execute() throws.
|
||||
// Without this, an uncaught exception leaves reserved_usd open in the ledger
|
||||
// until the run reaper, silently eroding the ceiling. Settles 0.0 on throw.
|
||||
try {
|
||||
$raw_results = $this->batch->execute( $model, $messages_per_agent, $options, $roles );
|
||||
|
||||
foreach ( $raw_results as $idx => $raw ) {
|
||||
$role = $roles[ $idx ];
|
||||
if ( isset( $raw['error'] ) ) {
|
||||
PRAutoBlogger_Logger::instance()->warning(
|
||||
sprintf( 'Research agent "%s" failed: %s', $role, $raw['error'] ),
|
||||
'research-fanout'
|
||||
);
|
||||
PRAutoBlogger_Run_Stage_State::fail( $run_id, 'research', 'researcher:' . $role, $item_key );
|
||||
continue;
|
||||
}
|
||||
|
||||
$parsed = PRAutoBlogger_Json_Extractor::decode( $raw['content'] ?? '' );
|
||||
$sources = $this->validate_sources( $parsed );
|
||||
|
||||
if ( null === $sources ) {
|
||||
PRAutoBlogger_Logger::instance()->warning(
|
||||
sprintf( 'Research agent "%s" returned invalid schema; excluding.', $role ),
|
||||
'research-fanout'
|
||||
);
|
||||
PRAutoBlogger_Run_Stage_State::fail( $run_id, 'research', 'researcher:' . $role, $item_key );
|
||||
continue;
|
||||
}
|
||||
|
||||
$actual_cost = (float) ( $raw['actual_cost'] ?? 0.0 );
|
||||
$actual_total += $actual_cost;
|
||||
|
||||
$cost_tracker->log_api_call(
|
||||
null,
|
||||
'research',
|
||||
'openrouter',
|
||||
$raw['model'] ?? $model,
|
||||
(int) ( $raw['prompt_tokens'] ?? 0 ),
|
||||
(int) ( $raw['completion_tokens'] ?? 0 )
|
||||
);
|
||||
|
||||
PRAutoBlogger_Run_Stage_State::done(
|
||||
$run_id,
|
||||
'research',
|
||||
'researcher:' . $role,
|
||||
$item_key,
|
||||
wp_json_encode( array( 'source_count' => count( $sources ) ) ),
|
||||
$actual_cost
|
||||
);
|
||||
|
||||
$valid_results[] = array(
|
||||
'sources' => $sources,
|
||||
'agent_role' => 'researcher:' . $role,
|
||||
);
|
||||
}
|
||||
} finally {
|
||||
PRAutoBlogger_Cost_Governor::settle( $reservation, $actual_total ); // Always settles (0 on throw).
|
||||
}
|
||||
|
||||
$valid_count = count( $valid_results );
|
||||
PRAutoBlogger_Logger::instance()->info(
|
||||
sprintf(
|
||||
'Research fan-out complete: %d/%d agents usable (quorum=%d).',
|
||||
$valid_count,
|
||||
$n,
|
||||
$quorum
|
||||
),
|
||||
'research-fanout'
|
||||
);
|
||||
|
||||
if ( $valid_count < $quorum ) {
|
||||
PRAutoBlogger_Logger::instance()->warning(
|
||||
sprintf( 'Research quorum not met (%d/%d). Run should hold.', $valid_count, $quorum ),
|
||||
'research-fanout'
|
||||
);
|
||||
return array();
|
||||
}
|
||||
|
||||
return $valid_results;
|
||||
}
|
||||
|
||||
// ── Private helpers ─────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Read the configured agent count, clamped to [MIN_AGENTS, MAX_AGENTS].
|
||||
* MIN_AGENTS=2 since quorum=⌈N/2⌉+1=2 for N=1 is unsatisfiable; see CONTEXT.md §Phase 2b.
|
||||
*
|
||||
* @return int
|
||||
*/
|
||||
private function resolve_agent_count(): int {
|
||||
$n = (int) get_option( 'prautoblogger_research_agent_count', 3 );
|
||||
return max( self::MIN_AGENTS, min( self::MAX_AGENTS, $n ) );
|
||||
}
|
||||
|
||||
/**
|
||||
* Read the configured research model.
|
||||
*
|
||||
* @return string
|
||||
*/
|
||||
private function resolve_model(): string {
|
||||
return (string) get_option( 'prautoblogger_research_model', PRAUTOBLOGGER_DEFAULT_ANALYSIS_MODEL );
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the per-agent message sets (system + specialist user prompt).
|
||||
*
|
||||
* @param string $topic Topic text.
|
||||
* @param string $title Suggested article title.
|
||||
* @param string[] $roles Specialist role names.
|
||||
* @return array<int, array<int, array{role: string, content: string}>>
|
||||
*/
|
||||
private function build_agent_messages( string $topic, string $title, array $roles ): array {
|
||||
$system = PRAutoBlogger_Prompt_Registry::render( 'research.system' );
|
||||
$sets = array();
|
||||
foreach ( $roles as $role ) {
|
||||
$sets[] = array(
|
||||
array(
|
||||
'role' => 'system',
|
||||
'content' => $system,
|
||||
),
|
||||
array(
|
||||
'role' => 'user',
|
||||
'content' => sprintf(
|
||||
"Topic: %s\nProposed title: %s\n\nSpecialist focus: %s\n\n" .
|
||||
"Find 3-6 high-quality sources for this topic from a '%s' perspective. " .
|
||||
'Return JSON: {"sources":[{"url":"","title":"","excerpt":"","relevance":0.0}]}',
|
||||
$topic,
|
||||
$title,
|
||||
$role,
|
||||
$role
|
||||
),
|
||||
),
|
||||
);
|
||||
}
|
||||
return $sets;
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate and normalise the parsed sources from one agent's response.
|
||||
*
|
||||
* Schema: `{sources: [{url, title, excerpt, relevance}]}`. Any entry
|
||||
* missing both url and title is silently skipped. Returns null when
|
||||
* the top-level schema is missing or sources is empty after filtering.
|
||||
*
|
||||
* @param array<string, mixed>|null $parsed Decoded JSON from one agent.
|
||||
* @return array<int, array{url: string, title: string, excerpt: string, relevance: float}>|null
|
||||
*
|
||||
* Side effects: none.
|
||||
*/
|
||||
private function validate_sources( ?array $parsed ): ?array {
|
||||
if ( ! is_array( $parsed ) || ! isset( $parsed['sources'] ) || ! is_array( $parsed['sources'] ) ) {
|
||||
return null;
|
||||
}
|
||||
$out = array();
|
||||
foreach ( $parsed['sources'] as $s ) {
|
||||
if ( ! is_array( $s ) ) {
|
||||
continue;
|
||||
}
|
||||
$url = sanitize_url( (string) ( $s['url'] ?? '' ) );
|
||||
$title = sanitize_text_field( (string) ( $s['title'] ?? '' ) );
|
||||
$excerpt = sanitize_textarea_field( (string) ( $s['excerpt'] ?? '' ) );
|
||||
$relevance = max( 0.0, min( 1.0, (float) ( $s['relevance'] ?? 0.0 ) ) );
|
||||
if ( '' === $url && '' === $title ) {
|
||||
continue;
|
||||
}
|
||||
$out[] = array(
|
||||
'url' => $url,
|
||||
'title' => $title,
|
||||
'excerpt' => $excerpt,
|
||||
'relevance' => $relevance,
|
||||
);
|
||||
}
|
||||
return count( $out ) > 0 ? $out : null;
|
||||
}
|
||||
}
|
||||
264
includes/core/class-research-judge.php
Normal file
264
includes/core/class-research-judge.php
Normal file
@@ -0,0 +1,264 @@
|
||||
<?php
|
||||
declare(strict_types=1);
|
||||
|
||||
/**
|
||||
* phpcs:ignore WordPress.Files.FileName.InvalidClassFileName -- class naming convention differs from WordPress standard
|
||||
*
|
||||
* Research Judge — the `curate` stage of the Authority pipeline.
|
||||
*
|
||||
* What: Deduplicates sources across all fan-out agent results using
|
||||
* URL-exact matching first, then semantic similarity (via the
|
||||
* OpenRouter embedding API + cosine similarity, matching the
|
||||
* Semantic_Dedup pattern) with keyword-overlap fallback. Assigns
|
||||
* a quality_score to each source (relevance × source-type weight).
|
||||
* Writes one run_sources row per source (kept=1 or kept=0) with
|
||||
* reason + quality_score via PRAutoBlogger_Research_Source_Writer.
|
||||
* Returns the kept, scored list for the draft stage.
|
||||
* Who triggers it: PRAutoBlogger_Authority_Pipeline (Phase 2b.4, the
|
||||
* tier router). NOT wired into the Economy path — additive only.
|
||||
* Dependencies: PRAutoBlogger_Research_Source_Writer (run_sources writes),
|
||||
* PRAutoBlogger_Research_Source_Scorer (quality scoring),
|
||||
* PRAutoBlogger_Audit_Writer (run_decisions write),
|
||||
* PRAutoBlogger_OpenRouter_Embedding_Provider (semantic dedup),
|
||||
* PRAutoBlogger_Run_Stage_State (curate stage row),
|
||||
* PRAutoBlogger_Stage_Display_Map (default agent role),
|
||||
* WordPress $wpdb, Logger.
|
||||
*
|
||||
* @see providers/interface-research-judge.php — Interface.
|
||||
* @see core/class-research-fanout.php — Produces the input.
|
||||
* @see core/class-research-source-writer.php — Writes run_sources rows.
|
||||
* @see core/class-research-source-scorer.php — Source-type weighting delegate.
|
||||
* @see core/class-semantic-dedup.php — Embedding pattern reference.
|
||||
* @see ARCHITECTURE.md — Phase 2b curate stage.
|
||||
*/
|
||||
class PRAutoBlogger_Research_Judge implements PRAutoBlogger_Research_Judge_Interface {
|
||||
|
||||
/** Cosine similarity threshold above which two sources are duplicates. */
|
||||
private const SEMANTIC_DEDUP_THRESHOLD = 0.90;
|
||||
|
||||
/** Maximum kept sources passed to the draft stage. */
|
||||
private const MAX_KEPT_SOURCES = 12;
|
||||
|
||||
/** @var PRAutoBlogger_Research_Source_Scorer Scores source authority. */
|
||||
private PRAutoBlogger_Research_Source_Scorer $scorer;
|
||||
|
||||
/** @var PRAutoBlogger_Research_Source_Writer Persists run_sources rows. */
|
||||
private PRAutoBlogger_Research_Source_Writer $writer;
|
||||
|
||||
/**
|
||||
* @param PRAutoBlogger_Research_Source_Scorer|null $scorer Optional scorer override.
|
||||
* @param PRAutoBlogger_Research_Source_Writer|null $writer Optional writer override.
|
||||
*/
|
||||
public function __construct(
|
||||
?PRAutoBlogger_Research_Source_Scorer $scorer = null,
|
||||
?PRAutoBlogger_Research_Source_Writer $writer = null
|
||||
) {
|
||||
$this->scorer = $scorer ?? new PRAutoBlogger_Research_Source_Scorer();
|
||||
$this->writer = $writer ?? new PRAutoBlogger_Research_Source_Writer();
|
||||
}
|
||||
|
||||
/**
|
||||
* Deduplicate and score fan-out results; write run_sources rows; return kept.
|
||||
*
|
||||
* Dedup order: (1) URL canonical match; (2) semantic similarity via embedding
|
||||
* API (fallback to keyword-overlap when the API is unavailable). Quality
|
||||
* score = relevance × source-type weight. Sources are sorted by quality_score
|
||||
* descending and capped at MAX_KEPT_SOURCES; overflow rows are written to
|
||||
* run_sources with kept=0 ("exceeded max").
|
||||
*
|
||||
* Side effects: run_sources DB writes, run_stages DB write (curate stage),
|
||||
* possible embedding API call, Logger calls.
|
||||
*
|
||||
* @param string $run_id Run UUID.
|
||||
* @param string $item_key Article-scoped item key.
|
||||
* @param array<int, array{sources: array<int, array{url: string, title: string, excerpt: string, relevance: float}>, agent_role: string}> $fanout_results Agent results from Research_Fanout::dispatch().
|
||||
* @return array<int, array{url: string, title: string, excerpt: string, relevance: float, quality_score: float}> Kept, scored sources.
|
||||
*/
|
||||
public function curate(
|
||||
string $run_id,
|
||||
string $item_key,
|
||||
array $fanout_results
|
||||
): array {
|
||||
PRAutoBlogger_Run_Stage_State::start(
|
||||
$run_id,
|
||||
'curate',
|
||||
(string) PRAutoBlogger_Stage_Display_Map::default_agent_role( 'curate' ),
|
||||
$item_key
|
||||
);
|
||||
|
||||
$all_sources = $this->flatten( $fanout_results );
|
||||
$deduped = $this->deduplicate( $all_sources );
|
||||
$scored = array_map( array( $this->scorer, 'score' ), $deduped );
|
||||
|
||||
usort(
|
||||
$scored,
|
||||
static fn( array $a, array $b ): int => $b['quality_score'] <=> $a['quality_score']
|
||||
);
|
||||
|
||||
$kept = array_slice( $scored, 0, self::MAX_KEPT_SOURCES );
|
||||
$discarded = array_slice( $scored, self::MAX_KEPT_SOURCES );
|
||||
|
||||
$this->writer->write( $run_id, $kept, $discarded );
|
||||
|
||||
PRAutoBlogger_Audit_Writer::record_decision(
|
||||
$run_id,
|
||||
'curate',
|
||||
'completed',
|
||||
sprintf(
|
||||
'Curated %d sources from %d candidates across %d agents (discarded %d).',
|
||||
count( $kept ),
|
||||
count( $all_sources ),
|
||||
count( $fanout_results ),
|
||||
count( $discarded )
|
||||
)
|
||||
);
|
||||
|
||||
PRAutoBlogger_Run_Stage_State::done(
|
||||
$run_id,
|
||||
'curate',
|
||||
(string) PRAutoBlogger_Stage_Display_Map::default_agent_role( 'curate' ),
|
||||
$item_key,
|
||||
wp_json_encode(
|
||||
array(
|
||||
'kept' => count( $kept ),
|
||||
'total' => count( $all_sources ),
|
||||
)
|
||||
)
|
||||
);
|
||||
|
||||
PRAutoBlogger_Logger::instance()->info(
|
||||
sprintf(
|
||||
'Research Judge: kept %d/%d sources (max=%d).',
|
||||
count( $kept ),
|
||||
count( $all_sources ),
|
||||
self::MAX_KEPT_SOURCES
|
||||
),
|
||||
'research-judge'
|
||||
);
|
||||
|
||||
return $kept;
|
||||
}
|
||||
|
||||
// ── Private helpers ─────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Flatten per-agent source lists into one array with agent_role attached.
|
||||
*
|
||||
* @param array<int, array{sources: array<int, array{url: string, title: string, excerpt: string, relevance: float}>, agent_role: string}> $fanout_results
|
||||
* @return array<int, array{url: string, title: string, excerpt: string, relevance: float, agent_role: string}>
|
||||
*/
|
||||
private function flatten( array $fanout_results ): array {
|
||||
$flat = array();
|
||||
foreach ( $fanout_results as $agent ) {
|
||||
foreach ( $agent['sources'] as $source ) {
|
||||
$flat[] = array_merge( $source, array( 'agent_role' => $agent['agent_role'] ) );
|
||||
}
|
||||
}
|
||||
return $flat;
|
||||
}
|
||||
|
||||
/**
|
||||
* Deduplicate: URL-exact first, then semantic similarity.
|
||||
*
|
||||
* @param array<int, array{url: string, title: string, excerpt: string, relevance: float, agent_role: string}> $sources
|
||||
* @return array<int, array{url: string, title: string, excerpt: string, relevance: float, agent_role: string}>
|
||||
*/
|
||||
private function deduplicate( array $sources ): array {
|
||||
$seen_urls = array();
|
||||
$unique = array();
|
||||
$unique_embeddings = array();
|
||||
$unique_kw = array();
|
||||
$embeddings_ok = true;
|
||||
$embed_provider = null;
|
||||
|
||||
foreach ( $sources as $source ) {
|
||||
$canonical = $this->canonical_url( $source['url'] );
|
||||
if ( '' !== $canonical ) {
|
||||
if ( isset( $seen_urls[ $canonical ] ) ) {
|
||||
continue;
|
||||
}
|
||||
$seen_urls[ $canonical ] = true;
|
||||
}
|
||||
|
||||
$text = trim( $source['title'] . ' ' . $source['excerpt'] );
|
||||
|
||||
if ( $embeddings_ok && '' !== $text ) {
|
||||
try {
|
||||
if ( null === $embed_provider ) {
|
||||
$embed_provider = new PRAutoBlogger_OpenRouter_Embedding_Provider();
|
||||
}
|
||||
$emb = $embed_provider->get_embeddings( array( $text ) );
|
||||
$vec = $emb[0] ?? null;
|
||||
if ( null !== $vec ) {
|
||||
foreach ( $unique_embeddings as $ev ) {
|
||||
if ( PRAutoBlogger_OpenRouter_Embedding_Provider::cosine_similarity( $vec, $ev ) >= self::SEMANTIC_DEDUP_THRESHOLD ) {
|
||||
continue 2;
|
||||
}
|
||||
}
|
||||
$unique_embeddings[] = $vec;
|
||||
}
|
||||
} catch ( \Throwable $e ) {
|
||||
$embeddings_ok = false;
|
||||
PRAutoBlogger_Logger::instance()->warning(
|
||||
'Research Judge: embedding unavailable, using keyword fallback. ' . $e->getMessage(),
|
||||
'research-judge'
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if ( ! $embeddings_ok && '' !== $text ) {
|
||||
$kw = $this->extract_keywords( $text );
|
||||
// Mirror Semantic_Dedup: require >=3 keywords before applying overlap
|
||||
// check. Sparse keyword sets (e.g. generic titles like "Source 0")
|
||||
// are kept unconditionally; URL-exact dedup already filtered real
|
||||
// duplicates with identical URLs.
|
||||
if ( count( $kw ) >= 3 ) {
|
||||
foreach ( $unique_kw as $existing ) {
|
||||
$overlap = count( array_intersect( $kw, $existing ) );
|
||||
if ( ( $overlap / count( $kw ) ) >= 0.6 ) {
|
||||
continue 2;
|
||||
}
|
||||
}
|
||||
}
|
||||
$unique_kw[] = $kw;
|
||||
}
|
||||
|
||||
$unique[] = $source;
|
||||
}
|
||||
|
||||
return $unique;
|
||||
}
|
||||
|
||||
/**
|
||||
* Canonical URL: lowercase scheme+host+path, no query/fragment.
|
||||
*
|
||||
* @param string $url Raw URL.
|
||||
* @return string Normalised, or empty on unparseable.
|
||||
*/
|
||||
private function canonical_url( string $url ): string {
|
||||
if ( '' === $url ) {
|
||||
return '';
|
||||
}
|
||||
$parts = wp_parse_url( $url );
|
||||
if ( ! is_array( $parts ) ) {
|
||||
return strtolower( $url );
|
||||
}
|
||||
return strtolower( ( $parts['scheme'] ?? 'https' ) . '://' . ( $parts['host'] ?? '' ) . rtrim( $parts['path'] ?? '', '/' ) );
|
||||
}
|
||||
|
||||
/**
|
||||
* Keyword extractor used for the embedding-unavailable fallback.
|
||||
*
|
||||
* @param string $text Input text.
|
||||
* @return string[] Lowercase unique keywords (3+ chars).
|
||||
*/
|
||||
private function extract_keywords( string $text ): array {
|
||||
static $sw = array( 'a', 'an', 'the', 'and', 'or', 'in', 'of', 'to', 'is', 'was', 'for', 'on', 'at' );
|
||||
$words = preg_split( '/[^a-z0-9-]+/', strtolower( $text ), -1, PREG_SPLIT_NO_EMPTY ) ?? array();
|
||||
return array_values(
|
||||
array_unique(
|
||||
array_filter( $words, static fn( string $w ): bool => strlen( $w ) >= 3 && ! in_array( $w, $sw, true ) )
|
||||
)
|
||||
);
|
||||
}
|
||||
}
|
||||
82
includes/core/class-research-source-scorer.php
Normal file
82
includes/core/class-research-source-scorer.php
Normal file
@@ -0,0 +1,82 @@
|
||||
<?php
|
||||
declare(strict_types=1);
|
||||
|
||||
/**
|
||||
* phpcs:ignore WordPress.Files.FileName.InvalidClassFileName -- class naming convention differs from WordPress standard
|
||||
*
|
||||
* Source authority scorer for the Authority pipeline curate stage.
|
||||
*
|
||||
* What: Computes a quality_score for each research source by multiplying
|
||||
* its agent-assigned relevance by a source-type authority weight
|
||||
* derived from the URL scheme and hostname. Weight tiers:
|
||||
* peer-reviewed (DOI/PubMed/NCBI → 1.0), institutional (.gov /
|
||||
* .edu / WHO / FDA → 0.85), preprint (medRxiv/bioRxiv → 0.70),
|
||||
* general HTTPS (→ 0.60), HTTP or missing (→ 0.40).
|
||||
* Who triggers it: PRAutoBlogger_Research_Judge (curate stage) only.
|
||||
* Not wired into the Economy path.
|
||||
* Dependencies: none — pure computation.
|
||||
*
|
||||
* @see core/class-research-judge.php — Sole consumer.
|
||||
* @see ARCHITECTURE.md — Phase 2b curate stage.
|
||||
*/
|
||||
class PRAutoBlogger_Research_Source_Scorer {
|
||||
|
||||
/**
|
||||
* Add a quality_score field to a source array in place.
|
||||
*
|
||||
* @param array{url: string, title: string, excerpt: string, relevance: float, agent_role: string} $source Source from a research agent.
|
||||
* @return array{url: string, title: string, excerpt: string, relevance: float, agent_role: string, quality_score: float} Source with quality_score added.
|
||||
*
|
||||
* Side effects: none.
|
||||
*/
|
||||
public function score( array $source ): array {
|
||||
$source['quality_score'] = round(
|
||||
$source['relevance'] * $this->weight( $source['url'] ?? '' ),
|
||||
4
|
||||
);
|
||||
return $source;
|
||||
}
|
||||
|
||||
// ── Private helpers ─────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Authority weight for a source URL.
|
||||
*
|
||||
* @param string $url Source URL.
|
||||
* @return float Weight in [0.40, 1.0].
|
||||
*/
|
||||
private function weight( string $url ): float {
|
||||
$u = strtolower( $url );
|
||||
|
||||
if (
|
||||
false !== strpos( $u, 'doi.org' ) ||
|
||||
false !== strpos( $u, 'pubmed.ncbi' ) ||
|
||||
false !== strpos( $u, 'ncbi.nlm.nih' )
|
||||
) {
|
||||
return 1.0;
|
||||
}
|
||||
|
||||
if (
|
||||
false !== strpos( $u, '.gov' ) ||
|
||||
false !== strpos( $u, '.edu' ) ||
|
||||
false !== strpos( $u, 'who.int' ) ||
|
||||
false !== strpos( $u, 'fda.gov' )
|
||||
) {
|
||||
return 0.85;
|
||||
}
|
||||
|
||||
if (
|
||||
false !== strpos( $u, 'medrxiv' ) ||
|
||||
false !== strpos( $u, 'biorxiv' ) ||
|
||||
false !== strpos( $u, 'preprint' )
|
||||
) {
|
||||
return 0.70;
|
||||
}
|
||||
|
||||
if ( 0 === strpos( $u, 'https://' ) ) {
|
||||
return 0.60;
|
||||
}
|
||||
|
||||
return 0.40;
|
||||
}
|
||||
}
|
||||
88
includes/core/class-research-source-writer.php
Normal file
88
includes/core/class-research-source-writer.php
Normal file
@@ -0,0 +1,88 @@
|
||||
<?php
|
||||
declare(strict_types=1);
|
||||
|
||||
/**
|
||||
* phpcs:ignore WordPress.Files.FileName.InvalidClassFileName -- class naming convention differs from WordPress standard
|
||||
*
|
||||
* Run-sources DB writer for the Authority pipeline curate stage.
|
||||
*
|
||||
* What: Persists kept and discarded source records to the
|
||||
* wp_prautoblogger_run_sources table via PRAutoBlogger_Audit_Writer.
|
||||
* Builds the reason string for kept rows (quality/relevance) and the
|
||||
* fixed "exceeded max" reason for discarded rows. Also extracts DOI
|
||||
* identifiers from source URLs for storage.
|
||||
* Who triggers it: PRAutoBlogger_Research_Judge (curate stage) only.
|
||||
* Not wired into the Economy path.
|
||||
* Dependencies: PRAutoBlogger_Audit_Writer (availability check + DB insert),
|
||||
* WordPress $wpdb, current_time().
|
||||
*
|
||||
* @see core/class-research-judge.php — Sole consumer.
|
||||
* @see core/class-audit-writer.php — Underlying DB insert layer.
|
||||
* @see ARCHITECTURE.md — Phase 2b curate stage.
|
||||
*/
|
||||
class PRAutoBlogger_Research_Source_Writer {
|
||||
|
||||
/**
|
||||
* Write run_sources rows for kept (kept=1) and discarded (kept=0) sources.
|
||||
*
|
||||
* @param string $run_id Run UUID.
|
||||
* @param array<int, array{url: string, agent_role?: string, quality_score: float, relevance: float}> $kept Scored kept sources.
|
||||
* @param array<int, array{url: string, agent_role?: string, quality_score: float, relevance: float}> $discarded Scored discarded sources (over max).
|
||||
* @return void Side effects: DB inserts; no-op when audit tables absent.
|
||||
*/
|
||||
public function write( string $run_id, array $kept, array $discarded ): void {
|
||||
if ( ! PRAutoBlogger_Audit_Writer::is_available() ) {
|
||||
return;
|
||||
}
|
||||
global $wpdb;
|
||||
$table = $wpdb->prefix . 'prautoblogger_run_sources';
|
||||
$now = current_time( 'mysql' );
|
||||
$sources = array_merge(
|
||||
array_map(
|
||||
static fn( $s ) => $s + array(
|
||||
'is_kept' => 1,
|
||||
'discard_reason' => '',
|
||||
),
|
||||
$kept
|
||||
),
|
||||
array_map(
|
||||
static fn( $s ) => $s + array(
|
||||
'is_kept' => 0,
|
||||
'discard_reason' => 'Exceeded maximum kept sources per run',
|
||||
),
|
||||
$discarded
|
||||
)
|
||||
);
|
||||
foreach ( $sources as $s ) {
|
||||
// phpcs:ignore WordPress.DB.DirectDatabaseQuery.DirectQuery
|
||||
$wpdb->insert(
|
||||
$table,
|
||||
array(
|
||||
'run_id' => $run_id,
|
||||
'agent_role' => $s['agent_role'] ?? 'curator',
|
||||
'source_url' => $s['url'],
|
||||
'doi' => $this->extract_doi( $s['url'] ),
|
||||
'kept' => $s['is_kept'],
|
||||
'reason' => $s['is_kept'] ? sprintf( 'Quality %.3f (relevance=%.2f)', $s['quality_score'], $s['relevance'] ) : $s['discard_reason'],
|
||||
'quality_score' => $s['quality_score'],
|
||||
'created_at' => $now,
|
||||
)
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// ── Private helpers ─────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Extract a DOI from a URL if one is present.
|
||||
*
|
||||
* @param string $url Source URL.
|
||||
* @return string|null DOI, or null.
|
||||
*/
|
||||
private function extract_doi( string $url ): ?string {
|
||||
if ( preg_match( '#10\.\d{4,}/\S+#', $url, $matches ) ) {
|
||||
return rtrim( $matches[0], '.' );
|
||||
}
|
||||
return null;
|
||||
}
|
||||
}
|
||||
27
includes/providers/interface-research-fanout.php
Normal file
27
includes/providers/interface-research-fanout.php
Normal file
@@ -0,0 +1,27 @@
|
||||
<?php
|
||||
declare(strict_types=1);
|
||||
|
||||
/**
|
||||
* Contract for parallel specialist research fan-out.
|
||||
*
|
||||
* @see core/class-research-fanout.php — Primary implementation.
|
||||
* @see ARCHITECTURE.md — Phase 2b design.
|
||||
*/
|
||||
interface PRAutoBlogger_Research_Fanout_Interface {
|
||||
|
||||
/**
|
||||
* Dispatch N specialist research agents in parallel and collect results.
|
||||
*
|
||||
* @param string $run_id Pipeline run UUID.
|
||||
* @param string $item_key Article item key for stage scoping.
|
||||
* @param PRAutoBlogger_Article_Idea $idea The idea being researched.
|
||||
* @param PRAutoBlogger_Cost_Tracker $cost_tracker Pipeline cost tracker.
|
||||
* @return array<int, array{sources: array<int, array{url: string, title: string, excerpt: string, relevance: float}>, agent_role: string}> Results indexed by agent slot (0..N-1). Empty array means quorum not met (run should hold).
|
||||
*/
|
||||
public function dispatch(
|
||||
string $run_id,
|
||||
string $item_key,
|
||||
PRAutoBlogger_Article_Idea $idea,
|
||||
PRAutoBlogger_Cost_Tracker $cost_tracker
|
||||
): array;
|
||||
}
|
||||
26
includes/providers/interface-research-judge.php
Normal file
26
includes/providers/interface-research-judge.php
Normal file
@@ -0,0 +1,26 @@
|
||||
<?php
|
||||
declare(strict_types=1);
|
||||
|
||||
/**
|
||||
* Contract for the research judge that curates fan-out results.
|
||||
*
|
||||
* @see core/class-research-judge.php — Primary implementation.
|
||||
* @see ARCHITECTURE.md — Phase 2b curate stage.
|
||||
*/
|
||||
interface PRAutoBlogger_Research_Judge_Interface {
|
||||
|
||||
/**
|
||||
* Deduplicate and score the fan-out results; write keep/discard decisions
|
||||
* to run_sources; return kept sources for the draft stage.
|
||||
*
|
||||
* @param string $run_id Run UUID.
|
||||
* @param string $item_key Article-scoped item key.
|
||||
* @param array<int, array{sources: array<int, array{url: string, title: string, excerpt: string, relevance: float}>, agent_role: string}> $fanout_results Agent results from Research_Fanout::dispatch().
|
||||
* @return array<int, array{url: string, title: string, excerpt: string, relevance: float, quality_score: float}> Kept, deduplicated, scored sources.
|
||||
*/
|
||||
public function curate(
|
||||
string $run_id,
|
||||
string $item_key,
|
||||
array $fanout_results
|
||||
): array;
|
||||
}
|
||||
@@ -9,7 +9,7 @@
|
||||
* Plugin Name: PRAutoBlogger
|
||||
* Plugin URI: https://peptiderepo.com/prautoblogger
|
||||
* Description: Monitors social media for trending topics, generates SEO-friendly blog posts using AI, and publishes them on a daily schedule with full cost tracking and self-improvement metrics.
|
||||
* Version: 0.27.1
|
||||
* Version: 0.28.0
|
||||
* Requires at least: 6.0
|
||||
* Requires PHP: 7.4
|
||||
* Author: PeptideRepo
|
||||
@@ -35,8 +35,8 @@ if ( ! defined( 'ABSPATH' ) ) {
|
||||
| Defined here so every file in the plugin can reference paths, versions,
|
||||
| and limits without magic strings.
|
||||
*/
|
||||
define( 'PRAUTOBLOGGER_VERSION', '0.27.1' );
|
||||
define( 'PRAUTOBLOGGER_DB_VERSION', '1.4.0' );
|
||||
define( 'PRAUTOBLOGGER_VERSION', '0.28.0' );
|
||||
define( 'PRAUTOBLOGGER_DB_VERSION', '1.4.0' ); // No schema change in v0.28.0 (Research_Fanout/Judge are additive code only).
|
||||
define( 'PRAUTOBLOGGER_DEFAULT_RUN_CEILING_USD', 0.50 );
|
||||
define( 'PRAUTOBLOGGER_DEFAULT_REQUEST_JSON_RETENTION_DAYS', 14 );
|
||||
// Default max cards shown per board column and per ideas-browser page.
|
||||
|
||||
280
tests/unit/Core/ResearchFanoutTest.php
Normal file
280
tests/unit/Core/ResearchFanoutTest.php
Normal file
@@ -0,0 +1,280 @@
|
||||
<?php
|
||||
/**
|
||||
* Tests for PRAutoBlogger_Research_Fanout.
|
||||
*
|
||||
* Covers: quorum logic, batch cost-reserve wiring, partial-failure handling.
|
||||
* The curl_multi execution is delegated to PRAutoBlogger_Research_Batch;
|
||||
* these tests inject a stub batch to isolate the fanout orchestration logic.
|
||||
*
|
||||
* @package PRAutoBlogger\Tests\Core
|
||||
*/
|
||||
|
||||
namespace PRAutoBlogger\Tests\Core;
|
||||
|
||||
use PRAutoBlogger\Tests\BaseTestCase;
|
||||
use Brain\Monkey\Functions;
|
||||
|
||||
class ResearchFanoutTest extends BaseTestCase {
|
||||
|
||||
/** @var \PHPUnit\Framework\MockObject\MockObject Mock $wpdb. */
|
||||
private $wpdb;
|
||||
|
||||
protected function setUp(): void {
|
||||
parent::setUp();
|
||||
$this->wpdb = $this->create_mock_wpdb();
|
||||
$GLOBALS['wpdb'] = $this->wpdb;
|
||||
|
||||
$this->wpdb->method( 'prepare' )->willReturnCallback(
|
||||
static function ( $sql, ...$args ) {
|
||||
return $sql . ' /* ' . implode( ',', array_map( 'strval', $args ) ) . ' */';
|
||||
}
|
||||
);
|
||||
|
||||
\PRAutoBlogger_Run_Context::clear();
|
||||
\PRAutoBlogger_Run_State::flush_cache();
|
||||
\PRAutoBlogger_Run_Stage_State::flush_cache();
|
||||
|
||||
$this->stub_get_option(
|
||||
array(
|
||||
'prautoblogger_per_run_cost_ceiling_usd' => 0.50,
|
||||
'prautoblogger_research_agent_count' => 3,
|
||||
'prautoblogger_research_model' => 'google/gemini-2.5-flash-lite',
|
||||
'prautoblogger_log_level' => 'error',
|
||||
)
|
||||
);
|
||||
|
||||
Functions\when( 'wp_json_encode' )->alias( 'json_encode' );
|
||||
Functions\when( 'sanitize_url' )->alias( static fn( $u ) => $u );
|
||||
Functions\when( 'sanitize_text_field' )->alias( 'trim' );
|
||||
Functions\when( 'sanitize_textarea_field' )->alias( 'trim' );
|
||||
}
|
||||
|
||||
protected function tearDown(): void {
|
||||
\PRAutoBlogger_Run_Context::clear();
|
||||
\PRAutoBlogger_Run_State::flush_cache();
|
||||
\PRAutoBlogger_Run_Stage_State::flush_cache();
|
||||
unset( $GLOBALS['wpdb'] );
|
||||
parent::tearDown();
|
||||
}
|
||||
|
||||
// ── Quorum logic ────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* With N=3, quorum = ceil(3/2)+1 = 3. All 3 agents succeed → results returned.
|
||||
*/
|
||||
public function test_all_agents_succeed_meets_quorum(): void {
|
||||
$batch = $this->make_stub_batch( $this->make_valid_raw_results( 3 ) );
|
||||
$fanout = $this->make_fanout( $batch );
|
||||
|
||||
$results = $fanout->dispatch( 'run-1', 'idea:abc', $this->make_idea(), $this->make_cost_tracker() );
|
||||
|
||||
$this->assertCount( 3, $results );
|
||||
foreach ( $results as $r ) {
|
||||
$this->assertArrayHasKey( 'sources', $r );
|
||||
$this->assertArrayHasKey( 'agent_role', $r );
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* N=3, quorum=3. Only 2 agents succeed (one fails) → quorum NOT met → empty.
|
||||
*/
|
||||
public function test_partial_failure_below_quorum_returns_empty(): void {
|
||||
$raw = $this->make_valid_raw_results( 3 );
|
||||
$raw[2] = array( 'error' => 'timeout' ); // Slot 2 fails.
|
||||
$batch = $this->make_stub_batch( $raw );
|
||||
$fanout = $this->make_fanout( $batch );
|
||||
|
||||
$results = $fanout->dispatch( 'run-2', 'idea:abc', $this->make_idea(), $this->make_cost_tracker() );
|
||||
|
||||
$this->assertEmpty( $results, 'Quorum not met: should return empty array.' );
|
||||
}
|
||||
|
||||
/**
|
||||
* N=3, quorum=3. Agent returns invalid JSON schema → excluded → empty.
|
||||
*/
|
||||
public function test_invalid_schema_agent_excluded(): void {
|
||||
$raw = $this->make_valid_raw_results( 3 );
|
||||
$raw[0] = array( 'content' => '{"bad":"schema"}', 'actual_cost' => 0.001 );
|
||||
$raw[1] = array( 'error' => 'bad' );
|
||||
$raw[2] = array( 'error' => 'bad' );
|
||||
$batch = $this->make_stub_batch( $raw );
|
||||
$fanout = $this->make_fanout( $batch );
|
||||
|
||||
$results = $fanout->dispatch( 'run-3', 'idea:abc', $this->make_idea(), $this->make_cost_tracker() );
|
||||
|
||||
$this->assertEmpty( $results );
|
||||
}
|
||||
|
||||
/**
|
||||
* N=5, quorum = ceil(5/2)+1 = 4. 4 agents succeed, 1 fails → quorum met.
|
||||
*/
|
||||
public function test_quorum_formula_for_n_5(): void {
|
||||
$this->stub_get_option(
|
||||
array(
|
||||
'prautoblogger_per_run_cost_ceiling_usd' => 0.50,
|
||||
'prautoblogger_research_agent_count' => 5,
|
||||
'prautoblogger_research_model' => 'google/gemini-2.5-flash-lite',
|
||||
'prautoblogger_log_level' => 'error',
|
||||
)
|
||||
);
|
||||
|
||||
$raw = $this->make_valid_raw_results( 5 );
|
||||
$raw[4] = array( 'error' => 'failed' ); // 4 succeed, 1 fails: 4 >= 4 → quorum met.
|
||||
$batch = $this->make_stub_batch( $raw );
|
||||
$fanout = $this->make_fanout( $batch );
|
||||
|
||||
$results = $fanout->dispatch( 'run-4', 'idea:abc', $this->make_idea(), $this->make_cost_tracker() );
|
||||
|
||||
$this->assertCount( 4, $results );
|
||||
}
|
||||
|
||||
// ── Cost reserve wiring ─────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Cost governor open_amount_reservation is NOT called when there is
|
||||
* no active run context (ungoverned behavior unchanged).
|
||||
*/
|
||||
public function test_no_run_context_ungoverned(): void {
|
||||
// No run context set → Cost_Governor::open_amount_reservation returns null.
|
||||
$batch = $this->make_stub_batch( $this->make_valid_raw_results( 3 ) );
|
||||
$fanout = $this->make_fanout( $batch );
|
||||
|
||||
// Should not throw — just proceeds ungoverned.
|
||||
$results = $fanout->dispatch( 'run-5', 'idea:abc', $this->make_idea(), $this->make_cost_tracker() );
|
||||
$this->assertCount( 3, $results );
|
||||
}
|
||||
|
||||
/**
|
||||
* When open_amount_reservation() throws PRAutoBlogger_Cost_Ceiling_Exception,
|
||||
* batch->execute() must NEVER be called — cost governance aborts before dispatch.
|
||||
*/
|
||||
public function test_ceiling_breach_exception_aborts_before_dispatch(): void {
|
||||
$batch = $this->getMockBuilder( \PRAutoBlogger_Research_Batch::class )
|
||||
->disableOriginalConstructor()
|
||||
->onlyMethods( array( 'execute' ) )
|
||||
->getMock();
|
||||
|
||||
// PHPUnit enforces this: execute() must NEVER be called when ceiling is breached.
|
||||
$batch->expects( $this->never() )->method( 'execute' );
|
||||
|
||||
// ceiling_usd=0.0: any reservation attempt breaches → UPDATE affects 0 rows → exception.
|
||||
// get_var must return the table name so Run_State::is_available() returns true;
|
||||
// without this, open_amount_reservation() short-circuits (returns null, no throw).
|
||||
$this->wpdb->method( 'get_var' )->willReturnCallback(
|
||||
static function ( $sql ) {
|
||||
// Run_State::table_name() = 'wp_prautoblogger_runs' (not run_states).
|
||||
// is_available() does SHOW TABLES LIKE %s where %s = wp_prautoblogger_runs.
|
||||
if ( false !== strpos( (string) $sql, 'prautoblogger_runs' ) ) {
|
||||
return 'wp_prautoblogger_runs';
|
||||
}
|
||||
return null;
|
||||
}
|
||||
);
|
||||
// ceiling_usd must be > 0; 0.0 means 'ceiling disabled' (governor returns null, no throw).
|
||||
// Set ceiling=0.01, already reserved=0.01 (at limit) so any additional reserve
|
||||
// fails — query() returns 0 rows affected → on_breach() → throws.
|
||||
// get_row() is called with ARRAY_A mode; get_run() does is_array() check.
|
||||
// Return an array (not an object) so get_run() returns the row instead of null.
|
||||
// ceiling=0.01, reserved=0.01 → at limit; any further reserve fails (query→0).
|
||||
$this->wpdb->method( 'get_row' )->willReturn( array(
|
||||
'run_id' => 'run-ceiling',
|
||||
'ceiling_usd' => 0.01,
|
||||
'reserved_usd' => 0.01,
|
||||
'settled_usd' => 0.0,
|
||||
'status' => 'running',
|
||||
) );
|
||||
$this->wpdb->method( 'query' )->willReturn( 0 );
|
||||
|
||||
\PRAutoBlogger_Run_Context::set_run_id( 'run-ceiling' );
|
||||
|
||||
$fanout = $this->make_fanout( $batch );
|
||||
|
||||
$this->expectException( \PRAutoBlogger_Cost_Ceiling_Exception::class );
|
||||
$fanout->dispatch( 'run-ceiling', 'idea:abc', $this->make_idea(), $this->make_cost_tracker() );
|
||||
}
|
||||
|
||||
// ── Helpers ─────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Build a stub Research_Batch that returns pre-canned raw results.
|
||||
*
|
||||
* @param array $raw_results
|
||||
* @return \PHPUnit\Framework\MockObject\MockObject
|
||||
*/
|
||||
private function make_stub_batch( array $raw_results ) {
|
||||
$stub = $this->getMockBuilder( \PRAutoBlogger_Research_Batch::class )
|
||||
->disableOriginalConstructor()
|
||||
->onlyMethods( array( 'execute' ) )
|
||||
->getMock();
|
||||
$stub->method( 'execute' )->willReturn( $raw_results );
|
||||
return $stub;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a fanout with an injected batch stub and a stubbed cost tracker.
|
||||
*
|
||||
* @param object $batch
|
||||
* @return \PRAutoBlogger_Research_Fanout
|
||||
*/
|
||||
private function make_fanout( $batch ): \PRAutoBlogger_Research_Fanout {
|
||||
// NOTE: do NOT re-stub get_option here. setUp() and individual tests call
|
||||
// stub_get_option() with the correct agent_count for that test. Re-stubbing
|
||||
// here with hardcoded agent_count=3 silently overrides the test's value and
|
||||
// caused test_quorum_formula_for_n_5 to dispatch 3 agents against 5 results.
|
||||
return new \PRAutoBlogger_Research_Fanout( $batch );
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate N valid raw results (the shape Research_Batch returns).
|
||||
*
|
||||
* @param int $n
|
||||
* @return array
|
||||
*/
|
||||
private function make_valid_raw_results( int $n ): array {
|
||||
$results = array();
|
||||
for ( $i = 0; $i < $n; $i++ ) {
|
||||
$results[] = array(
|
||||
'content' => json_encode( array(
|
||||
'sources' => array(
|
||||
array( 'url' => "https://example.com/source-$i", 'title' => "Source $i", 'excerpt' => 'Test', 'relevance' => 0.8 ),
|
||||
),
|
||||
) ),
|
||||
'model' => 'google/gemini-2.5-flash-lite',
|
||||
'prompt_tokens' => 100,
|
||||
'completion_tokens' => 200,
|
||||
'actual_cost' => 0.001,
|
||||
);
|
||||
}
|
||||
return $results;
|
||||
}
|
||||
|
||||
/**
|
||||
* @return \PRAutoBlogger_Article_Idea
|
||||
*/
|
||||
private function make_idea(): \PRAutoBlogger_Article_Idea {
|
||||
return new \PRAutoBlogger_Article_Idea( array(
|
||||
'topic' => 'BPC-157 peptide research',
|
||||
'article_type' => 'guide',
|
||||
'suggested_title' => 'BPC-157: Research Overview',
|
||||
'summary' => 'Test.',
|
||||
'score' => 0.9,
|
||||
'source_ids' => array(),
|
||||
'key_points' => array(),
|
||||
'target_keywords' => array(),
|
||||
) );
|
||||
}
|
||||
|
||||
/**
|
||||
* @return \PRAutoBlogger_Cost_Tracker
|
||||
*/
|
||||
private function make_cost_tracker(): \PRAutoBlogger_Cost_Tracker {
|
||||
$ct = $this->getMockBuilder( \PRAutoBlogger_Cost_Tracker::class )
|
||||
->disableOriginalConstructor()
|
||||
->onlyMethods( array( 'log_api_call', 'is_budget_exceeded' ) )
|
||||
->getMock();
|
||||
$ct->method( 'log_api_call' ); // void: no return value
|
||||
$ct->method( 'is_budget_exceeded' )->willReturn( false );
|
||||
return $ct;
|
||||
}
|
||||
}
|
||||
|
||||
237
tests/unit/Core/ResearchJudgeTest.php
Normal file
237
tests/unit/Core/ResearchJudgeTest.php
Normal file
@@ -0,0 +1,237 @@
|
||||
<?php
|
||||
/**
|
||||
* Tests for PRAutoBlogger_Research_Judge (curate stage).
|
||||
*
|
||||
* Covers: dedup/keep-discard logic, quality_score computation, run_sources
|
||||
* DB writes (keep=1 / keep=0), and graceful degradation when audit tables
|
||||
* are absent.
|
||||
*
|
||||
* @package PRAutoBlogger\Tests\Core
|
||||
*/
|
||||
|
||||
namespace PRAutoBlogger\Tests\Core;
|
||||
|
||||
use PRAutoBlogger\Tests\BaseTestCase;
|
||||
use Brain\Monkey\Functions;
|
||||
|
||||
class ResearchJudgeTest extends BaseTestCase {
|
||||
|
||||
/** @var \PHPUnit\Framework\MockObject\MockObject Mock $wpdb. */
|
||||
private $wpdb;
|
||||
|
||||
/** @var array<string, array<int, array>> Rows inserted, keyed by table name suffix (e.g. 'run_sources'). */
|
||||
private array $inserted_rows = array();
|
||||
|
||||
protected function setUp(): void {
|
||||
parent::setUp();
|
||||
$this->inserted_rows = array();
|
||||
$this->wpdb = $this->create_mock_wpdb();
|
||||
$GLOBALS['wpdb'] = $this->wpdb;
|
||||
|
||||
$this->wpdb->prefix = 'wp_';
|
||||
$this->wpdb->method( 'prepare' )->willReturnCallback(
|
||||
static function ( $sql, ...$args ) {
|
||||
return $sql . ' /* ' . implode( ',', array_map( 'strval', $args ) ) . ' */';
|
||||
}
|
||||
);
|
||||
|
||||
// Capture insert calls keyed by table name suffix (strip prefix) so tests
|
||||
// can distinguish run_sources rows from Logger event_log rows, etc.
|
||||
$test = $this;
|
||||
$this->wpdb->method( 'insert' )->willReturnCallback(
|
||||
function ( $table, $data ) use ( $test ) {
|
||||
// Strip the 'wp_prautoblogger_' prefix to get a short key.
|
||||
$key = preg_replace( '/^wp_prautoblogger_/', '', (string) $table );
|
||||
$test->inserted_rows[ $key ][] = $data;
|
||||
return 1;
|
||||
}
|
||||
);
|
||||
|
||||
\PRAutoBlogger_Audit_Writer::flush_cache();
|
||||
\PRAutoBlogger_Run_Stage_State::flush_cache();
|
||||
|
||||
Functions\when( 'wp_json_encode' )->alias( 'json_encode' );
|
||||
Functions\when( 'wp_parse_url' )->alias( 'parse_url' );
|
||||
Functions\when( 'current_time' )->justReturn( '2026-06-23 00:00:00' );
|
||||
Functions\when( 'get_option' )->justReturn( 'error' );
|
||||
}
|
||||
|
||||
protected function tearDown(): void {
|
||||
\PRAutoBlogger_Audit_Writer::flush_cache();
|
||||
\PRAutoBlogger_Run_Stage_State::flush_cache();
|
||||
unset( $GLOBALS['wpdb'] );
|
||||
parent::tearDown();
|
||||
}
|
||||
|
||||
// ── Keep / discard ──────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* All unique sources with quality > 0 are kept when under MAX_KEPT_SOURCES.
|
||||
*/
|
||||
public function test_all_unique_sources_kept_under_max(): void {
|
||||
$judge = $this->make_judge_with_available_tables();
|
||||
$fanout = $this->make_fanout_results( 3, 0 );
|
||||
$kept = $judge->curate( 'run-1', 'idea:abc', $fanout );
|
||||
|
||||
$this->assertCount( 3, $kept );
|
||||
foreach ( $kept as $s ) {
|
||||
$this->assertArrayHasKey( 'quality_score', $s );
|
||||
$this->assertGreaterThan( 0.0, $s['quality_score'] );
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Exact-URL duplicates across agents are deduplicated to one entry.
|
||||
*/
|
||||
public function test_url_duplicates_are_deduplicated(): void {
|
||||
$fanout = array(
|
||||
array(
|
||||
'agent_role' => 'researcher:mechanisms',
|
||||
'sources' => array(
|
||||
array( 'url' => 'https://example.com/a', 'title' => 'A', 'excerpt' => 'ex', 'relevance' => 0.8 ),
|
||||
),
|
||||
),
|
||||
array(
|
||||
'agent_role' => 'researcher:clinical',
|
||||
'sources' => array(
|
||||
array( 'url' => 'https://example.com/a', 'title' => 'A dup', 'excerpt' => 'ex2', 'relevance' => 0.9 ),
|
||||
),
|
||||
),
|
||||
);
|
||||
|
||||
$judge = $this->make_judge_with_available_tables();
|
||||
$kept = $judge->curate( 'run-2', 'idea:abc', $fanout );
|
||||
|
||||
$this->assertCount( 1, $kept );
|
||||
}
|
||||
|
||||
/**
|
||||
* Sources beyond MAX_KEPT_SOURCES are written to run_sources with kept=0.
|
||||
*/
|
||||
public function test_sources_over_max_written_as_discarded(): void {
|
||||
// Generate 14 distinct sources (max is 12 → 2 discarded).
|
||||
$judge = $this->make_judge_with_available_tables();
|
||||
$fanout = $this->make_fanout_results( 14, 0 );
|
||||
$kept = $judge->curate( 'run-3', 'idea:abc', $fanout );
|
||||
|
||||
$this->assertCount( 12, $kept );
|
||||
|
||||
// Only examine run_sources rows (keyed by table); Logger rows are separate.
|
||||
$run_source_rows = $this->inserted_rows['run_sources'] ?? array();
|
||||
$discarded_rows = array_filter( $run_source_rows, static fn( $r ) => 0 === (int) $r['kept'] );
|
||||
$this->assertCount( 2, array_values( $discarded_rows ) );
|
||||
}
|
||||
|
||||
// ── Quality scoring ─────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* A PubMed URL should receive a higher quality_score than a plain HTTPS URL.
|
||||
*/
|
||||
public function test_doi_url_scores_higher_than_generic_https(): void {
|
||||
$fanout = array(
|
||||
array(
|
||||
'agent_role' => 'researcher:mechanisms',
|
||||
'sources' => array(
|
||||
array( 'url' => 'https://doi.org/10.1234/abc', 'title' => 'DOI', 'excerpt' => 'test', 'relevance' => 0.8 ),
|
||||
array( 'url' => 'https://example.com/article', 'title' => 'Generic', 'excerpt' => 'test', 'relevance' => 0.8 ),
|
||||
),
|
||||
),
|
||||
);
|
||||
|
||||
$judge = $this->make_judge_with_available_tables();
|
||||
$kept = $judge->curate( 'run-4', 'idea:abc', $fanout );
|
||||
|
||||
$this->assertCount( 2, $kept );
|
||||
// DOI should be first (sorted descending by quality_score).
|
||||
$this->assertStringContainsString( 'doi.org', $kept[0]['url'] );
|
||||
$this->assertGreaterThan( $kept[1]['quality_score'], $kept[0]['quality_score'] );
|
||||
}
|
||||
|
||||
// ── run_sources writes ───────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Each kept source is inserted with kept=1 and a non-empty reason.
|
||||
*/
|
||||
public function test_kept_sources_written_with_kept_1(): void {
|
||||
$judge = $this->make_judge_with_available_tables();
|
||||
$judge->curate( 'run-5', 'idea:abc', $this->make_fanout_results( 2, 0 ) );
|
||||
|
||||
// Only examine run_sources rows; Logger rows have no 'kept' key.
|
||||
$run_source_rows = $this->inserted_rows['run_sources'] ?? array();
|
||||
$kept_rows = array_filter( $run_source_rows, static fn( $r ) => 1 === (int) $r['kept'] );
|
||||
$this->assertCount( 2, array_values( $kept_rows ) );
|
||||
|
||||
foreach ( $kept_rows as $row ) {
|
||||
$this->assertNotEmpty( $row['reason'] );
|
||||
$this->assertGreaterThanOrEqual( 0.0, $row['quality_score'] );
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* No-op when the audit tables are absent (is_available() returns false).
|
||||
*/
|
||||
public function test_no_db_writes_when_tables_absent(): void {
|
||||
$judge = $this->make_judge_without_tables();
|
||||
$fanout = $this->make_fanout_results( 3, 0 );
|
||||
$kept = $judge->curate( 'run-6', 'idea:abc', $fanout );
|
||||
|
||||
$this->assertCount( 3, $kept );
|
||||
// When audit tables are absent, run_sources and run_decisions must be empty.
|
||||
// The Logger may still write to event_log (wpdb->insert check skips only when
|
||||
// $wpdb is null) — that is expected behaviour and NOT tested here.
|
||||
$this->assertEmpty( $this->inserted_rows['run_sources'] ?? array() );
|
||||
$this->assertEmpty( $this->inserted_rows['run_decisions'] ?? array() );
|
||||
}
|
||||
|
||||
// ── Helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Build a judge where audit tables are reported as available.
|
||||
*/
|
||||
private function make_judge_with_available_tables(): \PRAutoBlogger_Research_Judge {
|
||||
// is_available() probes with SHOW TABLES LIKE — return the table name.
|
||||
$this->wpdb->method( 'get_var' )->willReturnCallback(
|
||||
function ( $sql ) {
|
||||
if ( false !== strpos( (string) $sql, 'run_decisions' ) ) {
|
||||
return 'wp_prautoblogger_run_decisions';
|
||||
}
|
||||
if ( false !== strpos( (string) $sql, 'run_stages' ) ) {
|
||||
return 'wp_prautoblogger_run_stages';
|
||||
}
|
||||
return null;
|
||||
}
|
||||
);
|
||||
return new \PRAutoBlogger_Research_Judge();
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a judge where audit tables are absent (is_available() → false).
|
||||
*/
|
||||
private function make_judge_without_tables(): \PRAutoBlogger_Research_Judge {
|
||||
$this->wpdb->method( 'get_var' )->willReturn( null );
|
||||
return new \PRAutoBlogger_Research_Judge();
|
||||
}
|
||||
|
||||
/**
|
||||
* Build fanout_results with $count unique sources and $dup_count duplicates.
|
||||
*
|
||||
* @param int $count Number of unique sources across all agents.
|
||||
* @param int $dup_count Number of duplicate (same-URL) sources to add.
|
||||
* @return array
|
||||
*/
|
||||
private function make_fanout_results( int $count, int $dup_count ): array {
|
||||
$sources = array();
|
||||
for ( $i = 0; $i < $count; $i++ ) {
|
||||
$sources[] = array( 'url' => "https://example.com/src-$i", 'title' => "Source $i", 'excerpt' => "Excerpt $i", 'relevance' => 0.7 );
|
||||
}
|
||||
for ( $i = 0; $i < $dup_count; $i++ ) {
|
||||
$sources[] = array( 'url' => "https://example.com/src-0", 'title' => 'Dup', 'excerpt' => 'dup', 'relevance' => 0.5 ); // same URL as first.
|
||||
}
|
||||
return array(
|
||||
array(
|
||||
'agent_role' => 'researcher:mechanisms',
|
||||
'sources' => $sources,
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user