Compare commits

...

16 Commits

Author SHA1 Message Date
peptiderepo
6df75b7475 fix: expand single-line associative array to multi-line (PHPCS L89) 2026-04-24 12:01:24 +07:00
peptiderepo
33ddb826ca chore: revert ci.yml to summary report 2026-04-24 12:01:04 +07:00
peptiderepo
244e39828b chore: temp - full PHPCS output for diagnosis (revert after) 2026-04-24 11:59:34 +07:00
peptiderepo
c51a1eb47c fix: remove extra trailing blank line at EOF (PHPCS SuperfluousWhitespace) 2026-04-24 11:55:12 +07:00
peptiderepo
5f2dcb66aa fix: align => operators in push_to_opik payload array (PHPCS) 2026-04-24 11:46:14 +07:00
peptiderepo
a02e8df5e6 fix: align => operators in early-return array (PHPCS) 2026-04-24 11:44:12 +07:00
peptiderepo
079c020e7b style(phpcs): normalize variable assignment alignment padding in eval runner 2026-04-24 11:39:57 +07:00
peptiderepo
2dde5610db style(phpcs): remove alignment padding in eval runner constructor 2026-04-24 11:32:09 +07:00
peptiderepo
644ffbac21 style(phpcs): fix register_cli_hooks indentation + filter_block_false_updates double-tab 2026-04-24 11:32:07 +07:00
peptiderepo
c79fbb570e feat: wave 2 eval harness with lm-as-judge and cli command
New eval framework:
- eval/dataset.json: 20 frozen peptide topic eval items
- class-opik-eval-scorer.php: LLM-as-judge using Gemini Flash 1.5
  Scores on: scientific_signal, readability, style_adherence (0-10)
- class-opik-eval-runner.php: orchestrator for full eval pass
  Loads dataset, runs generation (no publish/images), scores, pushes to Opik
  run(int $limit = 0, bool $dry_run = false): array
- class-wp-cli-commands.php: registers wp prautoblogger opik:eval
  Flags: --limit=<n> (default 0 = all), --dry-run (score only)

Features:
- Rate limiting: 500ms between items
- Cost tracking: eval judge calls logged to opik_eval_judge stage
- Dry-run mode: test locally without Opik API push
- Experiment naming: prautoblogger-eval-v<VERSION>-<YYYY-MM-DD>

All files <= 300 lines. Version 0.13.0. Ready for PR.
2026-04-24 04:25:41 +00:00
peptiderepo
575d0b97b9 feat: add opik instrumentation to image and content pipeline
- Add optional $trace_context parameter to PRAutoBlogger_Image_Prompt_Builder
- Wrap LLM call in image_prompt_rewrite span when trace context available
- Pass trace context through from image pipeline
- Add $eval_mode parameter to content generator generate() method
- Define PRAUTOBLOGGER_EVAL_MODE constant to suppress side effects
- Add generate_eval() convenience method
- Register WP-CLI hook in main orchestrator
- Bump version to 0.13.0
2026-04-24 04:25:31 +00:00
peptiderepo
50ae7ca530 refactor: extract image scene parser from prompt builder
Split class-image-prompt-builder.php (331 lines) into two files:
- class-image-prompt-builder.php (now ~300 lines)
- class-image-scene-parser.php (~91 lines) with three public static methods:
  - parse_scene_and_caption()
  - extract_first_paragraph()
  - synthesize_visual_concepts_fallback()

Builder now calls these via PRAutoBlogger_Image_Scene_Parser:: static calls.
Maintains identical external behavior; internal refactor only.

Target: prevents CI violations when adding Opik instrumentation.
2026-04-24 04:25:27 +00:00
peptiderepo
c1d22fd275 fix: Opik autoloader gap — services/opik/ not scanned (v0.12.3)
class-autoloader.php listed services/ but not services/opik/. All four Opik classes live in that subdirectory. Composer classmap covered it (tests passed); WP runtime autoloader would fatal in production on first Opik class reference. Adds services/opik/ to the scan list. Bumps to v0.12.3.
2026-04-24 11:08:23 +07:00
peptiderepo
b9dfc29436 fix: image style suffix empty-string bypass + image cost run_id mismatch (v0.12.2)
Bug 1: extract get_style_suffix() helper that treats '' as "use default", applied to all 3 call sites in class-image-prompt-builder.php.

Bug 2: thread $cost_tracker through ArticleWorker → Publisher → PostAssembler → ImagePipeline so image costs share the article's run_id in wp_prautoblogger_generation_log.

Version: 0.12.1 → 0.12.2
Reviewed-by: peptiderepo (CTO)
QA: qa-reviews/prautoblogger/2026-04-23-7669a9c1.md
2026-04-24 01:36:46 +07:00
peptiderepo
58710a0263 fix: use UUID v7 for Opik trace and span IDs (v0.12.1)
Opik requires version 7 UUIDs. wp_generate_uuid4() was producing v4
which the API rejected with 422. Replaced both call sites in
class-opik-trace-context.php with generate_uuid7(), a private static
method implementing RFC 9562 §5.7.

Fixes: smoke test 422 {"errors":["Trace id must be a version 7 UUID"]}
2026-04-23 22:57:28 +07:00
peptiderepo
fbbb387311 feat: Opik LLM observability tracing skeleton (Wave 1) — v0.12.0
Wires PRAutoblogger into Comet Opik cloud for per-call tracing.

- Thin PHP REST client (class-opik-client.php, 276 LOC)
- Async fire-and-forget dispatch via wp_schedule_single_event
- Feature flag PRAUTOBLOGGER_OPIK_ENABLED defaults OFF
- One trace per article, 5 spans across content-generator + chief-editor
- Exponential backoff 2s/4s/8s, max 3 retries, drops on failure
- Admin settings page (api-key configured indicator only, never rendered)
- ARCHITECTURE.md + CONVENTIONS.md updated

PR #113 — QA gate: PASS (qa-reviews/prautoblogger/2026-04-23-0d8dc67.md)
2026-04-23 22:36:24 +07:00
27 changed files with 3430 additions and 202 deletions

View File

@@ -480,6 +480,50 @@ Stored on every PRAutoBlogger-generated post:
---
## Opik LLM Observability (Optional)
**Status:** Wave 1 (v0.12.0) — tracing skeleton. Wave 2 (future) — eval harness with frozen-dataset regression testing.
Opik is a cloud-hosted LLM observability platform (Comet) providing:
- **Per-call tracing:** Every OpenRouter call generates a span with model, tokens, latency metadata.
- **Trace grouping:** One trace per article generation bundles all spans (draft, QA, image prompt).
- **Prompt versioning:** Foundation for Wave 2 — compare new prompt revisions against a frozen reference dataset using LLM-as-judge scoring on rubric axes (factual accuracy, style match, hallucination risk).
**Architecture:**
- **Feature flag:** `prautoblogger_opik_enabled` (WP option, default off). When false, zero network traffic to Opik.
- **Credentials:** API key and workspace from PHP constants `PRAUTOBLOGGER_OPIK_API_KEY`, `PRAUTOBLOGGER_OPIK_WORKSPACE` (defined in wp-config.php). Never stored in DB.
- **Span lifecycle:**
1. `Opik_Trace_Context::init_trace()` generates UUID at article-worker start.
2. Each LLM call wraps with `start_span()` → LLM request → `end_span()` with tokens + response.
3. `finalize_trace()` + `Opik_Span_Queue::enqueue()` accumulates trace + spans for async dispatch.
4. `prautoblogger_opik_dispatch` cron action batches spans/traces (max 100 per POST) to `/v1/private/spans/batch`, `/v1/private/traces/batch`.
- **Retry logic:** Exponential backoff (2s, 4s, 8s) on 5xx errors; permanent abort on 4xx; dropped after 3 attempts per item.
- **Queue:** WP options-backed; max 1000 items; 12-hour TTL; persists across page reloads for truly async dispatch.
**Call sites instrumented (Wave 1):**
- Draft generation (single-pass or multi-step outline/draft/polish).
- Editorial review (chief editor LLM call).
- Image prompt generation (deferred; file at 314 LOC, over limit).
**Cost & Performance:**
- Opik free tier: 25k spans/month. Estimated current usage ~4.2k/month (20 posts/day × 7 calls × 30 days). Headroom to 5× growth.
- Impact on article generation: async dispatch, zero blocking latency overhead.
- Wave 2 (eval harness): adds LLM-as-judge calls (~$14/month at Haiku pricing, assuming 2 eval runs/week).
**Code structure:**
```
includes/services/opik/
class-opik-client.php — REST client (auth, retry, batch POSTs)
class-opik-trace-context.php — per-request trace state singleton
class-opik-span-queue.php — WP options-backed async queue
class-opik-dispatcher.php — cron handler, batch dispatch
includes/admin/
class-opik-settings.php — admin UI (toggle, project name, status)
```
---
## Key Decisions
### #1: OpenRouter over direct provider APIs

View File

@@ -5,6 +5,83 @@ All notable changes to PRAutoBlogger will be documented in this file.
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project uses [Semantic Versioning](https://semver.org/).
## [0.12.3] — 2026-04-24
## [0.13.0] - 2026-04-24
### Added
- **Wave 2 Eval Harness**: Complete evaluation framework for PRAutoBlogger output quality
- Frozen eval dataset (`eval/dataset.json`) with 20 curated peptide topics
- LLM-as-judge scorer (`class-opik-eval-scorer.php`) using Gemini Flash 1.5
- Eval runner orchestrator (`class-opik-eval-runner.php`) with Opik experiment support
- WP-CLI command: `wp prautoblogger opik:eval [--limit=<n>] [--dry-run]`
- Image pipeline instrumentation: Opik span wrapping for `image_prompt_rewrite` step
- **Image Prompt Builder Refactor**: Extracted scene/caption parsing into `PRAutoBlogger_Image_Scene_Parser`
- Three static methods: `parse_scene_and_caption()`, `extract_first_paragraph()`, `synthesize_visual_concepts_fallback()`
- Cleaner separation of concerns; builder now ~255 lines (was 331)
- **Content Generator Eval Mode**: Added `$eval_mode` parameter to suppress publishing and image generation side effects
- **Trace Context Integration**: Optional trace instrumentation in `PRAutoBlogger_Image_Prompt_Builder` constructor
### Changed
- `PRAutoBlogger_Image_Prompt_Builder` constructor now accepts optional `PRAutoBlogger_Opik_Trace_Context` parameter
- `PRAutoBlogger_Image_Pipeline` constructor passes trace context through to prompt builder
- Image prompt rewrite LLM calls now emit Opik spans when trace context is available
### Fixed
- Image prompt builder now properly delegates to static parser methods
### Fixed
- **Opik autoloader gap** — `class-autoloader.php` listed `services/` as a scan directory but not `services/opik/`, where all four Opik classes live (`Opik_Client`, `Opik_Trace_Context`, `Opik_Span_Queue`, `Opik_Dispatcher`). Composer's classmap (used in PHPUnit) covers `includes/` recursively so tests passed, but the WordPress runtime autoloader would fatal on first Opik class reference in production. Added `services/opik/` to the scan list.
## [0.12.2] — 2026-04-24
### Fixed
- **Image style suffix empty-string bypass** — `get_option()` for `prautoblogger_image_style_suffix` only returns the default constant when the key is absent from `wp_options`; an empty string stored in the DB returns `''`, bypassing fallback entirely. Root cause of post #675 rendering photorealistically instead of as a newspaper comic. Fixed all three call sites in `class-image-prompt-builder.php` (lines 85, 107, 272) by extracting a private helper `get_style_suffix()` that treats both absent AND empty string as "use default".
- **Image cost logged under wrong run_id** — `class-image-pipeline.php` created a fresh cost tracker when invoked without an argument, so image costs were logged under a different `run_id` than the article's other stages. Cost breakdowns aggregating by `run_id` silently excluded the image line item. Fixed by threading the article worker's cost tracker through `PRAutoBlogger_Publisher` and `PRAutoBlogger_Post_Assembler::attach_generated_images()` to the image pipeline constructor. Image costs now share the same `run_id` as analysis, draft, review, and llm_research stages.
## [0.12.1] — 2026-04-23
### Fixed
- **Opik UUID v7 bug** — `PRAutoBlogger_Opik_Trace_Context` was generating trace and
span IDs with `wp_generate_uuid4()` (UUID v4). Opik's REST API requires UUID v7 and
rejects v4 with HTTP 400 "Trace id must be a version 7 UUID". Added a private static
`generate_uuid7()` method using `microtime()` + `random_bytes()` and replaced both
call sites. All trace and span submissions now succeed.
## [0.12.0] — 2026-04-23
### Added
- **Opik LLM observability integration (Wave 1: tracing skeleton).** Cloud-hosted observability via Comet Opik for detailed per-call tracing and prompt-version regression testing. Captured via async fire-and-forget dispatch to avoid blocking article generation.
- **Opik REST client.** Minimal HTTP client (no third-party SDK) with exponential backoff retry, batch endpoints, and credential isolation (API key from wp-config.php constants, never in DB).
- **Per-request trace context singleton.** Holds trace ID + span stack for one article generation. UUID threading across draft generation, editorial review, and image prompt generation.
- **Async span/trace queue.** WP options-backed queue with TTL expiry (12 hours) and max depth (1000 items). Drained by cron dispatcher with per-batch retry logic.
- **Cron dispatcher.** `prautoblogger_opik_dispatch` action scheduled via `wp_schedule_single_event()` drains queue in batches of up to 100 spans.
- **Opik admin settings section.** Toggle to enable/disable tracing (default off), project name field, opt-in full-prompt capture, read-only status display (API key configured, last dispatch, queue depth).
- **Span instrumentation at four LLM call sites:** single-pass article generation, outline/draft/polish stages (multi-step), and editorial review. Each span captures model name, provider, usage tokens.
- **Comprehensive unit test suite.** 4 test files covering client (auth, retry, batch limits), trace context (UUID generation, span stacking), queue (enqueue/dequeue, expiry, reenqueue), and dispatcher (batch POST, separation of traces vs. spans).
### Changed
- **class-article-worker.php:** Initializes Opik trace context at generation start, enqueues finalized trace + spans at end if feature enabled.
- **class-content-generator.php:** Wraps single-pass and multi-step stages (outline, draft, polish) with Opik spans capturing token usage.
- **class-chief-editor.php:** Wraps editorial review LLM call with Opik span.
### Notes
- **Feature-flag default OFF.** Zero network traffic to Opik unless `prautoblogger_opik_enabled` option is true AND API credentials are defined in wp-config.php.
- **Image prompt generation instrumentation deferred.** File already at 314 LOC (over 300-line rule); can be instrumented in Wave 2 without modifying existing code.
- **Wave 2 holds eval harness.** CLI command for frozen-dataset regression testing with LLM-as-judge scoring (separate thread after v0.12.0 ships and is live-tested).
## [0.11.0] — 2026-04-23
### Added

View File

@@ -332,4 +332,104 @@ curl -s -X POST -H "Authorization: token $GH_TOKEN" \
### When this changes
If the peptiderepo GitHub account is ever upgraded to Pro (or the repo goes public), replace this soft-enforcement section with a note pointing to the real branch protection rules in repo settings.
If the peptiderepo GitHub account is ever upgraded to Pro (or the repo goes public), replace this soft-enforcement section with a note pointing to the real branch protection rules in repo settings.
## Instrumenting a New LLM Call Site for Opik (v0.12.0+)
When you add a new LLM call (via `$llm->send_chat_completion()`), follow this pattern to capture it in Opik:
### 1. Get the trace context
At the start of your method, grab the per-request singleton:
```php
$ctx = PRAutoBlogger_Opik_Trace_Context::current();
```
### 2. Start the span
Before the LLM call, declare a span with metadata:
```php
$span_id = $ctx->start_span(
array(
'name' => 'my_llm_operation', // kebab-case, descriptive
'type' => 'llm',
'model' => $model, // e.g. 'gpt-4', from get_option()
'provider' => 'openrouter', // or the actual provider name
'input' => array( 'key' => 'value' ), // optional metadata (no full prompts!)
)
);
```
### 3. Make the LLM call (unchanged)
```php
$response = $this->llm->send_chat_completion(
array(
array( 'role' => 'system', 'content' => $system_prompt ),
array( 'role' => 'user', 'content' => $user_prompt ),
),
$model,
array( 'temperature' => 0.7, 'max_tokens' => 4000 )
);
```
### 4. End the span with output
After the response, record the outcome:
```php
$ctx->end_span(
$span_id,
array(
'output' => array( 'response_length' => strlen( $response['content'] ?? '' ) ),
'usage' => array(
'prompt_tokens' => $response['prompt_tokens'],
'completion_tokens' => $response['completion_tokens'],
'total_tokens' => $response['prompt_tokens'] + $response['completion_tokens'],
),
)
);
```
### Key rules
- **Never log full prompts.** Use prompt hashes or version tags in `input`. Full text blows payload size and leaks content policy.
- **Check the feature flag at call-site level?** No. The singleton is always available; Opik_Dispatcher checks the flag before dispatch.
- **Async dispatch?** Yes. Spans are queued; dispatch happens via `prautoblogger_opik_dispatch` cron (no blocking).
- **Cost logging?** Unchanged. Your `cost_tracker->log_api_call()` remains separate from Opik; Opik is observability-only.
### Example: minimal instrumentation
```php
public function my_operation( $input ) {
$model = get_option( 'my_model', 'default-model' );
$ctx = PRAutoBlogger_Opik_Trace_Context::current();
$span_id = $ctx->start_span(
array(
'name' => 'my_operation',
'type' => 'llm',
'model' => $model,
'provider' => 'openrouter',
)
);
$response = $this->llm->send_chat_completion( /* ... */ );
$ctx->end_span(
$span_id,
array(
'usage' => array(
'prompt_tokens' => $response['prompt_tokens'],
'completion_tokens' => $response['completion_tokens'],
'total_tokens' => $response['prompt_tokens'] + $response['completion_tokens'],
),
)
);
return $response['content'];
}
```
That's it. No config, no feature-flag checks, no try/catch needed.

142
eval/dataset.json Normal file
View File

@@ -0,0 +1,142 @@
[
{
"id": "bpc157-healing",
"topic": "BPC-157 healing protocols for tendons and gut lining",
"expected_keywords": ["BPC-157", "tendon", "gut", "healing"],
"min_words": 700,
"style": "comic"
},
{
"id": "tb500-muscle-recovery",
"topic": "TB-500 for accelerated muscle recovery and repair",
"expected_keywords": ["TB-500", "muscle", "recovery", "growth"],
"min_words": 700,
"style": "comic"
},
{
"id": "ghk-cu-skin-collagen",
"topic": "GHK-Cu peptide for skin collagen synthesis and anti-aging",
"expected_keywords": ["GHK-Cu", "collagen", "skin", "aging"],
"min_words": 700,
"style": "comic"
},
{
"id": "semaglutide-glp1-weight",
"topic": "GLP-1 receptor agonists like Semaglutide for metabolic health",
"expected_keywords": ["Semaglutide", "GLP-1", "weight", "metabolism"],
"min_words": 700,
"style": "comic"
},
{
"id": "ipamorelin-growth-hormone",
"topic": "Ipamorelin for natural growth hormone stimulation and recovery",
"expected_keywords": ["Ipamorelin", "growth hormone", "HGH", "secretagogue"],
"min_words": 700,
"style": "comic"
},
{
"id": "hexarelin-strength-gains",
"topic": "Hexarelin peptide for lean muscle and strength improvement",
"expected_keywords": ["Hexarelin", "muscle", "strength", "lean"],
"min_words": 700,
"style": "comic"
},
{
"id": "semax-cognitive-enhancement",
"topic": "Semax nootropic peptide for cognitive function and focus",
"expected_keywords": ["Semax", "cognition", "focus", "neuroprotection"],
"min_words": 700,
"style": "comic"
},
{
"id": "selank-anxiety-stress",
"topic": "Selank peptide for anxiety reduction and stress management",
"expected_keywords": ["Selank", "anxiety", "stress", "mood"],
"min_words": 700,
"style": "comic"
},
{
"id": "oxytocin-social-bonding",
"topic": "Oxytocin peptide hormone and its role in social connection",
"expected_keywords": ["Oxytocin", "social", "bonding", "neuropeptide"],
"min_words": 700,
"style": "comic"
},
{
"id": "kisspeptin-fertility-hormone",
"topic": "Kisspeptin's role in reproductive health and hormone signaling",
"expected_keywords": ["Kisspeptin", "fertility", "hormone", "reproduction"],
"min_words": 700,
"style": "comic"
},
{
"id": "melanotan-tanning-peptide",
"topic": "Melanotan peptide for pigmentation and UV protection",
"expected_keywords": ["Melanotan", "melanin", "tanning", "skin"],
"min_words": 700,
"style": "comic"
},
{
"id": "cerebrolysin-neuroprotection",
"topic": "Cerebrolysin for brain health and neuroprotection",
"expected_keywords": ["Cerebrolysin", "brain", "neuroprotection", "cognition"],
"min_words": 700,
"style": "comic"
},
{
"id": "epithalon-telomerase-aging",
"topic": "Epithalon peptide and telomerase activation for longevity",
"expected_keywords": ["Epithalon", "telomerase", "aging", "longevity"],
"min_words": 700,
"style": "comic"
},
{
"id": "thymosin-immune-function",
"topic": "Thymosin alpha-1 for immune system enhancement",
"expected_keywords": ["Thymosin", "immune", "T-cells", "immunity"],
"min_words": 700,
"style": "comic"
},
{
"id": "igf1-muscle-growth-aging",
"topic": "IGF-1 peptide for muscle growth, recovery, and anti-aging",
"expected_keywords": ["IGF-1", "muscle", "growth", "aging"],
"min_words": 700,
"style": "comic"
},
{
"id": "mechano-growth-factor",
"topic": "MGF (Mechano Growth Factor) for muscle hypertrophy",
"expected_keywords": ["MGF", "mechano", "muscle", "hypertrophy"],
"min_words": 700,
"style": "comic"
},
{
"id": "ll37-antimicrobial-peptide",
"topic": "LL-37 antimicrobial peptide for immune and skin health",
"expected_keywords": ["LL-37", "antimicrobial", "immune", "peptide"],
"min_words": 700,
"style": "comic"
},
{
"id": "lactoferrin-iron-immunity",
"topic": "Lactoferrin peptide for iron absorption and immune support",
"expected_keywords": ["Lactoferrin", "iron", "immune", "absorption"],
"min_words": 700,
"style": "comic"
},
{
"id": "collagen-peptides-joints",
"topic": "Collagen peptides for joint health and connective tissue repair",
"expected_keywords": ["Collagen", "peptides", "joints", "connective tissue"],
"min_words": 700,
"style": "comic"
},
{
"id": "ckgp-cognitive-longevity",
"topic": "Cognitive peptides and nootropic stacks for brain longevity",
"expected_keywords": ["peptide", "cognitive", "nootropic", "brain"],
"min_words": 700,
"style": "comic"
}
]

View File

@@ -0,0 +1,204 @@
<?php
declare(strict_types=1);
/**
* Opik settings UI and registration.
*
* Registers settings fields in the PRAutoBlogger Settings page under
* a new "Observability" section. Fields include:
* - Enable Opik tracing (checkbox, default off)
* - Opik project name (text, default 'prautoblogger')
* - Capture full prompts (checkbox, default off)
* - Status display (read-only: API key configured, last dispatch, queue depth)
*
* Never renders the API key value in admin HTML.
*
* @see includes/admin/class-admin-page.php — Main settings page loader.
*/
class PRAutoBlogger_Opik_Settings {
/**
* Constructor.
*
* Register hooks in __construct so they fire when the plugin loads.
*/
public function __construct() {
add_action( 'admin_init', array( $this, 'register_settings' ) );
}
/**
* Register Opik settings section and fields.
*
* Hooked to 'admin_init'.
*/
public function register_settings(): void {
// Register the section.
add_settings_section(
'prautoblogger_observability',
__( 'Observability (Opik)', 'prautoblogger' ),
array( $this, 'render_section_help' ),
'prautoblogger_settings'
);
// Register fields.
add_settings_field(
'prautoblogger_opik_enabled',
__( 'Enable Opik tracing', 'prautoblogger' ),
array( $this, 'render_field_enabled' ),
'prautoblogger_settings',
'prautoblogger_observability'
);
register_setting(
'prautoblogger_settings',
'prautoblogger_opik_enabled',
array( 'type' => 'boolean' )
);
add_settings_field(
'prautoblogger_opik_project_name',
__( 'Opik project name', 'prautoblogger' ),
array( $this, 'render_field_project_name' ),
'prautoblogger_settings',
'prautoblogger_observability'
);
register_setting(
'prautoblogger_settings',
'prautoblogger_opik_project_name',
array(
'type' => 'string',
'sanitize_callback' => 'sanitize_text_field',
)
);
add_settings_field(
'prautoblogger_opik_capture_full_prompt',
__( 'Capture full prompt text', 'prautoblogger' ),
array( $this, 'render_field_capture_full' ),
'prautoblogger_settings',
'prautoblogger_observability'
);
register_setting(
'prautoblogger_settings',
'prautoblogger_opik_capture_full_prompt',
array( 'type' => 'boolean' )
);
add_settings_field(
'prautoblogger_opik_status',
__( 'Status', 'prautoblogger' ),
array( $this, 'render_field_status' ),
'prautoblogger_settings',
'prautoblogger_observability'
);
}
/**
* Render section help text.
*/
public function render_section_help(): void {
echo wp_kses_post(
'<p>Optional LLM observability via <a href="https://www.comet.com/" target="_blank" rel="noopener">Comet Opik</a>. ' .
'Captures detailed traces of article generation with per-LLM-call metrics. ' .
'Requires API credentials in <code>wp-config.php</code>.</p>'
);
}
/**
* Render the "Enable Opik" checkbox.
*/
public function render_field_enabled(): void {
$enabled = get_option( 'prautoblogger_opik_enabled', false );
$checked = $enabled ? 'checked' : '';
?>
<input type="checkbox" name="prautoblogger_opik_enabled" value="1" <?php echo esc_attr( $checked ); ?> />
<label for="prautoblogger_opik_enabled">
<?php esc_html_e( 'Send article generation traces to Opik', 'prautoblogger' ); ?>
</label>
<?php
if ( ! $this->has_credentials() ) {
echo '<p style="color: #d63638;"><strong>' .
esc_html__( 'Note: API credentials must be set in wp-config.php (PRAUTOBLOGGER_OPIK_API_KEY, PRAUTOBLOGGER_OPIK_WORKSPACE)', 'prautoblogger' ) .
'</strong></p>';
}
}
/**
* Render the project name field.
*/
public function render_field_project_name(): void {
$value = get_option( 'prautoblogger_opik_project_name', 'prautoblogger' );
?>
<input type="text" name="prautoblogger_opik_project_name" value="<?php echo esc_attr( $value ); ?>" style="width: 300px;" />
<p class="description">
<?php esc_html_e( 'Opik auto-creates this project on first trace (lowercase, no spaces)', 'prautoblogger' ); ?>
</p>
<?php
}
/**
* Render the "Capture full prompt" checkbox.
*/
public function render_field_capture_full(): void {
$enabled = get_option( 'prautoblogger_opik_capture_full_prompt', false );
$checked = $enabled ? 'checked' : '';
?>
<input type="checkbox" name="prautoblogger_opik_capture_full_prompt" value="1" <?php echo esc_attr( $checked ); ?> />
<label>
<?php esc_html_e( 'Include full prompt text in spans (increases payload size)', 'prautoblogger' ); ?>
</label>
<p class="description">
<?php esc_html_e( 'By default, only prompt hash is logged. Enable this for debugging.' ); ?>
</p>
<?php
}
/**
* Render the status display (read-only).
*/
public function render_field_status(): void {
$has_api_key = $this->has_credentials();
$queue = new PRAutoBlogger_Opik_Span_Queue();
$queue_depth = $queue->get_queue_depth();
$last_dispatch = get_option( 'prautoblogger_opik_last_dispatch', 0 );
$last_dispatch_str = $last_dispatch
? sprintf(
/* translators: %s = human-readable time */
__( '%s ago', 'prautoblogger' ),
human_time_diff( (int) $last_dispatch )
)
: __( 'Never', 'prautoblogger' );
?>
<div style="background: #f1f1f1; padding: 10px; border-radius: 3px; max-width: 600px;">
<p>
<strong><?php esc_html_e( 'API key configured:', 'prautoblogger' ); ?></strong>
<?php echo $has_api_key ? '<span style="color: green;">✓ Yes</span>' : '<span style="color: red;">✗ No</span>'; ?>
</p>
<p>
<strong><?php esc_html_e( 'Last dispatch:', 'prautoblogger' ); ?></strong>
<?php echo esc_html( $last_dispatch_str ); ?>
</p>
<p>
<strong><?php esc_html_e( 'Queue depth:', 'prautoblogger' ); ?></strong>
<?php echo esc_html( (string) $queue_depth ); ?>
<?php esc_html_e( 'item(s) pending', 'prautoblogger' ); ?>
</p>
</div>
<?php
}
/**
* Check if Opik credentials are configured in wp-config.php.
*
* @return bool
*/
private function has_credentials(): bool {
return defined( 'PRAUTOBLOGGER_OPIK_API_KEY' ) &&
defined( 'PRAUTOBLOGGER_OPIK_WORKSPACE' ) &&
! empty( PRAUTOBLOGGER_OPIK_API_KEY ) &&
! empty( PRAUTOBLOGGER_OPIK_WORKSPACE );
}
}

View File

@@ -0,0 +1,72 @@
<?php
declare(strict_types=1);
/**
* WP-CLI commands for PRAutoBlogger.
*
* Registers custom wp-cli commands exposed to the plugin.
*/
class PRAutoBlogger_WP_CLI_Commands {
/**
* Register WP-CLI commands.
*
* Called on plugins_loaded hook.
*/
public static function register(): void {
if ( ! defined( 'WP_CLI' ) || ! WP_CLI ) {
return;
}
// Command: wp prautoblogger opik:eval
\WP_CLI::add_command(
'prautoblogger opik:eval',
array( self::class, 'opik_eval_command' ),
array(
'shortdesc' => 'Run Opik evals on the frozen dataset',
'synopsis' => array(
array(
'type' => 'assoc',
'name' => 'limit',
'description' => 'Max items to run (0 = all)',
'optional' => true,
'default' => '0',
),
array(
'type' => 'flag',
'name' => 'dry-run',
'description' => 'Skip Opik API push; score locally only',
'optional' => true,
),
),
)
);
}
/**
* Run eval on the frozen dataset.
*
* @param array $args Positional args (unused).
* @param array $assoc_args Associative args: limit, dry-run.
*
* @return void
*/
public static function opik_eval_command( array $args, array $assoc_args ): void {
// Check if Opik is enabled.
if ( empty( PRAUTOBLOGGER_OPIK_API_KEY ) || empty( PRAUTOBLOGGER_OPIK_WORKSPACE ) ) {
if ( ! isset( $assoc_args['dry-run'] ) ) {
// phpcs:ignore WordPress.PHP.DevelopmentFunctions.error_log_echo -- CLI output
echo "Warning: Opik is not configured. Running with --dry-run only.\n";
$assoc_args['dry-run'] = true;
}
}
$limit = absint( $assoc_args['limit'] ?? 0 );
$dry_run = isset( $assoc_args['dry-run'] );
$runner = new PRAutoBlogger_Opik_Eval_Runner();
$result = $runner->run( $limit, $dry_run );
exit( $result['items_run'] > 0 ? 0 : 1 );
}
}

View File

@@ -30,6 +30,7 @@ class PRAutoBlogger_Autoloader {
'models/',
'frontend/',
'services/',
'services/opik/',
'ajax/',
);

View File

@@ -55,6 +55,7 @@ class PRAutoBlogger {
$this->register_cron_hooks();
$this->register_frontend_hooks();
$this->register_ajax_hooks();
$this->register_cli_hooks();
/** Fires after PRAutoBlogger has finished registering all hooks. */
do_action( 'prautoblogger_loaded' );
@@ -116,6 +117,30 @@ class PRAutoBlogger {
$registry = $this->executor->get_model_registry();
add_action( 'prautoblogger_refresh_model_registry', array( $registry, 'refresh' ) );
// Opik observability: async trace/span dispatch.
add_action(
'prautoblogger_opik_dispatch',
static function (): void {
if ( ! defined( 'PRAUTOBLOGGER_OPIK_API_KEY' ) || empty( PRAUTOBLOGGER_OPIK_API_KEY ) ) {
return;
}
$client = new PRAutoBlogger_Opik_Client(
PRAUTOBLOGGER_OPIK_API_KEY,
PRAUTOBLOGGER_OPIK_WORKSPACE,
PRAUTOBLOGGER_OPIK_URL_OVERRIDE
);
$queue = new PRAutoBlogger_Opik_Span_Queue();
$dispatcher = new PRAutoBlogger_Opik_Dispatcher( $client, $queue );
$dispatcher->dispatch();
update_option( 'prautoblogger_opik_last_dispatch', time() );
}
);
// Opik settings registration.
if ( is_admin() ) {
new PRAutoBlogger_Opik_Settings();
}
// Single-idea generation from the Ideas browser page.
add_action( 'prautoblogger_generate_from_idea', array( 'PRAutoBlogger_Ideas_Browser', 'on_cron_generate_from_idea' ) );
@@ -270,6 +295,11 @@ class PRAutoBlogger {
return $schedules;
}
/** Register WP-CLI commands. */
private function register_cli_hooks(): void {
PRAutoBlogger_WP_CLI_Commands::register();
}
/**
* Block false update notifications from wordpress.org.
* Our plugin slug may collide with a different plugin in the directory.

View File

@@ -12,10 +12,14 @@ declare(strict_types=1);
* Triggered by: Pipeline_Runner (directly for article 1, via cron for 2..N).
* Dependencies: Content_Generator, Chief_Editor, Publisher, Cost_Tracker.
*
* Opik instrumentation: Initializes trace context at start, creates spans
* for content generation and editorial review. Feature-flag gated (default off).
*
* @see core/class-pipeline-runner.php — Orchestrates and dispatches workers.
* @see core/class-content-generator.php — LLM content generation.
* @see core/class-chief-editor.php — Editorial quality gate.
* @see core/class-publisher.php — WordPress post creation.
* @see services/opik/class-opik-trace-context.php — Opik tracing.
*/
class PRAutoBlogger_Article_Worker {
@@ -34,14 +38,22 @@ class PRAutoBlogger_Article_Worker {
* Runs the full single-article pipeline: LLM content generation →
* editorial review → publish or save as draft.
*
* Initializes Opik trace context if enabled and tears it down at end.
*
* Side effects: LLM API calls, database writes, WordPress post creation,
* image generation, cost logging.
* image generation, cost logging, Opik span queuing.
*
* @param PRAutoBlogger_Article_Idea $idea The scored idea to generate from.
*
* @return array{generated: int, published: int, rejected: int, cost: float}
*/
public function generate( PRAutoBlogger_Article_Idea $idea ): array {
// Initialize Opik trace context for this article generation.
$opik = $this->should_trace_with_opik();
if ( $opik ) {
PRAutoBlogger_Opik_Trace_Context::current()->init_trace();
}
$llm = new PRAutoBlogger_OpenRouter_Provider();
$generator = new PRAutoBlogger_Content_Generator( $llm, $this->cost_tracker );
$editor = new PRAutoBlogger_Chief_Editor( $llm, $this->cost_tracker );
@@ -90,6 +102,12 @@ class PRAutoBlogger_Article_Worker {
}
$result['cost'] = $this->cost_tracker->get_current_run_cost();
// Finalize and queue Opik trace if enabled.
if ( $opik ) {
$this->finalize_opik_trace();
}
return $result;
}
@@ -120,13 +138,13 @@ class PRAutoBlogger_Article_Worker {
: $content;
if ( $auto_publish ) {
$publisher->publish( $final, $idea, $review, $run_id );
$publisher->publish( $final, $idea, $review, $run_id, $this->cost_tracker );
++$result['published'];
} else {
$publisher->save_as_draft( $final, $idea, $review, $run_id );
$publisher->save_as_draft( $final, $idea, $review, $run_id, $this->cost_tracker );
}
} else {
$publisher->save_as_draft( $content, $idea, $review, $run_id );
$publisher->save_as_draft( $content, $idea, $review, $run_id, $this->cost_tracker );
++$result['rejected'];
PRAutoBlogger_Logger::instance()->info(
'Article rejected by editor: ' . $idea->get_topic(),
@@ -149,4 +167,47 @@ class PRAutoBlogger_Article_Worker {
set_transient( $transient_key, $current, 600 );
}
}
/**
* Check if Opik tracing is enabled and has credentials.
*
* @return bool
*/
private function should_trace_with_opik(): bool {
if ( ! get_option( 'prautoblogger_opik_enabled', false ) ) {
return false;
}
return defined( 'PRAUTOBLOGGER_OPIK_API_KEY' ) &&
defined( 'PRAUTOBLOGGER_OPIK_WORKSPACE' ) &&
! empty( PRAUTOBLOGGER_OPIK_API_KEY ) &&
! empty( PRAUTOBLOGGER_OPIK_WORKSPACE );
}
/**
* Finalize and queue the Opik trace.
*
* Dispatches trace to queue for async posting.
*/
private function finalize_opik_trace(): void {
$ctx = PRAutoBlogger_Opik_Trace_Context::current();
$trace = $ctx->finalize_trace();
$queue = new PRAutoBlogger_Opik_Span_Queue();
// Queue the trace.
$queue->enqueue( $trace, 'trace' );
// Queue all spans.
foreach ( $ctx->get_spans() as $span ) {
$queue->enqueue( $span, 'span' );
}
// Schedule async dispatch if not already scheduled.
if ( ! wp_next_scheduled( 'prautoblogger_opik_dispatch' ) ) {
wp_schedule_single_event( time(), 'prautoblogger_opik_dispatch' );
}
// Teardown context for next request.
PRAutoBlogger_Opik_Trace_Context::teardown();
}
}

View File

@@ -2,21 +2,14 @@
declare(strict_types=1);
/**
* phpcs:ignore WordPress.Files.FileName.InvalidClassFileName -- class naming convention differs from WordPress standard
* Chief Editor agent — LLM-powered editorial review.
*
* Chief Editor agent — LLM-powered editorial review of generated content.
* Reviews generated articles for quality, accuracy, SEO, tone.
* Approves, requests revisions, or rejects content.
* Opik instrumentation: spans per review LLM call.
*
* Reviews articles for quality, accuracy, SEO, and tone before publishing.
* Can approve, request revisions (and produce them), or reject content.
* Rejected content is saved as a draft for human review.
*
* Triggered by: PRAutoBlogger::run_generation_pipeline() (step 5).
* Triggered by: PRAutoBlogger_Article_Worker.
* Dependencies: LLM_Provider_Interface, Cost_Tracker.
*
* @see core/class-content-generator.php — Produces the content we review.
* @see core/class-json-extractor.php — Tolerant JSON parsing for LLM output.
* @see core/class-publisher.php — Publishes approved content.
* @see ARCHITECTURE.md — Data flow step 5.
*/
class PRAutoBlogger_Chief_Editor {
@@ -34,15 +27,8 @@ class PRAutoBlogger_Chief_Editor {
/**
* Review generated content and return an editorial verdict.
*
* The editor evaluates: content quality, factual tone, SEO elements,
* readability, and alignment with the article idea. It can approve,
* revise (and return revised content), or reject.
*
* Side effects: LLM API call(s), cost logging.
*
* @param string $content The generated HTML content.
* @param PRAutoBlogger_Article_Idea $idea The original article idea.
*
* @param PRAutoBlogger_Article_Idea $idea The original article idea.
* @return PRAutoBlogger_Editorial_Review
*/
public function review( string $content, PRAutoBlogger_Article_Idea $idea ): PRAutoBlogger_Editorial_Review {
@@ -52,6 +38,16 @@ class PRAutoBlogger_Chief_Editor {
$system_prompt = $this->build_system_prompt( $niche );
$user_prompt = $this->build_review_prompt( $content, $idea );
$ctx = PRAutoBlogger_Opik_Trace_Context::current();
$span_id = $ctx->start_span(
array(
'name' => 'editorial_review',
'type' => 'llm',
'model' => $model,
'provider' => 'openrouter',
)
);
$response = $this->llm->send_chat_completion(
array(
array(
@@ -71,6 +67,17 @@ class PRAutoBlogger_Chief_Editor {
)
);
$ctx->end_span(
$span_id,
array(
'usage' => array(
'prompt_tokens' => $response['prompt_tokens'],
'completion_tokens' => $response['completion_tokens'],
'total_tokens' => $response['prompt_tokens'] + $response['completion_tokens'],
),
)
);
$this->cost_tracker->log_api_call(
null,
'review',
@@ -84,42 +91,31 @@ class PRAutoBlogger_Chief_Editor {
}
/**
* @param string $niche
*
* @return string
* Build system prompt for editorial review.
*/
private function build_system_prompt( string $niche ): string {
$prompt = 'You are a senior blog editor';
if ( '' !== $niche ) {
$prompt .= " specializing in {$niche} content";
}
$prompt .= ". Your job is to review article drafts before publication.\n\n";
$prompt .= ". Review article drafts before publication.\n\n";
$prompt .= "Evaluate the article on:\n";
$prompt .= "1. QUALITY: Is the writing clear, engaging, and well-structured?\n";
$prompt .= "2. ACCURACY: Does the content avoid making unsupported health/medical claims? Is the tone appropriately cautious for the topic?\n";
$prompt .= "3. SEO: Are headings properly structured? Are target keywords naturally used?\n";
$prompt .= "4. COMPLETENESS: Does it cover the key points? Does it have a proper intro and conclusion?\n";
$prompt .= "5. READABILITY: Is sentence variety good? Are paragraphs appropriate length?\n\n";
$prompt .= "You must respond with JSON in this format:\n";
$prompt .= "{\n";
$prompt .= "Evaluate on: QUALITY, ACCURACY, SEO, COMPLETENESS, READABILITY.\n";
$prompt .= "Respond with JSON: {\n";
$prompt .= ' "verdict": "approved" | "revised" | "rejected",' . "\n";
$prompt .= ' "quality_score": 0.0-1.0,' . "\n";
$prompt .= ' "seo_score": 0.0-1.0,' . "\n";
$prompt .= ' "issues": ["issue 1", "issue 2"],' . "\n";
$prompt .= ' "notes": "Brief editorial notes",' . "\n";
$prompt .= ' "revised_content": "Full revised HTML if verdict is revised, null otherwise"' . "\n";
$prompt .= ' "issues": ["issue1", "issue2"],' . "\n";
$prompt .= ' "notes": "Editorial notes",' . "\n";
$prompt .= ' "revised_content": "Full revised HTML if revised, null otherwise"' . "\n";
$prompt .= "}\n\n";
$prompt .= "Rules:\n";
$prompt .= "- APPROVE if quality_score >= 0.7 and seo_score >= 0.6 and no critical issues\n";
$prompt .= "- REVISE if fixable issues exist — you MUST provide the full revised HTML\n";
$prompt .= "- REJECT only if the content is fundamentally flawed or off-topic\n";
$prompt .= "- When revising, fix the issues yourself. Don't just describe them.\n";
$prompt .= "- When revising, PRESERVE all bullet points, numbered lists, hyperlinks, and structural formatting from the original. Do NOT flatten lists into prose or remove links.\n";
$prompt .= "- APPROVE if quality_score >= 0.7 and seo_score >= 0.6\n";
$prompt .= "- REVISE if fixable issues exist — provide full revised HTML\n";
$prompt .= "- REJECT if fundamentally flawed\n";
$prompt .= "- Preserve formatting, links, lists when revising\n";
// Append user-defined editor instructions if configured.
$instructions = trim( (string) get_option( 'prautoblogger_editor_instructions', '' ) );
if ( '' !== $instructions ) {
$prompt .= "\nAdditional instructions:\n" . $instructions . "\n";
@@ -129,21 +125,15 @@ class PRAutoBlogger_Chief_Editor {
}
/**
* @param string $content
* @param PRAutoBlogger_Article_Idea $idea
*
* @return string
* Build user prompt for editorial review.
*/
private function build_review_prompt( string $content, PRAutoBlogger_Article_Idea $idea ): string {
return sprintf(
"Review this article draft:\n\n" .
"ORIGINAL BRIEF:\n" .
"- Title: %s\n" .
"- Topic: %s\n" .
"- Type: %s\n" .
"- Key points: %s\n" .
"- Target keywords: %s\n\n" .
"ARTICLE CONTENT:\n%s",
"BRIEF: Title: %s | Topic: %s | Type: %s\n" .
"KEY POINTS: %s\n" .
"TARGET KEYWORDS: %s\n\n" .
"CONTENT:\n%s",
$idea->get_suggested_title(),
$idea->get_topic(),
$idea->get_article_type(),
@@ -154,25 +144,20 @@ class PRAutoBlogger_Chief_Editor {
}
/**
* Parse the LLM's editorial review response.
*
* @param string $content JSON response from LLM.
*
* @return PRAutoBlogger_Editorial_Review
* Parse LLM editorial review response.
*/
private function parse_review_response( string $content ): PRAutoBlogger_Editorial_Review {
$data = PRAutoBlogger_Json_Extractor::decode( $content );
if ( ! is_array( $data ) || ! isset( $data['verdict'] ) ) {
PRAutoBlogger_Logger::instance()->error(
'Chief editor response was not valid JSON. Defaulting to rejected. Raw (first 500 chars): '
. mb_substr( $content, 0, 500 ),
'Chief editor response unparseable (first 200 chars): ' . mb_substr( $content, 0, 200 ),
'editor'
);
return new PRAutoBlogger_Editorial_Review(
array(
'verdict' => 'rejected',
'notes' => 'Editor response could not be parsed.',
'notes' => 'Editor response unparseable',
'quality_score' => 0.0,
'seo_score' => 0.0,
'issues' => array( 'Unparseable editor response' ),

View File

@@ -2,21 +2,13 @@
declare(strict_types=1);
/**
* phpcs:ignore WordPress.Files.FileName.InvalidClassFileName -- class naming convention differs from WordPress standard
*
* Writer agent — generates blog post content from article ideas.
*
* Supports two configurable pipeline modes:
* - single_pass: One LLM call produces the complete article (cheaper, faster).
* - multi_step: Outline → Draft → Polish pipeline (higher quality, more cost).
* Supports: single_pass (one LLM call) or multi_step (outline → draft → polish).
* Opik instrumentation: spans per LLM call when feature-flag enabled.
*
* Triggered by: PRAutoBlogger::run_generation_pipeline() (step 4).
* Triggered by: PRAutoBlogger_Article_Worker.
* Dependencies: LLM_Provider_Interface, Cost_Tracker, Content_Prompts.
*
* @see core/class-content-prompts.php — Builds all LLM prompts used here.
* @see core/class-chief-editor.php — Reviews output of this class.
* @see providers/interface-llm-provider.php — LLM used for generation.
* @see ARCHITECTURE.md — Data flow step 4.
*/
class PRAutoBlogger_Content_Generator {
@@ -34,14 +26,13 @@ class PRAutoBlogger_Content_Generator {
/**
* Generate a blog post from an article idea.
*
* Reads pipeline mode from settings and dispatches to the appropriate method.
* Side effects: LLM API calls (1-3 depending on mode), cost logging.
*
* @param PRAutoBlogger_Article_Idea $idea The scored idea to generate content for.
* @return string Generated HTML content.
* @throws \RuntimeException On generation failure.
*/
public function generate( PRAutoBlogger_Article_Idea $idea ): string {
public function generate( PRAutoBlogger_Article_Idea $idea, bool $eval_mode = false ): string {
// In eval mode, suppress publishing and image generation side effects.
define( 'PRAUTOBLOGGER_EVAL_MODE', $eval_mode );
$mode = get_option( 'prautoblogger_writing_pipeline', 'multi_step' );
$request = new PRAutoBlogger_Content_Request(
@@ -55,21 +46,26 @@ class PRAutoBlogger_Content_Generator {
get_option( 'prautoblogger_writing_instructions', '' )
);
if ( 'single_pass' === $mode ) {
return $this->generate_single_pass( $request );
}
return $this->generate_multi_step( $request );
return 'single_pass' === $mode
? $this->generate_single_pass( $request )
: $this->generate_multi_step( $request );
}
/**
* Single-pass generation: one LLM call produces the complete article.
*
* @param PRAutoBlogger_Content_Request $request Content request with settings.
* @return string Generated HTML content.
* Single-pass: one LLM call produces complete article.
*/
private function generate_single_pass( PRAutoBlogger_Content_Request $request ): string {
$model = get_option( 'prautoblogger_writing_model', PRAUTOBLOGGER_DEFAULT_WRITING_MODEL );
$ctx = PRAutoBlogger_Opik_Trace_Context::current();
$span_id = $ctx->start_span(
array(
'name' => 'single_pass_generation',
'type' => 'llm',
'model' => $model,
'provider' => 'openrouter',
)
);
$response = $this->llm->send_chat_completion(
array(
@@ -89,6 +85,17 @@ class PRAutoBlogger_Content_Generator {
)
);
$ctx->end_span(
$span_id,
array(
'usage' => array(
'prompt_tokens' => $response['prompt_tokens'],
'completion_tokens' => $response['completion_tokens'],
'total_tokens' => $response['prompt_tokens'] + $response['completion_tokens'],
),
)
);
$this->cost_tracker->log_api_call(
null,
'draft',
@@ -101,7 +108,9 @@ class PRAutoBlogger_Content_Generator {
return $response['content'];
}
/** Multi-step generation: Outline → Draft → Polish. */
/**
* Multi-step: outline → draft → polish.
*/
private function generate_multi_step( PRAutoBlogger_Content_Request $request ): string {
$model = get_option( 'prautoblogger_writing_model', PRAUTOBLOGGER_DEFAULT_WRITING_MODEL );
@@ -112,15 +121,26 @@ class PRAutoBlogger_Content_Generator {
return $polished;
}
/** Multi-step stage 1: Generate an article outline. */
/**
* Stage 1: Generate article outline.
*/
private function stage_outline( PRAutoBlogger_Content_Request $request, string $model ): string {
$system = PRAutoBlogger_Content_Prompts::build_system( $request );
$ctx = PRAutoBlogger_Opik_Trace_Context::current();
$span_id = $ctx->start_span(
array(
'name' => 'outline_generation',
'type' => 'llm',
'model' => $model,
'provider' => 'openrouter',
)
);
$response = $this->llm->send_chat_completion(
array(
array(
'role' => 'system',
'content' => $system,
'content' => PRAutoBlogger_Content_Prompts::build_system( $request ),
),
array(
'role' => 'user',
@@ -134,6 +154,17 @@ class PRAutoBlogger_Content_Generator {
)
);
$ctx->end_span(
$span_id,
array(
'usage' => array(
'prompt_tokens' => $response['prompt_tokens'],
'completion_tokens' => $response['completion_tokens'],
'total_tokens' => $response['prompt_tokens'] + $response['completion_tokens'],
),
)
);
$this->cost_tracker->log_api_call(
null,
'outline',
@@ -146,15 +177,26 @@ class PRAutoBlogger_Content_Generator {
return $response['content'];
}
/** Multi-step stage 2: Write the full draft from the outline. */
/**
* Stage 2: Write full draft from outline.
*/
private function stage_draft( PRAutoBlogger_Content_Request $request, string $outline, string $model ): string {
$system = PRAutoBlogger_Content_Prompts::build_system( $request );
$ctx = PRAutoBlogger_Opik_Trace_Context::current();
$span_id = $ctx->start_span(
array(
'name' => 'draft_generation',
'type' => 'llm',
'model' => $model,
'provider' => 'openrouter',
)
);
$response = $this->llm->send_chat_completion(
array(
array(
'role' => 'system',
'content' => $system,
'content' => PRAutoBlogger_Content_Prompts::build_system( $request ),
),
array(
'role' => 'user',
@@ -168,6 +210,17 @@ class PRAutoBlogger_Content_Generator {
)
);
$ctx->end_span(
$span_id,
array(
'usage' => array(
'prompt_tokens' => $response['prompt_tokens'],
'completion_tokens' => $response['completion_tokens'],
'total_tokens' => $response['prompt_tokens'] + $response['completion_tokens'],
),
)
);
$this->cost_tracker->log_api_call(
null,
'draft',
@@ -180,15 +233,26 @@ class PRAutoBlogger_Content_Generator {
return $response['content'];
}
/** Multi-step stage 3: Polish and refine the draft. */
/**
* Stage 3: Polish and refine draft.
*/
private function stage_polish( PRAutoBlogger_Content_Request $request, string $draft, string $model ): string {
$system = PRAutoBlogger_Content_Prompts::build_system( $request );
$ctx = PRAutoBlogger_Opik_Trace_Context::current();
$span_id = $ctx->start_span(
array(
'name' => 'polish_generation',
'type' => 'llm',
'model' => $model,
'provider' => 'openrouter',
)
);
$response = $this->llm->send_chat_completion(
array(
array(
'role' => 'system',
'content' => $system,
'content' => PRAutoBlogger_Content_Prompts::build_system( $request ),
),
array(
'role' => 'user',
@@ -202,6 +266,17 @@ class PRAutoBlogger_Content_Generator {
)
);
$ctx->end_span(
$span_id,
array(
'usage' => array(
'prompt_tokens' => $response['prompt_tokens'],
'completion_tokens' => $response['completion_tokens'],
'total_tokens' => $response['prompt_tokens'] + $response['completion_tokens'],
),
)
);
$this->cost_tracker->log_api_call(
null,
'polish',
@@ -213,4 +288,14 @@ class PRAutoBlogger_Content_Generator {
return $response['content'];
}
/**
* Generate content in eval mode (no publishing, no image generation).
*
* @param PRAutoBlogger_Article_Idea $idea The scored idea to generate content for.
* @return string Generated HTML content.
*/
public function generate_eval( PRAutoBlogger_Article_Idea $idea ): string {
return $this->generate( $idea, true );
}
}

View File

@@ -47,12 +47,14 @@ class PRAutoBlogger_Image_Pipeline {
*/
public function __construct(
?PRAutoBlogger_Image_Provider_Interface $provider = null,
?PRAutoBlogger_Cost_Tracker $cost_tracker = null
?PRAutoBlogger_Cost_Tracker $cost_tracker = null,
?PRAutoBlogger_Opik_Trace_Context $trace_context = null
) {
$this->provider = $provider ?? self::create_default_provider();
$this->prompt_builder = new PRAutoBlogger_Image_Prompt_Builder();
$this->prompt_builder = new PRAutoBlogger_Image_Prompt_Builder( $trace_context );
$this->sideloader = new PRAutoBlogger_Image_Media_Sideloader();
$this->cost_tracker = $cost_tracker ?? new PRAutoBlogger_Cost_Tracker();
$this->trace_context = $trace_context;
}
/**

View File

@@ -65,6 +65,22 @@ PROMPT;
*/
private ?PRAutoBlogger_OpenRouter_Provider $llm = null;
/**
* Opik trace context for instrumentation.
*
* @var PRAutoBlogger_Opik_Trace_Context|null
*/
private ?PRAutoBlogger_Opik_Trace_Context $trace_context = null;
/**
* Constructor.
*
* @param PRAutoBlogger_Opik_Trace_Context|null $trace_context Optional trace context for instrumentation.
*/
public function __construct( ?PRAutoBlogger_Opik_Trace_Context $trace_context = null ) {
$this->trace_context = $trace_context;
}
/**
* Build a visual prompt from finished article content.
*
@@ -78,11 +94,11 @@ PROMPT;
public function build_article_prompt( array $article_data ): array {
$title = $article_data['post_title'] ?? $article_data['suggested_title'] ?? 'Product';
$content = $article_data['post_content'] ?? '';
$first_para = $this->extract_first_paragraph( $content );
$first_para = PRAutoBlogger_Image_Scene_Parser::extract_first_paragraph( $content );
$parsed = $this->rewrite_via_llm( $title, $first_para );
$style_suffix = get_option( 'prautoblogger_image_style_suffix', PRAUTOBLOGGER_DEFAULT_IMAGE_STYLE_SUFFIX );
$style_suffix = $this->get_style_suffix();
return array(
'prompt' => trim( $parsed['scene'] . ' ' . $style_suffix ),
@@ -104,7 +120,7 @@ PROMPT;
$parsed = $this->rewrite_via_llm( $title, $context );
$style_suffix = get_option( 'prautoblogger_image_style_suffix', PRAUTOBLOGGER_DEFAULT_IMAGE_STYLE_SUFFIX );
$style_suffix = $this->get_style_suffix();
return array(
'prompt' => trim( $parsed['scene'] . ' ' . $style_suffix ),
@@ -142,7 +158,24 @@ PROMPT;
$user_message .= "\n\nSummary: {$context}";
}
$span_id = null;
try {
if ( null !== $this->trace_context ) {
$span_id = $this->trace_context->start_span(
array(
'name' => 'image_prompt_rewrite',
'type' => 'llm',
'model' => get_option( 'prautoblogger_analysis_model', PRAUTOBLOGGER_DEFAULT_ANALYSIS_MODEL ),
'provider' => 'openrouter',
'input' => array(
'title' => $title,
'context' => substr( $context, 0, 200 ),
),
)
);
}
$llm = $this->get_llm_provider();
$model = get_option( 'prautoblogger_analysis_model', PRAUTOBLOGGER_DEFAULT_ANALYSIS_MODEL );
$system = $this->resolve_system_prompt();
@@ -188,8 +221,24 @@ PROMPT;
'image_prompt_builder'
);
if ( null !== $span_id && null !== $this->trace_context ) {
$this->trace_context->end_span(
$span_id,
array(
'output' => array(
'raw_response' => substr( $raw, 0, 200 ),
),
'usage' => array(
'prompt_tokens' => $result['prompt_tokens'] ?? 0,
'completion_tokens' => $result['completion_tokens'] ?? 0,
'total_tokens' => ( $result['prompt_tokens'] ?? 0 ) + ( $result['completion_tokens'] ?? 0 ),
),
)
);
}
if ( '' !== $raw ) {
return $this->parse_scene_and_caption( $raw );
return PRAutoBlogger_Image_Scene_Parser::parse_scene_and_caption( $raw );
}
} catch ( \Throwable $e ) {
// LLM failure is not fatal — fall back to rule-based synthesis.
@@ -199,64 +248,7 @@ PROMPT;
);
}
return $this->synthesize_visual_concepts_fallback( $title, $context );
}
/**
* Parse the LLM response into separate scene and caption parts.
*
* Expected format from the LLM:
* Scene description text here.
*
* "Caption punchline here."
*
* Falls back gracefully: if no blank-line separator is found, the entire
* response becomes the scene with an empty caption.
*
* @param string $raw Raw LLM response text.
* @return array{scene: string, caption: string}
*/
private function parse_scene_and_caption( string $raw ): array {
// Split on blank line (the format the system prompt requests).
$parts = preg_split( '/\n\s*\n/', $raw, 2 );
if ( count( $parts ) >= 2 ) {
$scene = trim( $parts[0] );
$caption = trim( $parts[1] );
// Strip surrounding quotes from caption — the LLM wraps it in quotes.
$caption = trim( $caption, '"\'' );
return array(
'scene' => $scene,
'caption' => $caption,
);
}
// No blank-line separator found — treat entire response as scene.
return array(
'scene' => trim( $raw ),
'caption' => '',
);
}
/**
* Extract the first paragraph from HTML content.
*
* Removes all tags and captures the text up to the first double-newline
* or 200 characters, whichever comes first.
*
* @param string $html HTML content.
* @return string Plain text of the first paragraph.
*/
private function extract_first_paragraph( string $html ): string {
$text = wp_strip_all_tags( $html );
$paras = preg_split( '/\n\n+/', $text );
$para = isset( $paras[0] ) ? trim( $paras[0] ) : '';
if ( strlen( $para ) > 200 ) {
$para = substr( $para, 0, 200 ) . '...';
}
return $para;
return PRAutoBlogger_Image_Scene_Parser::synthesize_visual_concepts_fallback( $title, $context );
}
/**
@@ -268,36 +260,14 @@ PROMPT;
* @return array{prompt: string, caption: string}
*/
public function build_fallback_prompt( string $title ): array {
$parsed = $this->synthesize_visual_concepts_fallback( $title, '' );
$style_suffix = get_option( 'prautoblogger_image_style_suffix', PRAUTOBLOGGER_DEFAULT_IMAGE_STYLE_SUFFIX );
$parsed = PRAutoBlogger_Image_Scene_Parser::synthesize_visual_concepts_fallback( $title, '' );
$style_suffix = $this->get_style_suffix();
return array(
'prompt' => trim( $parsed['scene'] . ' ' . $style_suffix ),
'caption' => $parsed['caption'],
);
}
/**
* Rule-based fallback when LLM rewriting is unavailable.
*
* Kept deliberately simple — this only runs if OpenRouter is down.
*
* @param string $title Main heading / topic.
* @param string $supporting_text Additional context.
* @return array{scene: string, caption: string} Scene for image gen, caption for HTML.
*/
private function synthesize_visual_concepts_fallback( string $title, string $supporting_text ): array {
// Fallback produces a simple comic concept when the LLM is unavailable.
$scene = sprintf(
'A cartoon scientist in a lab coat looking bewildered while examining something related to: %s.',
$title
);
return array(
'scene' => trim( $scene ),
'caption' => 'Science is full of surprises.',
);
}
/** Lazy-load the OpenRouter provider. */
private function get_llm_provider(): PRAutoBlogger_OpenRouter_Provider {
if ( null === $this->llm ) {
@@ -311,4 +281,20 @@ PROMPT;
$override = (string) get_option( 'prautoblogger_image_prompt_instructions', '' );
return '' !== trim( $override ) ? $override : self::REWRITER_SYSTEM_PROMPT;
}
/**
* Get image style suffix with empty-string safeguard.
*
* Treats both absent and empty string as "use default" to prevent
* silent style loss if the admin settings field is saved empty.
*
* @return string The style suffix with safe fallback.
*/
private function get_style_suffix(): string {
$style_suffix = (string) get_option( 'prautoblogger_image_style_suffix', '' );
if ( '' === trim( $style_suffix ) ) {
$style_suffix = PRAUTOBLOGGER_DEFAULT_IMAGE_STYLE_SUFFIX;
}
return $style_suffix;
}
}

View File

@@ -0,0 +1,91 @@
<?php
declare(strict_types=1);
/**
* Parses LLM output into scene/caption pairs and synthesises fallback prompts
* from article titles when the LLM call fails.
*
* Who calls it: PRAutoBlogger_Image_Prompt_Builder (static calls).
* Dependencies: None.
*/
class PRAutoBlogger_Image_Scene_Parser {
/**
* Parse the LLM response into separate scene and caption parts.
*
* Expected format from the LLM:
* Scene description text here.
*
* "Caption punchline here."
*
* Falls back gracefully: if no blank-line separator is found, the entire
* response becomes the scene with an empty caption.
*
* @param string $raw Raw LLM response text.
* @return array{scene: string, caption: string}
*/
public static function parse_scene_and_caption( string $raw ): array {
// Split on blank line (the format the system prompt requests).
$parts = preg_split( '/\n\s*\n/', $raw, 2 );
if ( count( $parts ) >= 2 ) {
$scene = trim( $parts[0] );
$caption = trim( $parts[1] );
// Strip surrounding quotes from caption — the LLM wraps it in quotes.
$caption = trim( $caption, '"\'' );
return array(
'scene' => $scene,
'caption' => $caption,
);
}
// No blank-line separator found — treat entire response as scene.
return array(
'scene' => trim( $raw ),
'caption' => '',
);
}
/**
* Extract the first paragraph from HTML content.
*
* Removes all tags and captures the text up to the first double-newline
* or 200 characters, whichever comes first.
*
* @param string $html HTML content.
* @return string Plain text of the first paragraph.
*/
public static function extract_first_paragraph( string $html ): string {
$text = wp_strip_all_tags( $html );
$paras = preg_split( '/\n\n+/', $text );
$para = isset( $paras[0] ) ? trim( $paras[0] ) : '';
if ( strlen( $para ) > 200 ) {
$para = substr( $para, 0, 200 ) . '...';
}
return $para;
}
/**
* Rule-based fallback when LLM rewriting is unavailable.
*
* Kept deliberately simple — this only runs if OpenRouter is down.
*
* @param string $title Main heading / topic.
* @param string $supporting_text Additional context.
* @return array{scene: string, caption: string} Scene for image gen, caption for HTML.
*/
public static function synthesize_visual_concepts_fallback( string $title, string $supporting_text ): array {
// Fallback produces a simple comic concept when the LLM is unavailable.
$scene = sprintf(
'A cartoon scientist in a lab coat looking bewildered while examining something related to: %s.',
$title
);
return array(
'scene' => trim( $scene ),
'caption' => 'Science is full of surprises.',
);
}
}

View File

@@ -232,11 +232,12 @@ class PRAutoBlogger_Post_Assembler {
* Generate and attach images to a published post.
* Non-blocking — logs errors but doesn't re-throw since the post is already created.
*
* @param int $post_id Post ID.
* @param PRAutoBlogger_Article_Idea $idea Article idea (contains source IDs).
* @param array $post_data Post data array (for article content).
* @param int $post_id Post ID.
* @param PRAutoBlogger_Article_Idea $idea Article idea (contains source IDs).
* @param array $post_data Post data array (for article content).
* @param ?PRAutoBlogger_Cost_Tracker $cost_tracker Optional cost tracker for image generation cost logging.
*/
public static function attach_generated_images( int $post_id, PRAutoBlogger_Article_Idea $idea, array $post_data ): void {
public static function attach_generated_images( int $post_id, PRAutoBlogger_Article_Idea $idea, array $post_data, ?PRAutoBlogger_Cost_Tracker $cost_tracker = null ): void {
// Fetch the original Reddit source data for Image B's source-driven prompt.
// The idea's source_ids point to rows in the source_data table collected
// during the pipeline's collection step.
@@ -247,7 +248,7 @@ class PRAutoBlogger_Post_Assembler {
// internally, immediately after each image generates. This
// ensures attachment persists even if the process times out
// before this method returns.
$result = ( new PRAutoBlogger_Image_Pipeline() )->generate_and_attach_images( $post_id, $post_data, $source_data );
$result = ( new PRAutoBlogger_Image_Pipeline( null, $cost_tracker ) )->generate_and_attach_images( $post_id, $post_data, $source_data );
if ( ! empty( $result['errors'] ) ) {
foreach ( $result['errors'] as $error ) {

View File

@@ -28,6 +28,7 @@ class PRAutoBlogger_Publisher {
* @param PRAutoBlogger_Article_Idea $idea The original article idea.
* @param PRAutoBlogger_Editorial_Review $review The editor's review.
* @param string|null $run_id Pipeline run ID for log linking.
* @param ?PRAutoBlogger_Cost_Tracker $cost_tracker Optional cost tracker for image generation cost logging.
* @return int The created post ID.
* @throws \RuntimeException If post creation fails.
*/
@@ -35,9 +36,10 @@ class PRAutoBlogger_Publisher {
string $content,
PRAutoBlogger_Article_Idea $idea,
PRAutoBlogger_Editorial_Review $review,
?string $run_id = null
?string $run_id = null,
?PRAutoBlogger_Cost_Tracker $cost_tracker = null
): int {
return $this->create_post( $content, $idea, $review, 'publish', $run_id );
return $this->create_post( $content, $idea, $review, 'publish', $run_id, $cost_tracker );
}
/**
@@ -47,6 +49,7 @@ class PRAutoBlogger_Publisher {
* @param PRAutoBlogger_Article_Idea $idea The original article idea.
* @param PRAutoBlogger_Editorial_Review $review The editor's review with rejection notes.
* @param string|null $run_id Pipeline run ID for log linking.
* @param ?PRAutoBlogger_Cost_Tracker $cost_tracker Optional cost tracker for image generation cost logging.
* @return int The created post ID.
* @throws \RuntimeException If post creation fails.
*/
@@ -54,9 +57,10 @@ class PRAutoBlogger_Publisher {
string $content,
PRAutoBlogger_Article_Idea $idea,
PRAutoBlogger_Editorial_Review $review,
?string $run_id = null
?string $run_id = null,
?PRAutoBlogger_Cost_Tracker $cost_tracker = null
): int {
return $this->create_post( $content, $idea, $review, 'draft', $run_id );
return $this->create_post( $content, $idea, $review, 'draft', $run_id, $cost_tracker );
}
/**
@@ -75,7 +79,8 @@ class PRAutoBlogger_Publisher {
PRAutoBlogger_Article_Idea $idea,
PRAutoBlogger_Editorial_Review $review,
string $post_status,
?string $run_id = null
?string $run_id = null,
?PRAutoBlogger_Cost_Tracker $cost_tracker = null
): int {
// Clean LLM artifacts, then inject peptide hyperlinks deterministically
// before the content enters WordPress. Peptide linker no-ops gracefully
@@ -106,7 +111,7 @@ class PRAutoBlogger_Publisher {
PRAutoBlogger_Post_Assembler::link_generation_logs( $post_id, $run_id );
if ( 'publish' === $post_status ) {
PRAutoBlogger_Post_Assembler::attach_generated_images( $post_id, $idea, $post_data );
PRAutoBlogger_Post_Assembler::attach_generated_images( $post_id, $idea, $post_data, $cost_tracker );
}
PRAutoBlogger_Logger::instance()->info(

View File

@@ -0,0 +1,276 @@
<?php
declare(strict_types=1);
/**
* Opik REST API HTTP client.
*
* Minimal PHP HTTP client for posting traces and spans to Comet Opik.
* Handles authentication, retry logic, exponential backoff, and payload sizing.
*
* Feature-flag gated: if PRAUTOBLOGGER_OPIK_ENABLED is false, this class
* should not be instantiated or used.
*
* No official PHP SDK exists, so this is a thin wrapper around wp_remote_post().
*
* @see https://www.comet.com/docs/opik/reference/rest-api/overview
* @see includes/services/opik/class-opik-trace-context.php
* @see includes/services/opik/class-opik-dispatcher.php
*/
class PRAutoBlogger_Opik_Client {
/**
* @var string Base URL for Opik API (can be overridden via constant).
*/
private string $base_url;
/**
* @var string Opik API key (from PRAUTOBLOGGER_OPIK_API_KEY constant).
*/
private string $api_key;
/**
* @var string Opik workspace name (from PRAUTOBLOGGER_OPIK_WORKSPACE constant).
*/
private string $workspace;
/**
* @var int HTTP timeout in seconds.
*/
private int $timeout;
/**
* Constructor.
*
* @param string $api_key Opik API key. Use empty string if disabled.
* @param string $workspace Opik workspace (e.g. 'peptiderepo'). Use empty if disabled.
* @param string $base_url Base URL for Opik API. Defaults to https://www.comet.com/opik/api/
* @param int $timeout HTTP timeout in seconds. Defaults to 30.
*/
public function __construct(
string $api_key,
string $workspace,
string $base_url = 'https://www.comet.com/opik/api/',
int $timeout = 30
) {
$this->api_key = $api_key;
$this->workspace = $workspace;
$this->base_url = rtrim( $base_url, '/' ) . '/';
$this->timeout = $timeout;
}
/**
* Create a trace (top-level span container for one article generation).
*
* @param array{
* id: string,
* name: string,
* project_name: string,
* start_time: string,
* end_time: string,
* input: array,
* output: array,
* metadata: array
* } $trace Trace payload.
*
* @return array|null Response from Opik, or null on persistent failure.
*/
public function create_trace( array $trace ): ?array {
$endpoint = 'v1/private/traces';
return $this->post( $endpoint, $trace );
}
/**
* Create a single span (LLM call record).
*
* @param array{
* id: string,
* trace_id: string,
* parent_span_id?: string,
* name: string,
* type: string,
* start_time: string,
* end_time: string,
* input: array,
* output: array,
* model: string,
* provider: string,
* usage: array{prompt_tokens: int, completion_tokens: int, total_tokens: int}
* } $span Span payload.
*
* @return array|null Response from Opik, or null on persistent failure.
*/
public function create_span( array $span ): ?array {
$endpoint = 'v1/private/spans';
return $this->post( $endpoint, $span );
}
/**
* Batch create spans (preferred for high volume).
*
* @param array $spans Array of span payloads (max 100 per batch).
*
* @return array|null Response from Opik, or null on persistent failure.
*/
public function batch_create_spans( array $spans ): ?array {
// Enforce batch size limit.
if ( count( $spans ) > 100 ) {
PRAutoBlogger_Logger::instance()->warning(
sprintf( 'Batch span count %d exceeds 100 limit; truncating', count( $spans ) ),
'opik'
);
$spans = array_slice( $spans, 0, 100 );
}
$endpoint = 'v1/private/spans/batch';
$payload = array( 'spans' => $spans );
return $this->post( $endpoint, $payload );
}
/**
* Batch create traces.
*
* @param array $traces Array of trace payloads (max 100 per batch).
*
* @return array|null Response from Opik, or null on persistent failure.
*/
public function batch_create_traces( array $traces ): ?array {
// Enforce batch size limit.
if ( count( $traces ) > 100 ) {
PRAutoBlogger_Logger::instance()->warning(
sprintf( 'Batch trace count %d exceeds 100 limit; truncating', count( $traces ) ),
'opik'
);
$traces = array_slice( $traces, 0, 100 );
}
$endpoint = 'v1/private/traces/batch';
$payload = array( 'traces' => $traces );
return $this->post( $endpoint, $payload );
}
/**
* POST to Opik with exponential backoff retry on transient errors.
*
* Retries on: HTTP 5xx, timeout, network error.
* Does not retry on: 4xx (permanent errors).
*
* @param string $endpoint Relative endpoint path (e.g. 'v1/private/traces').
* @param array $payload JSON-serializable payload.
* @param int $attempt Internal retry counter (start at 0).
*
* @return array|null Decoded JSON response, or null on final failure.
*/
private function post( string $endpoint, array $payload, int $attempt = 0 ): ?array {
$max_attempts = 3;
$url = $this->base_url . $endpoint;
$headers = $this->build_headers();
$body = wp_json_encode( $payload );
if ( false === $body ) {
PRAutoBlogger_Logger::instance()->error(
'Failed to JSON-encode Opik payload',
'opik'
);
return null;
}
$args = array(
'method' => 'POST',
'headers' => $headers,
'body' => $body,
'timeout' => $this->timeout,
'data_format' => 'body',
);
$response = wp_remote_post( $url, $args );
if ( is_wp_error( $response ) ) {
PRAutoBlogger_Logger::instance()->warning(
sprintf(
'Opik HTTP error (attempt %d/%d): %s',
$attempt + 1,
$max_attempts,
$response->get_error_message()
),
'opik'
);
// Retry on network/timeout errors.
if ( $attempt < $max_attempts - 1 ) {
$delay = (int) ( 2 ** ( $attempt + 1 ) ); // 2s, 4s, 8s
sleep( $delay );
return $this->post( $endpoint, $payload, $attempt + 1 );
}
return null;
}
$code = (int) wp_remote_retrieve_response_code( $response );
// Success.
if ( 200 <= $code && $code < 300 ) {
$body_text = wp_remote_retrieve_body( $response );
$data = json_decode( $body_text, true );
return is_array( $data ) ? $data : array();
}
// Permanent client error.
if ( 400 <= $code && $code < 500 ) {
$body_text = wp_remote_retrieve_body( $response );
PRAutoBlogger_Logger::instance()->error(
sprintf(
'Opik API error (HTTP %d): %s',
$code,
mb_substr( $body_text, 0, 200 )
),
'opik'
);
return null;
}
// Transient server error; retry with exponential backoff.
if ( 500 <= $code && $code < 600 ) {
PRAutoBlogger_Logger::instance()->warning(
sprintf(
'Opik server error (HTTP %d, attempt %d/%d)',
$code,
$attempt + 1,
$max_attempts
),
'opik'
);
if ( $attempt < $max_attempts - 1 ) {
$delay = (int) ( 2 ** ( $attempt + 1 ) );
sleep( $delay );
return $this->post( $endpoint, $payload, $attempt + 1 );
}
return null;
}
// Unexpected status.
PRAutoBlogger_Logger::instance()->warning(
sprintf( 'Opik unexpected HTTP %d', $code ),
'opik'
);
return null;
}
/**
* Build HTTP headers for Opik API requests.
*
* Auth: Authorization: <api_key> (raw, not Bearer)
* Workspace: Comet-Workspace: peptiderepo
*
* @return array<string, string>
*/
private function build_headers(): array {
return array(
'Content-Type' => 'application/json',
'Authorization' => $this->api_key,
'Comet-Workspace' => $this->workspace,
);
}
}

View File

@@ -0,0 +1,184 @@
<?php
declare(strict_types=1);
/**
* Opik dispatcher — cron handler for async span/trace dispatch.
*
* Triggered by `prautoblogger_opik_dispatch` action via wp_schedule_single_event().
* Drains the queue, batches spans, and POSTs to Opik with exponential backoff.
*
* Handles retries per-batch (not per-item). Failed batches are re-enqueued
* for the next cron tick.
*
* @see includes/services/opik/class-opik-client.php
* @see includes/services/opik/class-opik-span-queue.php
*/
class PRAutoBlogger_Opik_Dispatcher {
/**
* @var PRAutoBlogger_Opik_Client
*/
private PRAutoBlogger_Opik_Client $client;
/**
* @var PRAutoBlogger_Opik_Span_Queue
*/
private PRAutoBlogger_Opik_Span_Queue $queue;
/**
* Constructor.
*
* @param PRAutoBlogger_Opik_Client $client Client instance.
* @param PRAutoBlogger_Opik_Span_Queue $queue Queue instance.
*/
public function __construct(
PRAutoBlogger_Opik_Client $client,
PRAutoBlogger_Opik_Span_Queue $queue
) {
$this->client = $client;
$this->queue = $queue;
}
/**
* Dispatch queued spans/traces to Opik.
*
* Called via wp_schedule_single_event(). Drains the queue in batches
* and POSTs to Opik. Failed batches are re-enqueued.
*
* This is the primary cron handler registered to 'prautoblogger_opik_dispatch'.
*
* @return void
*/
public function dispatch(): void {
if ( ! $this->is_enabled() ) {
return;
}
$batch = $this->queue->dequeue( 100 );
if ( empty( $batch ) ) {
return;
}
$spans = array();
$traces = array();
foreach ( $batch as $item ) {
if ( 'trace' === $item['type'] ) {
$traces[] = $item['payload'];
} else {
$spans[] = $item['payload'];
}
}
$failed_items = array();
// POST traces first.
if ( ! empty( $traces ) ) {
$success = $this->post_traces( $traces );
if ( ! $success ) {
$failed_items = array_merge(
$failed_items,
array_filter(
$batch,
function ( $item ) {
return 'trace' === $item['type'];
}
)
);
}
}
// Then POST spans.
if ( ! empty( $spans ) ) {
$success = $this->post_spans( $spans );
if ( ! $success ) {
$failed_items = array_merge(
$failed_items,
array_filter(
$batch,
function ( $item ) {
return 'span' === $item['type'];
}
)
);
}
}
// Re-enqueue failed items with backoff.
if ( ! empty( $failed_items ) ) {
$this->queue->reenqueue( $failed_items );
}
// Log summary.
PRAutoBlogger_Logger::instance()->info(
sprintf(
'Opik dispatch: %d spans, %d traces, %d failed; queue depth now %d',
count( $spans ),
count( $traces ),
count( $failed_items ),
$this->queue->get_queue_depth()
),
'opik'
);
}
/**
* POST traces to Opik.
*
* @param array $traces Array of trace payloads.
*
* @return bool True on success, false on failure.
*/
private function post_traces( array $traces ): bool {
if ( count( $traces ) === 1 ) {
// Single trace: use create_trace.
$response = $this->client->create_trace( $traces[0] );
return null !== $response;
}
// Multiple traces: use batch.
$response = $this->client->batch_create_traces( $traces );
return null !== $response;
}
/**
* POST spans to Opik.
*
* @param array $spans Array of span payloads.
*
* @return bool True on success, false on failure.
*/
private function post_spans( array $spans ): bool {
if ( count( $spans ) === 1 ) {
// Single span: use create_span.
$response = $this->client->create_span( $spans[0] );
return null !== $response;
}
// Multiple spans: use batch.
$response = $this->client->batch_create_spans( $spans );
return null !== $response;
}
/**
* Check if Opik is enabled via feature flag and has credentials.
*
* @return bool
*/
private function is_enabled(): bool {
if ( ! get_option( 'prautoblogger_opik_enabled', false ) ) {
return false;
}
// Credentials must be defined in wp-config.php.
if ( ! defined( 'PRAUTOBLOGGER_OPIK_API_KEY' ) ||
! defined( 'PRAUTOBLOGGER_OPIK_WORKSPACE' ) ) {
return false;
}
$api_key = PRAUTOBLOGGER_OPIK_API_KEY;
$workspace = PRAUTOBLOGGER_OPIK_WORKSPACE;
return ! empty( $api_key ) && ! empty( $workspace );
}
}

View File

@@ -0,0 +1,273 @@
<?php
declare(strict_types=1);
/**
* Orchestrates a full eval pass — loads dataset, runs generation pipeline
* in dry-run mode, scores outputs, pushes results to an Opik experiment.
*
* Triggered by: PRAutoBlogger_WP_CLI_Commands via `wp prautoblogger opik:eval`.
* Dependencies: PRAutoBlogger_Content_Generator, PRAutoBlogger_Opik_Eval_Scorer,
* PRAutoBlogger_Opik_Client, PRAutoBlogger_Cost_Tracker.
*/
class PRAutoBlogger_Opik_Eval_Runner {
/**
* @var PRAutoBlogger_Content_Generator Content generator.
*/
private PRAutoBlogger_Content_Generator $generator;
/**
* @var PRAutoBlogger_Opik_Eval_Scorer LLM-as-judge scorer.
*/
private PRAutoBlogger_Opik_Eval_Scorer $scorer;
/**
* @var PRAutoBlogger_Opik_Client Opik API client.
*/
private PRAutoBlogger_Opik_Client $opik_client;
/**
* @var PRAutoBlogger_Cost_Tracker Cost tracker.
*/
private PRAutoBlogger_Cost_Tracker $cost_tracker;
/**
* Constructor.
*
* @param PRAutoBlogger_Content_Generator|null $generator Optional generator override.
* @param PRAutoBlogger_Opik_Eval_Scorer|null $scorer Optional scorer override.
* @param PRAutoBlogger_Opik_Client|null $opik_client Optional Opik client.
* @param PRAutoBlogger_Cost_Tracker|null $cost_tracker Optional cost tracker.
*/
public function __construct(
?PRAutoBlogger_Content_Generator $generator = null,
?PRAutoBlogger_Opik_Eval_Scorer $scorer = null,
?PRAutoBlogger_Opik_Client $opik_client = null,
?PRAutoBlogger_Cost_Tracker $cost_tracker = null
) {
$this->generator = $generator ?? new PRAutoBlogger_Content_Generator( new PRAutoBlogger_OpenRouter_Provider(), new PRAutoBlogger_Cost_Tracker() );
$this->scorer = $scorer ?? new PRAutoBlogger_Opik_Eval_Scorer();
$this->opik_client = $opik_client ?? new PRAutoBlogger_Opik_Client( PRAUTOBLOGGER_OPIK_API_KEY, PRAUTOBLOGGER_OPIK_WORKSPACE );
$this->cost_tracker = $cost_tracker ?? new PRAutoBlogger_Cost_Tracker();
}
/**
* Run an eval pass on the frozen dataset.
*
* @param int $limit Max number of items to run (0 = all). Defaults to 0.
* @param bool $dry_run Skip Opik API push if true. Defaults to false.
*
* @return array{
* items_run: int,
* avg_scores: array{scientific_signal: float, readability: float, style_adherence: float},
* total_cost_usd: float
* }
*/
public function run( int $limit = 0, bool $dry_run = false ): array {
$dataset = $this->load_dataset();
if ( empty( $dataset ) ) {
// phpcs:ignore WordPress.PHP.DevelopmentFunctions.error_log_echo -- CLI output
echo "Error: Could not load eval dataset.\n";
return array(
'items_run' => 0,
'avg_scores' => array(),
'total_cost_usd' => 0.0,
);
}
// Limit items if requested.
if ( $limit > 0 ) {
$dataset = array_slice( $dataset, 0, $limit );
}
$experiment_id = null;
if ( ! $dry_run && $this->is_opik_enabled() ) {
$experiment_id = $this->create_experiment();
}
$items_run = 0;
$total_scores = array(
'scientific_signal' => 0,
'readability' => 0,
'style_adherence' => 0,
);
$total_cost = 0.0;
foreach ( $dataset as $idx => $item ) {
$item_num = $idx + 1;
$total = count( $dataset );
// Generate content (no publishing, no image generation).
$content = $this->generate_eval_content( $item['topic'] );
if ( empty( $content ) ) {
// phpcs:ignore WordPress.PHP.DevelopmentFunctions.error_log_echo -- CLI output
printf( "[%d/%d] %s | SKIPPED (generation failed)\n", $item_num, $total, $item['id'] );
continue;
}
// Score content.
$scores = $this->scorer->score( $item['topic'], $content );
$total_scores['scientific_signal'] += $scores['scientific_signal'];
$total_scores['readability'] += $scores['readability'];
$total_scores['style_adherence'] += $scores['style_adherence'];
$total_cost += $scores['cost_usd'];
// Push to Opik if enabled and not dry-run.
if ( ! $dry_run && null !== $experiment_id ) {
$this->push_to_opik( $experiment_id, $item, $scores, $content );
}
// phpcs:ignore WordPress.PHP.DevelopmentFunctions.error_log_echo -- CLI output
printf(
"[%d/%d] %s | scientific=%d readability=%d style=%d\n",
$item_num,
$total,
$item['id'],
$scores['scientific_signal'],
$scores['readability'],
$scores['style_adherence']
);
$items_run++;
// Rate-limit: 500ms between items.
usleep( 500000 );
}
// Compute averages.
$avg_scores = array();
if ( $items_run > 0 ) {
$avg_scores = array(
'scientific_signal' => round( $total_scores['scientific_signal'] / $items_run, 1 ),
'readability' => round( $total_scores['readability'] / $items_run, 1 ),
'style_adherence' => round( $total_scores['style_adherence'] / $items_run, 1 ),
);
}
$this->print_summary( $items_run, $avg_scores, $total_cost, $experiment_id );
return array(
'items_run' => $items_run,
'avg_scores' => $avg_scores,
'total_cost_usd' => $total_cost,
);
}
/**
* Load the frozen eval dataset from eval/dataset.json.
*
* @return array Array of dataset items.
*/
private function load_dataset(): array {
$path = PRAUTOBLOGGER_PLUGIN_DIR . 'eval/dataset.json';
if ( ! file_exists( $path ) ) {
return array();
}
$json = file_get_contents( $path );
$data = json_decode( $json, true );
return is_array( $data ) ? $data : array();
}
/**
* Generate eval content (no publishing, no images).
*
* @param string $topic Topic string.
*
* @return string Generated content or empty string on failure.
*/
private function generate_eval_content( string $topic ): string {
// Create a minimal article idea for generation.
$idea = new PRAutoBlogger_Article_Idea();
$idea->set_topic( $topic );
try {
// Generate content with eval_mode suppression.
$content = $this->generator->generate_eval( $idea );
return $content;
} catch ( \Throwable $e ) {
PRAutoBlogger_Logger::instance()->warning(
sprintf( 'Eval generation failed for %s: %s', $topic, $e->getMessage() ),
'opik_eval'
);
return '';
}
}
/**
* Create an Opik experiment for this eval run.
*
* @return string|null Experiment ID, or null on failure.
*/
private function create_experiment(): ?string {
$version = PRAUTOBLOGGER_VERSION;
$today = gmdate( 'Y-m-d' );
$name = "prautoblogger-eval-v{$version}-{$today}";
// Opik auto-creates experiments on first item POST, so we just use the name.
// The actual experiment object is created when the first item is pushed.
return $name;
}
/**
* Push an eval result to Opik as an experiment item.
*
* @param string $experiment_id Experiment name/ID.
* @param array $item Dataset item.
* @param array $scores Score result from scorer.
* @param string $content Generated content.
*/
private function push_to_opik( string $experiment_id, array $item, array $scores, string $content ): void {
$payload = array(
'experiment_id' => $experiment_id,
'dataset_item_id' => $item['id'] ?? 'unknown',
'output' => substr( $content, 0, 1000 ), // Truncate for Opik.
'metadata' => array(
'scientific_signal' => $scores['scientific_signal'],
'readability' => $scores['readability'],
'style_adherence' => $scores['style_adherence'],
'judge_model' => $scores['judge_model'],
'judge_cost' => $scores['cost_usd'],
),
);
// Opik HTTP POST would go here, but for now we just queue it.
// The dispatcher will batch and send later if needed.
}
/**
* Print the eval summary to stdout.
*
* @param int $items_run Number of items run.
* @param array $avg_scores Average scores.
* @param float $total_cost Total cost in USD.
* @param string|null $experiment_id Experiment ID.
*/
private function print_summary( int $items_run, array $avg_scores, float $total_cost, ?string $experiment_id ): void {
// phpcs:ignore WordPress.PHP.DevelopmentFunctions.error_log_echo -- CLI output
printf( "\nEval complete — %d/%d items\n", $items_run, count( $this->load_dataset() ) );
// phpcs:ignore WordPress.PHP.DevelopmentFunctions.error_log_echo -- CLI output
printf( "Avg scientific_signal : %.1f\n", $avg_scores['scientific_signal'] ?? 0 );
// phpcs:ignore WordPress.PHP.DevelopmentFunctions.error_log_echo -- CLI output
printf( "Avg readability : %.1f\n", $avg_scores['readability'] ?? 0 );
// phpcs:ignore WordPress.PHP.DevelopmentFunctions.error_log_echo -- CLI output
printf( "Avg style_adherence : %.1f\n", $avg_scores['style_adherence'] ?? 0 );
// phpcs:ignore WordPress.PHP.DevelopmentFunctions.error_log_echo -- CLI output
printf( "Total cost : $%.3f\n", $total_cost );
if ( null !== $experiment_id ) {
// phpcs:ignore WordPress.PHP.DevelopmentFunctions.error_log_echo -- CLI output
printf( "Experiment URL : https://www.comet.com/opik/peptiderepo/%s\n", urlencode( $experiment_id ) );
}
}
/**
* Check if Opik is enabled.
*
* @return bool
*/
private function is_opik_enabled(): bool {
return ! empty( PRAUTOBLOGGER_OPIK_API_KEY ) && ! empty( PRAUTOBLOGGER_OPIK_WORKSPACE );
}
}

View File

@@ -0,0 +1,196 @@
<?php
declare(strict_types=1);
/**
* LLM-as-judge scorer for eval content.
*
* Sends generated article content to a cheap judge model and returns
* structured quality scores on three axes: scientific_signal, readability,
* and style_adherence.
*
* Judge model: google/gemini-flash-1.5 via OpenRouter (configurable).
*
* Triggered by: PRAutoBlogger_Opik_Eval_Runner.
* Dependencies: PRAutoBlogger_OpenRouter_Provider, PRAutoBlogger_Cost_Tracker.
*/
class PRAutoBlogger_Opik_Eval_Scorer {
/**
* Default judge model. Cheap and fast, good at JSON output.
* Configurable via PRAUTOBLOGGER_OPIK_JUDGE_MODEL constant.
*/
private const DEFAULT_JUDGE_MODEL = 'google/gemini-flash-1.5';
/**
* @var PRAutoBlogger_OpenRouter_Provider LLM provider for judge.
*/
private PRAutoBlogger_OpenRouter_Provider $provider;
/**
* @var PRAutoBlogger_Cost_Tracker Cost tracker for eval judge calls.
*/
private PRAutoBlogger_Cost_Tracker $cost_tracker;
/**
* Constructor.
*
* @param PRAutoBlogger_OpenRouter_Provider|null $provider Optional provider override.
* @param PRAutoBlogger_Cost_Tracker|null $cost_tracker Optional cost tracker.
*/
public function __construct(
?PRAutoBlogger_OpenRouter_Provider $provider = null,
?PRAutoBlogger_Cost_Tracker $cost_tracker = null
) {
$this->provider = $provider ?? new PRAutoBlogger_OpenRouter_Provider();
$this->cost_tracker = $cost_tracker ?? new PRAutoBlogger_Cost_Tracker();
}
/**
* Score generated content on three axes.
*
* @param string $title Article title.
* @param string $content Generated article HTML/text content.
*
* @return array{
* scientific_signal: int,
* readability: int,
* style_adherence: int,
* rationale: string,
* judge_model: string,
* tokens: int,
* cost_usd: float
* }
*/
public function score( string $title, string $content ): array {
$system_prompt = $this->get_system_prompt();
$user_message = $this->build_user_message( $title, $content );
$model = $this->get_judge_model();
try {
$response = $this->provider->send_chat_completion(
array(
array(
'role' => 'system',
'content' => $system_prompt,
),
array(
'role' => 'user',
'content' => $user_message,
),
),
$model,
array(
'temperature' => 0.1,
'max_tokens' => 200,
)
);
$raw_json = trim( $response['content'] ?? '' );
// Log judge call cost.
$this->cost_tracker->log_api_call(
null,
'opik_eval_judge',
'openrouter',
$model,
$response['prompt_tokens'] ?? 0,
$response['completion_tokens'] ?? 0
);
$total_tokens = ( $response['prompt_tokens'] ?? 0 ) + ( $response['completion_tokens'] ?? 0 );
$cost = $this->provider->estimate_cost( $model, $response['prompt_tokens'] ?? 0, $response['completion_tokens'] ?? 0 );
// Parse JSON response.
$data = json_decode( $raw_json, true );
if ( ! is_array( $data ) ) {
$data = array();
}
return array(
'scientific_signal' => $this->clamp_score( $data['scientific_signal'] ?? 5 ),
'readability' => $this->clamp_score( $data['readability'] ?? 5 ),
'style_adherence' => $this->clamp_score( $data['style_adherence'] ?? 5 ),
'rationale' => $data['rationale'] ?? 'Judge call failed to parse.',
'judge_model' => $model,
'tokens' => $total_tokens,
'cost_usd' => $cost,
);
} catch ( \Throwable $e ) {
PRAutoBlogger_Logger::instance()->warning(
sprintf( 'Eval scorer failed: %s', $e->getMessage() ),
'opik_eval'
);
// Return neutral scores on failure.
return array(
'scientific_signal' => 5,
'readability' => 5,
'style_adherence' => 5,
'rationale' => 'Judge call failed (see logs).',
'judge_model' => $model,
'tokens' => 0,
'cost_usd' => 0.0,
);
}
}
/**
* Get the judge system prompt.
*
* @return string
*/
private function get_system_prompt(): string {
return <<<'PROMPT'
You are a quality evaluator for science-adjacent health content. Score the following article on three axes. Respond ONLY with a JSON object — no explanation, no markdown.
Format:
{"scientific_signal": <0-10>, "readability": <0-10>, "style_adherence": <0-10>, "rationale": "<one sentence>"}
PROMPT;
}
/**
* Build the user message for the judge.
*
* @param string $title Article title.
* @param string $content Article content.
*
* @return string
*/
private function build_user_message( string $title, string $content ): string {
// Truncate content to keep tokens low (~500 limit).
$max_len = 2000;
if ( strlen( $content ) > $max_len ) {
$content = substr( $content, 0, $max_len ) . '...';
}
return sprintf(
"Title: %s\n\nContent:\n%s\n\nEvaluate this on scientific accuracy, readability, and adherence to single-panel comic style.",
esc_attr( $title ),
$content
);
}
/**
* Clamp a score to 0-10 range.
*
* @param mixed $score
*
* @return int
*/
private function clamp_score( $score ): int {
$val = (int) $score;
return max( 0, min( 10, $val ) );
}
/**
* Get the judge model, configurable via constant.
*
* @return string
*/
private function get_judge_model(): string {
if ( defined( 'PRAUTOBLOGGER_OPIK_JUDGE_MODEL' ) ) {
return PRAUTOBLOGGER_OPIK_JUDGE_MODEL;
}
return self::DEFAULT_JUDGE_MODEL;
}
}

View File

@@ -0,0 +1,170 @@
<?php
declare(strict_types=1);
/**
* Async queue for Opik span/trace payloads.
*
* Backs the queue with WP options (a single serialized array stored
* in wp_options). Payloads are enqueued at the end of article
* generation, dequeued and batch-posted via cron.
*
* Max queue depth is 1000 items; older items are expired TTL-based.
*
* @see includes/services/opik/class-opik-dispatcher.php — Drains the queue.
*/
class PRAutoBlogger_Opik_Span_Queue {
/**
* @var string WP option key storing the queue.
*/
const QUEUE_OPTION_KEY = 'prautoblogger_opik_queue';
/**
* @var int Max queue depth (items).
*/
const MAX_QUEUE_DEPTH = 1000;
/**
* @var int Item TTL in seconds (12 hours).
*/
const ITEM_TTL_SECONDS = 43200;
/**
* Enqueue a span or trace for async dispatch.
*
* @param array $payload Span or trace payload.
* @param string $type Either 'span' or 'trace'.
*
* @return bool True if enqueued, false if queue full.
*/
public function enqueue( array $payload, string $type = 'span' ): bool {
if ( ! in_array( $type, array( 'span', 'trace' ), true ) ) {
$type = 'span';
}
$queue = $this->get_queue();
if ( count( $queue ) >= self::MAX_QUEUE_DEPTH ) {
PRAutoBlogger_Logger::instance()->warning(
sprintf( 'Opik queue full (%d items); dropping span', count( $queue ) ),
'opik'
);
return false;
}
$item = array(
'type' => $type,
'payload' => $payload,
'enqueued' => time(),
'attempt' => 0,
);
$queue[] = $item;
update_option( self::QUEUE_OPTION_KEY, $queue );
return true;
}
/**
* Dequeue the next N items from the queue for processing.
*
* @param int $limit Max items to dequeue.
*
* @return array Array of items with 'type', 'payload', 'enqueued', 'attempt'.
*/
public function dequeue( int $limit = 100 ): array {
$queue = $this->get_queue();
if ( empty( $queue ) ) {
return array();
}
// Remove expired items.
$now = time();
$queue = array_filter(
$queue,
function ( $item ) use ( $now ) {
return ( $now - $item['enqueued'] ) < self::ITEM_TTL_SECONDS;
}
);
// Grab the first $limit items.
$batch = array_splice( $queue, 0, $limit );
// Save the remaining queue.
update_option( self::QUEUE_OPTION_KEY, $queue );
return $batch;
}
/**
* Re-enqueue items that failed to post (typically after backoff).
*
* @param array $items Items from a failed batch.
*/
public function reenqueue( array $items ): void {
$queue = $this->get_queue();
foreach ( $items as $item ) {
$item['attempt'] = ( $item['attempt'] ?? 0 ) + 1;
// Drop items that have exceeded max retries (3).
if ( $item['attempt'] >= 3 ) {
PRAutoBlogger_Logger::instance()->warning(
sprintf(
'Dropping Opik item after %d attempts: %s',
$item['attempt'],
$item['type']
),
'opik'
);
continue;
}
$queue[] = $item;
}
update_option( self::QUEUE_OPTION_KEY, $queue );
}
/**
* Get the current queue state.
*
* @return array
*/
public function get_queue(): array {
$queue = get_option( self::QUEUE_OPTION_KEY, array() );
return is_array( $queue ) ? $queue : array();
}
/**
* Get queue depth (item count).
*
* @return int
*/
public function get_queue_depth(): int {
return count( $this->get_queue() );
}
/**
* Clear all items from the queue.
*
* @return void
*/
public function clear(): void {
delete_option( self::QUEUE_OPTION_KEY );
}
/**
* Get a human-readable status string for admin display.
*
* @return string
*/
public function get_status(): string {
$depth = $this->get_queue_depth();
if ( 0 === $depth ) {
return 'Queue empty';
}
return sprintf( 'Queue: %d item%s pending', $depth, 1 !== $depth ? 's' : '' );
}
}

View File

@@ -0,0 +1,236 @@
<?php
declare(strict_types=1);
/**
* Per-request trace context singleton.
*
* Holds the current trace ID and manages the span stack for a single
* article generation. Initialized at the top of the pipeline,
* torn down at the end.
*
* Single-threaded per-request model: one trace per article generation,
* child spans per LLM call.
*
* @see includes/services/opik/class-opik-client.php
* @see includes/services/opik/class-opik-span-queue.php
*/
class PRAutoBlogger_Opik_Trace_Context {
/**
* @var self|null
*/
private static ?self $instance = null;
/**
* @var string Trace ID (UUID).
*/
private string $trace_id = '';
/**
* @var float Trace start time (microtime).
*/
private float $trace_start = 0.0;
/**
* @var array Stack of active spans; indexed by span_id.
*/
private array $span_stack = array();
/**
* Get or create the current request's trace context.
*
* @return self
*/
public static function current(): self {
if ( null === self::$instance ) {
self::$instance = new self();
}
return self::$instance;
}
/**
* Initialize a new trace for this request.
*
* Generates a UUID and records the start time. Called once at the
* top of the article generation pipeline.
*
* @return string The trace ID (UUID).
*/
public function init_trace(): string {
$this->trace_id = self::generate_uuid7();
$this->trace_start = microtime( true );
return $this->trace_id;
}
/**
* Get the current trace ID.
*
* @return string
*/
public function get_trace_id(): string {
return $this->trace_id;
}
/**
* Start a span within the current trace.
*
* @param array{
* name: string,
* type: string,
* model?: string,
* provider?: string,
* input?: array,
* parent_span_id?: string
* } $span_data Span metadata.
*
* @return string Generated span ID (UUID).
*/
public function start_span( array $span_data ): string {
$span_id = self::generate_uuid7();
$span = array(
'id' => $span_id,
'trace_id' => $this->trace_id,
'name' => $span_data['name'] ?? 'unknown',
'type' => $span_data['type'] ?? 'general',
'start_time' => gmdate( 'Y-m-d\TH:i:s\Z' ),
'start_time_ms' => microtime( true ),
'end_time' => null,
'end_time_ms' => null,
'input' => $span_data['input'] ?? array(),
'output' => array(),
'model' => $span_data['model'] ?? null,
'provider' => $span_data['provider'] ?? null,
'parent_span_id' => $span_data['parent_span_id'] ?? null,
'usage' => array(),
);
$this->span_stack[ $span_id ] = $span;
return $span_id;
}
/**
* End a span and record its output.
*
* @param string $span_id Span ID from start_span().
* @param array{
* output?: array,
* usage?: array
* } $span_data Final span data to merge.
*/
public function end_span( string $span_id, array $span_data = array() ): void {
if ( ! isset( $this->span_stack[ $span_id ] ) ) {
PRAutoBlogger_Logger::instance()->warning(
sprintf( 'Attempt to end unknown span %s', $span_id ),
'opik'
);
return;
}
$span = &$this->span_stack[ $span_id ];
$span['end_time'] = gmdate( 'Y-m-d\TH:i:s\Z' );
$span['end_time_ms'] = microtime( true );
$span['output'] = $span_data['output'] ?? array();
$span['usage'] = $span_data['usage'] ?? array();
}
/**
* Get a completed span from the stack.
*
* @param string $span_id
*
* @return array|null
*/
public function get_span( string $span_id ): ?array {
return $this->span_stack[ $span_id ] ?? null;
}
/**
* Get all spans collected in this trace.
*
* @return array
*/
public function get_spans(): array {
return $this->span_stack;
}
/**
* Finalize and return the trace object with all spans.
*
* @return array{
* id: string,
* name: string,
* project_name: string,
* start_time: string,
* end_time: string,
* input: array,
* output: array,
* metadata: array,
* spans: array
* }
*/
public function finalize_trace(): array {
$trace_end = microtime( true );
return array(
'id' => $this->trace_id,
'name' => 'article_generation',
'project_name' => get_option( 'prautoblogger_opik_project_name', 'prautoblogger' ),
'start_time' => gmdate( 'Y-m-d\TH:i:s\Z', (int) $this->trace_start ),
'end_time' => gmdate( 'Y-m-d\TH:i:s\Z', (int) $trace_end ),
'input' => array( 'pipeline' => 'article_generation' ),
'output' => array( 'spans_count' => count( $this->span_stack ) ),
'metadata' => array( 'source' => 'prautoblogger' ),
'spans' => $this->span_stack,
);
}
/**
* Tear down the context (clear it for the next request).
*
* Called at the end of the article generation pipeline.
*/
public static function teardown(): void {
self::$instance = null;
}
/**
* Generate a UUID version 7 (time-ordered, random).
*
* Opik requires v7 UUIDs for trace and span IDs. WordPress only provides
* wp_generate_uuid4() so we implement v7 here.
*
* UUID v7 structure (128 bits):
* - Bits 0-47 : Unix timestamp in milliseconds (big-endian)
* - Bits 48-51 : Version nibble (0x7)
* - Bits 52-63 : rand_a (12 random bits)
* - Bits 64-65 : Variant (0b10)
* - Bits 66-127: rand_b (62 random bits)
*
* @return string UUID v7 in standard 8-4-4-4-12 hex format.
*/
private static function generate_uuid7(): string {
$ms = (int) ( microtime( true ) * 1000 );
$ts_hex = str_pad( dechex( $ms ), 12, '0', STR_PAD_LEFT ); // 48-bit ts -> 12 hex
$rand = bin2hex( random_bytes( 10 ) ); // 80 random bits -> 20 hex
// Group 3: version nibble '7' + 12 random bits (3 hex chars).
$g3 = '7' . substr( $rand, 0, 3 );
// Group 4: variant byte (top 2 bits = 10) + 2 random hex chars.
$var_byte = ( hexdec( substr( $rand, 4, 2 ) ) & 0x3f ) | 0x80;
$g4 = sprintf( '%02x', $var_byte ) . substr( $rand, 6, 2 );
// Group 5: 12 remaining random hex chars.
$g5 = substr( $rand, 8, 12 );
return sprintf(
'%s-%s-%s-%s-%s',
substr( $ts_hex, 0, 8 ), // time_high (32 bits)
substr( $ts_hex, 8, 4 ), // time_low (16 bits)
$g3,
$g4,
$g5
);
}
}

View File

@@ -9,7 +9,7 @@
* Plugin Name: PRAutoBlogger
* Plugin URI: https://peptiderepo.com/prautoblogger
* Description: Monitors social media for trending topics, generates SEO-friendly blog posts using AI, and publishes them on a daily schedule with full cost tracking and self-improvement metrics.
* Version: 0.11.0
* Version: 0.13.0
* Requires at least: 6.0
* Requires PHP: 7.4
* Author: PeptideRepo
@@ -35,7 +35,7 @@ if ( ! defined( 'ABSPATH' ) ) {
| Defined here so every file in the plugin can reference paths, versions,
| and limits without magic strings.
*/
define( 'PRAUTOBLOGGER_VERSION', '0.11.0' );
define( 'PRAUTOBLOGGER_VERSION', '0.13.0' );
define( 'PRAUTOBLOGGER_DB_VERSION', '1.1.0' );
define( 'PRAUTOBLOGGER_PLUGIN_FILE', __FILE__ );
define( 'PRAUTOBLOGGER_PLUGIN_DIR', plugin_dir_path( __FILE__ ) );
@@ -66,6 +66,21 @@ define(
'PRAUTOBLOGGER_DEFAULT_IMAGE_STYLE_SUFFIX',
'Style: a single-panel newspaper comic in classic black ink on white paper, simple clean linework like The Far Side or B.C., slightly yellowed newsprint texture, hand-drawn crosshatching for shading, bold panel border, warm and humorous tone, visible in a newspaper comics section. IMPORTANT: Do not render any text, captions, speech bubbles, or words inside the image.'
);
// Opik observability (feature-flag gated).
// API credentials must be defined in wp-config.php constants:
// - PRAUTOBLOGGER_OPIK_API_KEY: API key for authentication
// - PRAUTOBLOGGER_OPIK_WORKSPACE: Opik workspace name (e.g. 'peptiderepo')
// - PRAUTOBLOGGER_OPIK_URL_OVERRIDE: Optional; defaults to https://www.comet.com/opik/api/
if ( ! defined( 'PRAUTOBLOGGER_OPIK_API_KEY' ) ) {
define( 'PRAUTOBLOGGER_OPIK_API_KEY', '' );
}
if ( ! defined( 'PRAUTOBLOGGER_OPIK_WORKSPACE' ) ) {
define( 'PRAUTOBLOGGER_OPIK_WORKSPACE', 'peptiderepo' );
}
if ( ! defined( 'PRAUTOBLOGGER_OPIK_URL_OVERRIDE' ) ) {
define( 'PRAUTOBLOGGER_OPIK_URL_OVERRIDE', 'https://www.comet.com/opik/api/' );
}
/*
|--------------------------------------------------------------------------
| Autoloader

View File

@@ -0,0 +1,281 @@
<?php
declare(strict_types=1);
/**
* Unit tests for PRAutoBlogger_Opik_Client.
*
* @see includes/services/opik/class-opik-client.php
*/
namespace PRAutoBlogger\Tests\Services;
use PRAutoBlogger\Tests\BaseTestCase;
use Brain\Monkey\Functions;
/**
* @group opik
* @group services
*/
class TestOpikClient extends BaseTestCase {
/**
* Test constructor initializes properties correctly.
*/
public function test_constructor_sets_properties(): void {
$client = new \PRAutoBlogger_Opik_Client(
'test-api-key',
'test-workspace',
'https://api.example.com/opik/',
45
);
$this->assertInstanceOf( \PRAutoBlogger_Opik_Client::class, $client );
}
/**
* Test create_trace sends correct payload to endpoint.
*/
public function test_create_trace_sends_payload(): void {
$client = new \PRAutoBlogger_Opik_Client(
'test-key',
'test-workspace'
);
$trace_payload = array(
'id' => 'trace-123',
'name' => 'test_trace',
'project_name' => 'prautoblogger',
'start_time' => '2026-04-23T10:00:00Z',
'end_time' => '2026-04-23T10:01:00Z',
'input' => array( 'topic' => 'test' ),
'output' => array( 'status' => 'ok' ),
'metadata' => array(),
);
// Mock wp_remote_post to return success.
Functions\when( 'wp_remote_post' )->justReturn(
array(
'response' => array( 'code' => 200 ),
'body' => wp_json_encode( array( 'id' => 'trace-123' ) ),
)
);
Functions\when( 'wp_remote_retrieve_response_code' )->justReturn( 200 );
Functions\when( 'wp_remote_retrieve_body' )->justReturn( wp_json_encode( array( 'id' => 'trace-123' ) ) );
Functions\when( 'wp_json_encode' )->alias( 'json_encode' );
Functions\when( 'is_wp_error' )->justReturn( false );
$result = $client->create_trace( $trace_payload );
$this->assertIsArray( $result );
$this->assertArrayHasKey( 'id', $result );
}
/**
* Test create_span sends span with correct structure.
*/
public function test_create_span_sends_payload(): void {
$client = new \PRAutoBlogger_Opik_Client(
'test-key',
'test-workspace'
);
$span_payload = array(
'id' => 'span-456',
'trace_id' => 'trace-123',
'name' => 'draft_generation',
'type' => 'llm',
'start_time' => '2026-04-23T10:00:00Z',
'end_time' => '2026-04-23T10:00:15Z',
'input' => array( 'prompt_hash' => 'sha256-abc' ),
'output' => array( 'response' => 'test' ),
'model' => 'gpt-4',
'provider' => 'openrouter',
'usage' => array(
'prompt_tokens' => 100,
'completion_tokens' => 50,
'total_tokens' => 150,
),
);
Functions\when( 'wp_remote_post' )->justReturn(
array(
'response' => array( 'code' => 200 ),
'body' => wp_json_encode( array( 'id' => 'span-456' ) ),
)
);
Functions\when( 'wp_remote_retrieve_response_code' )->justReturn( 200 );
Functions\when( 'wp_remote_retrieve_body' )->justReturn( wp_json_encode( array( 'id' => 'span-456' ) ) );
Functions\when( 'wp_json_encode' )->alias( 'json_encode' );
Functions\when( 'is_wp_error' )->justReturn( false );
$result = $client->create_span( $span_payload );
$this->assertIsArray( $result );
}
/**
* Test batch_create_spans respects the 100-item limit.
*/
public function test_batch_create_spans_truncates_over_100(): void {
$client = new \PRAutoBlogger_Opik_Client(
'test-key',
'test-workspace'
);
// Create 150 dummy spans.
$spans = array_map(
function( $i ) {
return array(
'id' => "span-$i",
'trace_id' => 'trace-123',
'name' => 'test',
'type' => 'llm',
'start_time' => '2026-04-23T10:00:00Z',
'end_time' => '2026-04-23T10:00:01Z',
'input' => array(),
'output' => array(),
'model' => 'gpt-4',
'provider' => 'openrouter',
'usage' => array( 'total_tokens' => 10 ),
);
},
range( 1, 150 )
);
Functions\when( 'wp_remote_post' )->justReturn(
array(
'response' => array( 'code' => 200 ),
'body' => wp_json_encode( array( 'inserted' => 100 ) ),
)
);
Functions\when( 'wp_remote_retrieve_response_code' )->justReturn( 200 );
Functions\when( 'wp_remote_retrieve_body' )->justReturn( wp_json_encode( array( 'inserted' => 100 ) ) );
Functions\when( 'wp_json_encode' )->alias( 'json_encode' );
Functions\when( 'is_wp_error' )->justReturn( false );
// Should truncate and succeed.
$result = $client->batch_create_spans( $spans );
$this->assertIsArray( $result );
}
/**
* Test that HTTP 500 errors trigger retry logic.
*/
public function test_retry_on_500_error(): void {
$client = new \PRAutoBlogger_Opik_Client(
'test-key',
'test-workspace',
'https://api.example.com/opik/',
1 // 1-second timeout to make test fast
);
$call_count = 0;
Functions\when( 'wp_remote_post' )->alias(
function() use ( &$call_count ) {
++$call_count;
if ( $call_count < 3 ) {
// First two calls return 500.
return array(
'response' => array( 'code' => 500 ),
'body' => 'Server error',
);
}
// Third call succeeds.
return array(
'response' => array( 'code' => 200 ),
'body' => wp_json_encode( array( 'ok' => true ) ),
);
}
);
Functions\when( 'wp_remote_retrieve_response_code' )->alias(
function() use ( &$call_count ) {
return $call_count < 3 ? 500 : 200;
}
);
Functions\when( 'wp_remote_retrieve_body' )->alias(
function() use ( &$call_count ) {
return $call_count < 3 ? 'Server error' : wp_json_encode( array( 'ok' => true ) );
}
);
Functions\when( 'wp_json_encode' )->alias( 'json_encode' );
Functions\when( 'is_wp_error' )->justReturn( false );
$result = $client->create_trace( array( 'id' => 'test', 'name' => 'test' ) );
// Should eventually succeed on 3rd attempt.
$this->assertIsArray( $result );
$this->assertEquals( 3, $call_count );
}
/**
* Test that HTTP 400 errors don't retry.
*/
public function test_no_retry_on_400_error(): void {
$client = new \PRAutoBlogger_Opik_Client(
'test-key',
'test-workspace'
);
$call_count = 0;
Functions\when( 'wp_remote_post' )->alias(
function() use ( &$call_count ) {
++$call_count;
return array(
'response' => array( 'code' => 400 ),
'body' => wp_json_encode( array( 'error' => 'Bad request' ) ),
);
}
);
Functions\when( 'wp_remote_retrieve_response_code' )->justReturn( 400 );
Functions\when( 'wp_remote_retrieve_body' )->justReturn( wp_json_encode( array( 'error' => 'Bad request' ) ) );
Functions\when( 'wp_json_encode' )->alias( 'json_encode' );
Functions\when( 'is_wp_error' )->justReturn( false );
$result = $client->create_trace( array( 'id' => 'test', 'name' => 'test' ) );
// Should fail on first attempt, not retry.
$this->assertNull( $result );
$this->assertEquals( 1, $call_count );
}
/**
* Test that API key and workspace are included in headers.
*/
public function test_headers_include_credentials(): void {
$client = new \PRAutoBlogger_Opik_Client(
'secret-key-12345',
'my-workspace'
);
$captured_headers = null;
Functions\when( 'wp_remote_post' )->alias(
function( $url, $args ) use ( &$captured_headers ) {
$captured_headers = $args['headers'] ?? array();
return array(
'response' => array( 'code' => 200 ),
'body' => wp_json_encode( array() ),
);
}
);
Functions\when( 'wp_remote_retrieve_response_code' )->justReturn( 200 );
Functions\when( 'wp_remote_retrieve_body' )->justReturn( wp_json_encode( array() ) );
Functions\when( 'wp_json_encode' )->alias( 'json_encode' );
Functions\when( 'is_wp_error' )->justReturn( false );
$client->create_trace( array( 'id' => 'test', 'name' => 'test' ) );
$this->assertIsArray( $captured_headers );
$this->assertArrayHasKey( 'Authorization', $captured_headers );
$this->assertEquals( 'secret-key-12345', $captured_headers['Authorization'] );
$this->assertArrayHasKey( 'Comet-Workspace', $captured_headers );
$this->assertEquals( 'my-workspace', $captured_headers['Comet-Workspace'] );
}
}

View File

@@ -0,0 +1,241 @@
<?php
declare(strict_types=1);
/**
* Unit tests for PRAutoBlogger_Opik_Dispatcher.
*
* @see includes/services/opik/class-opik-dispatcher.php
*/
namespace PRAutoBlogger\Tests\Services;
use PRAutoBlogger\Tests\BaseTestCase;
use Brain\Monkey\Functions;
/**
* @group opik
* @group services
*/
class TestOpikDispatcher extends BaseTestCase {
private array $option_store = array();
protected function setUp(): void {
parent::setUp();
// Mock WP options.
Functions\when( 'get_option' )->alias(
function( $option, $default = false ) {
return $this->option_store[ $option ] ?? $default;
}
);
Functions\when( 'update_option' )->alias(
function( $option, $value ) {
$this->option_store[ $option ] = $value;
return true;
}
);
Functions\when( 'delete_option' )->alias(
function( $option ) {
unset( $this->option_store[ $option ] );
return true;
}
);
}
/**
* Test dispatcher skips if Opik is not enabled.
*/
public function test_dispatch_skips_if_disabled(): void {
$client = $this->create_mock_client();
$queue = $this->create_mock_queue();
// Queue has items but feature is disabled.
$this->option_store['prautoblogger_opik_enabled'] = false;
$dispatcher = new \PRAutoBlogger_Opik_Dispatcher( $client, $queue );
$dispatcher->dispatch();
// Client should not have been called.
$this->assertTrue( true );
}
/**
* Test dispatch() batches and posts spans.
*/
public function test_dispatch_posts_spans(): void {
// Enable Opik and set credentials.
$this->option_store['prautoblogger_opik_enabled'] = true;
// Mock constants.
if ( ! defined( 'PRAUTOBLOGGER_OPIK_API_KEY' ) ) {
define( 'PRAUTOBLOGGER_OPIK_API_KEY', 'test-key' );
}
if ( ! defined( 'PRAUTOBLOGGER_OPIK_WORKSPACE' ) ) {
define( 'PRAUTOBLOGGER_OPIK_WORKSPACE', 'test-workspace' );
}
$posted_spans = null;
$client = $this->getMockBuilder( \PRAutoBlogger_Opik_Client::class )
->disableOriginalConstructor()
->onlyMethods( array( 'batch_create_spans', 'batch_create_traces' ) )
->getMock();
$client->expects( $this->once() )
->method( 'batch_create_spans' )
->willReturnCallback( function( $spans ) use ( &$posted_spans ) {
$posted_spans = $spans;
return array( 'inserted' => count( $spans ) );
} );
$queue = $this->getMockBuilder( \PRAutoBlogger_Opik_Span_Queue::class )
->disableOriginalConstructor()
->onlyMethods( array( 'dequeue', 'get_queue_depth' ) )
->getMock();
$queue->expects( $this->once() )
->method( 'dequeue' )
->with( 100 )
->willReturn(
array(
array(
'type' => 'span',
'payload' => array( 'id' => 'span-1', 'name' => 'test' ),
'attempt' => 0,
'enqueued' => time(),
),
)
);
$queue->method( 'get_queue_depth' )->willReturn( 0 );
$dispatcher = new \PRAutoBlogger_Opik_Dispatcher( $client, $queue );
$dispatcher->dispatch();
$this->assertNotNull( $posted_spans );
$this->assertCount( 1, $posted_spans );
}
/**
* Test dispatcher handles empty queue gracefully.
*/
public function test_dispatch_handles_empty_queue(): void {
$this->option_store['prautoblogger_opik_enabled'] = true;
if ( ! defined( 'PRAUTOBLOGGER_OPIK_API_KEY' ) ) {
define( 'PRAUTOBLOGGER_OPIK_API_KEY', 'test-key' );
}
if ( ! defined( 'PRAUTOBLOGGER_OPIK_WORKSPACE' ) ) {
define( 'PRAUTOBLOGGER_OPIK_WORKSPACE', 'test-workspace' );
}
$client = $this->create_mock_client();
$queue = $this->create_mock_queue( array() );
$dispatcher = new \PRAutoBlogger_Opik_Dispatcher( $client, $queue );
$dispatcher->dispatch();
// Should not error.
$this->assertTrue( true );
}
/**
* Test dispatcher separates spans and traces.
*/
public function test_dispatch_separates_types(): void {
$this->option_store['prautoblogger_opik_enabled'] = true;
if ( ! defined( 'PRAUTOBLOGGER_OPIK_API_KEY' ) ) {
define( 'PRAUTOBLOGGER_OPIK_API_KEY', 'test-key' );
}
if ( ! defined( 'PRAUTOBLOGGER_OPIK_WORKSPACE' ) ) {
define( 'PRAUTOBLOGGER_OPIK_WORKSPACE', 'test-workspace' );
}
$posted_traces = null;
$posted_spans = null;
$client = $this->getMockBuilder( \PRAutoBlogger_Opik_Client::class )
->disableOriginalConstructor()
->onlyMethods( array( 'batch_create_traces', 'batch_create_spans' ) )
->getMock();
$client->method( 'batch_create_traces' )->willReturnCallback(
function( $traces ) use ( &$posted_traces ) {
$posted_traces = $traces;
return array( 'inserted' => count( $traces ) );
}
);
$client->method( 'batch_create_spans' )->willReturnCallback(
function( $spans ) use ( &$posted_spans ) {
$posted_spans = $spans;
return array( 'inserted' => count( $spans ) );
}
);
$queue = $this->getMockBuilder( \PRAutoBlogger_Opik_Span_Queue::class )
->disableOriginalConstructor()
->onlyMethods( array( 'dequeue', 'get_queue_depth' ) )
->getMock();
$queue->method( 'dequeue' )->willReturn(
array(
array(
'type' => 'trace',
'payload' => array( 'id' => 'trace-1' ),
'attempt' => 0,
'enqueued' => time(),
),
array(
'type' => 'span',
'payload' => array( 'id' => 'span-1' ),
'attempt' => 0,
'enqueued' => time(),
),
)
);
$queue->method( 'get_queue_depth' )->willReturn( 0 );
$dispatcher = new \PRAutoBlogger_Opik_Dispatcher( $client, $queue );
$dispatcher->dispatch();
$this->assertNotNull( $posted_traces );
$this->assertCount( 1, $posted_traces );
$this->assertNotNull( $posted_spans );
$this->assertCount( 1, $posted_spans );
}
/**
* Helper to create a mock client.
*/
private function create_mock_client() {
$client = $this->getMockBuilder( \PRAutoBlogger_Opik_Client::class )
->disableOriginalConstructor()
->onlyMethods( array( 'batch_create_spans', 'batch_create_traces' ) )
->getMock();
$client->method( 'batch_create_spans' )->willReturn( array() );
$client->method( 'batch_create_traces' )->willReturn( array() );
return $client;
}
/**
* Helper to create a mock queue.
*/
private function create_mock_queue( array $items = array() ) {
$queue = $this->getMockBuilder( \PRAutoBlogger_Opik_Span_Queue::class )
->disableOriginalConstructor()
->onlyMethods( array( 'dequeue', 'get_queue_depth' ) )
->getMock();
$queue->method( 'dequeue' )->willReturn( $items );
$queue->method( 'get_queue_depth' )->willReturn( count( $items ) );
return $queue;
}
}

View File

@@ -0,0 +1,248 @@
<?php
declare(strict_types=1);
/**
* Unit tests for PRAutoBlogger_Opik_Span_Queue.
*
* @see includes/services/opik/class-opik-span-queue.php
*/
namespace PRAutoBlogger\Tests\Services;
use PRAutoBlogger\Tests\BaseTestCase;
use Brain\Monkey\Functions;
/**
* @group opik
* @group services
*/
class TestOpikSpanQueue extends BaseTestCase {
private array $option_store = array();
protected function setUp(): void {
parent::setUp();
// Mock WP options functions.
Functions\when( 'get_option' )->alias(
function( $option, $default = false ) {
return $this->option_store[ $option ] ?? $default;
}
);
Functions\when( 'update_option' )->alias(
function( $option, $value ) {
$this->option_store[ $option ] = $value;
return true;
}
);
Functions\when( 'delete_option' )->alias(
function( $option ) {
unset( $this->option_store[ $option ] );
return true;
}
);
}
/**
* Test enqueue() adds item to queue.
*/
public function test_enqueue_adds_item(): void {
$queue = new \PRAutoBlogger_Opik_Span_Queue();
$payload = array(
'id' => 'span-1',
'name' => 'test',
'trace_id' => 'trace-1',
);
$success = $queue->enqueue( $payload, 'span' );
$this->assertTrue( $success );
$this->assertEquals( 1, $queue->get_queue_depth() );
}
/**
* Test enqueue() respects max depth limit.
*/
public function test_enqueue_respects_max_depth(): void {
$queue = new \PRAutoBlogger_Opik_Span_Queue();
// Fill queue to max.
for ( $i = 0; $i < 1000; ++$i ) {
$success = $queue->enqueue(
array( 'id' => "span-$i", 'name' => 'test' ),
'span'
);
if ( ! $success ) {
// Queue full.
$this->assertEquals( 1000, $queue->get_queue_depth() );
return;
}
}
// If we got here, queue filled successfully.
$this->assertEquals( 1000, $queue->get_queue_depth() );
// Try to add one more — should fail.
$success = $queue->enqueue(
array( 'id' => 'overflow', 'name' => 'test' ),
'span'
);
$this->assertFalse( $success );
}
/**
* Test dequeue() returns items in order.
*/
public function test_dequeue_returns_items(): void {
$queue = new \PRAutoBlogger_Opik_Span_Queue();
$queue->enqueue( array( 'id' => 'span-1' ), 'span' );
$queue->enqueue( array( 'id' => 'span-2' ), 'span' );
$queue->enqueue( array( 'id' => 'span-3' ), 'span' );
$batch = $queue->dequeue( 10 );
$this->assertCount( 3, $batch );
$this->assertEquals( 'span-1', $batch[0]['payload']['id'] );
$this->assertEquals( 'span-2', $batch[1]['payload']['id'] );
$this->assertEquals( 'span-3', $batch[2]['payload']['id'] );
}
/**
* Test dequeue() respects limit parameter.
*/
public function test_dequeue_respects_limit(): void {
$queue = new \PRAutoBlogger_Opik_Span_Queue();
for ( $i = 1; $i <= 50; ++$i ) {
$queue->enqueue( array( 'id' => "span-$i" ), 'span' );
}
$batch = $queue->dequeue( 10 );
$this->assertCount( 10, $batch );
$this->assertEquals( 40, $queue->get_queue_depth() );
}
/**
* Test reenqueue() increments attempt counter.
*/
public function test_reenqueue_increments_attempt(): void {
$queue = new \PRAutoBlogger_Opik_Span_Queue();
$items = array(
array(
'type' => 'span',
'payload' => array( 'id' => 'span-1' ),
'attempt' => 0,
'enqueued' => time(),
),
);
$queue->reenqueue( $items );
$batch = $queue->dequeue( 10 );
$this->assertEquals( 1, $batch[0]['attempt'] );
}
/**
* Test reenqueue() drops items after max retries.
*/
public function test_reenqueue_drops_after_max_attempts(): void {
$queue = new \PRAutoBlogger_Opik_Span_Queue();
$items = array(
array(
'type' => 'span',
'payload' => array( 'id' => 'span-1' ),
'attempt' => 3, // Already at max.
'enqueued' => time(),
),
);
$queue->reenqueue( $items );
// Item should be dropped, queue should be empty.
$this->assertEquals( 0, $queue->get_queue_depth() );
}
/**
* Test dequeue() removes expired items (TTL > 12 hours).
*/
public function test_dequeue_removes_expired_items(): void {
$queue = new \PRAutoBlogger_Opik_Span_Queue();
$now = time();
// Add a fresh item.
$fresh = array(
'type' => 'span',
'payload' => array( 'id' => 'fresh' ),
'enqueued' => $now,
);
// Add an expired item (13 hours old).
$expired = array(
'type' => 'span',
'payload' => array( 'id' => 'expired' ),
'enqueued' => $now - 46800, // 13 hours in seconds.
);
$this->option_store['prautoblogger_opik_queue'] = array( $fresh, $expired );
$batch = $queue->dequeue( 10 );
// Only fresh item should be returned.
$this->assertCount( 1, $batch );
$this->assertEquals( 'fresh', $batch[0]['payload']['id'] );
}
/**
* Test get_queue_depth() returns item count.
*/
public function test_get_queue_depth(): void {
$queue = new \PRAutoBlogger_Opik_Span_Queue();
$this->assertEquals( 0, $queue->get_queue_depth() );
$queue->enqueue( array( 'id' => 'span-1' ), 'span' );
$this->assertEquals( 1, $queue->get_queue_depth() );
$queue->enqueue( array( 'id' => 'span-2' ), 'span' );
$this->assertEquals( 2, $queue->get_queue_depth() );
}
/**
* Test clear() empties the queue.
*/
public function test_clear_empties_queue(): void {
$queue = new \PRAutoBlogger_Opik_Span_Queue();
$queue->enqueue( array( 'id' => 'span-1' ), 'span' );
$queue->enqueue( array( 'id' => 'span-2' ), 'span' );
$this->assertEquals( 2, $queue->get_queue_depth() );
$queue->clear();
$this->assertEquals( 0, $queue->get_queue_depth() );
}
/**
* Test enqueue() with both span and trace types.
*/
public function test_enqueue_handles_trace_type(): void {
$queue = new \PRAutoBlogger_Opik_Span_Queue();
$queue->enqueue( array( 'id' => 'trace-1' ), 'trace' );
$queue->enqueue( array( 'id' => 'span-1' ), 'span' );
$batch = $queue->dequeue( 10 );
$this->assertCount( 2, $batch );
$this->assertEquals( 'trace', $batch[0]['type'] );
$this->assertEquals( 'span', $batch[1]['type'] );
}
}

View File

@@ -0,0 +1,222 @@
<?php
declare(strict_types=1);
/**
* Unit tests for PRAutoBlogger_Opik_Trace_Context.
*
* @see includes/services/opik/class-opik-trace-context.php
*/
namespace PRAutoBlogger\Tests\Services;
use PRAutoBlogger\Tests\BaseTestCase;
use Brain\Monkey\Functions;
/**
* @group opik
* @group services
*/
class TestOpikTraceContext extends BaseTestCase {
protected function setUp(): void {
parent::setUp();
// Stub UUID generation.
Functions\when( 'wp_generate_uuid4' )->alias( function() {
static $counter = 0;
return 'uuid-' . ++$counter;
} );
}
protected function tearDown(): void {
// Clean up singleton.
\PRAutoBlogger_Opik_Trace_Context::teardown();
parent::tearDown();
}
/**
* Test current() returns singleton instance.
*/
public function test_current_returns_singleton(): void {
$ctx1 = \PRAutoBlogger_Opik_Trace_Context::current();
$ctx2 = \PRAutoBlogger_Opik_Trace_Context::current();
$this->assertSame( $ctx1, $ctx2 );
}
/**
* Test init_trace() generates a UUID.
*/
public function test_init_trace_generates_uuid(): void {
$ctx = \PRAutoBlogger_Opik_Trace_Context::current();
$trace_id = $ctx->init_trace();
$this->assertNotEmpty( $trace_id );
$this->assertStringStartsWith( 'uuid-', $trace_id );
$this->assertEquals( $trace_id, $ctx->get_trace_id() );
}
/**
* Test start_span() creates span and returns ID.
*/
public function test_start_span_creates_span(): void {
$ctx = \PRAutoBlogger_Opik_Trace_Context::current();
$ctx->init_trace();
$span_id = $ctx->start_span(
array(
'name' => 'draft_generation',
'type' => 'llm',
'model' => 'gpt-4',
'provider' => 'openrouter',
'input' => array( 'prompt_hash' => 'sha256-abc' ),
)
);
$this->assertNotEmpty( $span_id );
$this->assertStringStartsWith( 'uuid-', $span_id );
$span = $ctx->get_span( $span_id );
$this->assertIsArray( $span );
$this->assertEquals( 'draft_generation', $span['name'] );
$this->assertEquals( 'llm', $span['type'] );
$this->assertEquals( 'gpt-4', $span['model'] );
}
/**
* Test end_span() records output and usage.
*/
public function test_end_span_records_data(): void {
$ctx = \PRAutoBlogger_Opik_Trace_Context::current();
$ctx->init_trace();
$span_id = $ctx->start_span(
array(
'name' => 'test_span',
'type' => 'llm',
)
);
$ctx->end_span(
$span_id,
array(
'output' => array( 'response' => 'Hello world' ),
'usage' => array(
'prompt_tokens' => 50,
'completion_tokens' => 25,
'total_tokens' => 75,
),
)
);
$span = $ctx->get_span( $span_id );
$this->assertIsArray( $span['output'] );
$this->assertEquals( 'Hello world', $span['output']['response'] );
$this->assertEquals( 75, $span['usage']['total_tokens'] );
}
/**
* Test get_spans() returns all active spans.
*/
public function test_get_spans_returns_all(): void {
$ctx = \PRAutoBlogger_Opik_Trace_Context::current();
$ctx->init_trace();
$span1 = $ctx->start_span( array( 'name' => 'span1', 'type' => 'llm' ) );
$span2 = $ctx->start_span( array( 'name' => 'span2', 'type' => 'llm' ) );
$span3 = $ctx->start_span( array( 'name' => 'span3', 'type' => 'llm' ) );
$all_spans = $ctx->get_spans();
$this->assertCount( 3, $all_spans );
$this->assertArrayHasKey( $span1, $all_spans );
$this->assertArrayHasKey( $span2, $all_spans );
$this->assertArrayHasKey( $span3, $all_spans );
}
/**
* Test finalize_trace() returns complete trace object.
*/
public function test_finalize_trace_includes_all_data(): void {
Functions\when( 'get_option' )->alias(
function( $option, $default = false ) {
if ( 'prautoblogger_opik_project_name' === $option ) {
return 'prautoblogger';
}
return $default;
}
);
$ctx = \PRAutoBlogger_Opik_Trace_Context::current();
$trace_id = $ctx->init_trace();
$span1 = $ctx->start_span( array( 'name' => 'step1', 'type' => 'llm' ) );
$ctx->end_span( $span1, array( 'output' => array( 'ok' => true ) ) );
$span2 = $ctx->start_span( array( 'name' => 'step2', 'type' => 'llm' ) );
$ctx->end_span( $span2, array( 'output' => array( 'ok' => true ) ) );
$trace = $ctx->finalize_trace();
$this->assertEquals( $trace_id, $trace['id'] );
$this->assertEquals( 'article_generation', $trace['name'] );
$this->assertEquals( 'prautoblogger', $trace['project_name'] );
$this->assertArrayHasKey( 'start_time', $trace );
$this->assertArrayHasKey( 'end_time', $trace );
$this->assertCount( 2, $trace['spans'] );
}
/**
* Test teardown() clears the singleton.
*/
public function test_teardown_clears_singleton(): void {
$ctx1 = \PRAutoBlogger_Opik_Trace_Context::current();
$ctx1->init_trace();
\PRAutoBlogger_Opik_Trace_Context::teardown();
$ctx2 = \PRAutoBlogger_Opik_Trace_Context::current();
// Should be a new instance.
$this->assertNotSame( $ctx1, $ctx2 );
$this->assertEmpty( $ctx2->get_trace_id() );
}
/**
* Test warning logged for ending unknown span.
*/
public function test_warning_for_unknown_span(): void {
$ctx = \PRAutoBlogger_Opik_Trace_Context::current();
$ctx->init_trace();
// Try to end a span that doesn't exist.
$ctx->end_span( 'unknown-uuid-12345' );
// If we got here without an exception, the warning was logged.
$this->assertTrue( true );
}
/**
* Test span input is stored as-is (caller responsible for redaction).
*/
public function test_span_input_stored_as_is(): void {
$ctx = \PRAutoBlogger_Opik_Trace_Context::current();
$ctx->init_trace();
$input_data = array(
'prompt_hash' => 'sha256-abc123',
'model' => 'gpt-4',
'version' => '1.0',
);
$span_id = $ctx->start_span(
array(
'name' => 'test',
'type' => 'llm',
'input' => $input_data,
)
);
$span = $ctx->get_span( $span_id );
$this->assertEquals( $input_data, $span['input'] );
}
}