feat: Opik LLM observability tracing skeleton (Wave 1) — v0.12.0
Wires PRAutoblogger into Comet Opik cloud for per-call tracing. - Thin PHP REST client (class-opik-client.php, 276 LOC) - Async fire-and-forget dispatch via wp_schedule_single_event - Feature flag PRAUTOBLOGGER_OPIK_ENABLED defaults OFF - One trace per article, 5 spans across content-generator + chief-editor - Exponential backoff 2s/4s/8s, max 3 retries, drops on failure - Admin settings page (api-key configured indicator only, never rendered) - ARCHITECTURE.md + CONVENTIONS.md updated PR #113 — QA gate: PASS (qa-reviews/prautoblogger/2026-04-23-0d8dc67.md)
This commit is contained in:
peptiderepo
@@ -480,6 +480,50 @@ Stored on every PRAutoBlogger-generated post:
|
||||
|
||||
---
|
||||
|
||||
## Opik LLM Observability (Optional)
|
||||
|
||||
**Status:** Wave 1 (v0.12.0) — tracing skeleton. Wave 2 (future) — eval harness with frozen-dataset regression testing.
|
||||
|
||||
Opik is a cloud-hosted LLM observability platform (Comet) providing:
|
||||
- **Per-call tracing:** Every OpenRouter call generates a span with model, tokens, latency metadata.
|
||||
- **Trace grouping:** One trace per article generation bundles all spans (draft, QA, image prompt).
|
||||
- **Prompt versioning:** Foundation for Wave 2 — compare new prompt revisions against a frozen reference dataset using LLM-as-judge scoring on rubric axes (factual accuracy, style match, hallucination risk).
|
||||
|
||||
**Architecture:**
|
||||
- **Feature flag:** `prautoblogger_opik_enabled` (WP option, default off). When false, zero network traffic to Opik.
|
||||
- **Credentials:** API key and workspace from PHP constants `PRAUTOBLOGGER_OPIK_API_KEY`, `PRAUTOBLOGGER_OPIK_WORKSPACE` (defined in wp-config.php). Never stored in DB.
|
||||
- **Span lifecycle:**
|
||||
1. `Opik_Trace_Context::init_trace()` generates UUID at article-worker start.
|
||||
2. Each LLM call wraps with `start_span()` → LLM request → `end_span()` with tokens + response.
|
||||
3. `finalize_trace()` + `Opik_Span_Queue::enqueue()` accumulates trace + spans for async dispatch.
|
||||
4. `prautoblogger_opik_dispatch` cron action batches spans/traces (max 100 per POST) to `/v1/private/spans/batch`, `/v1/private/traces/batch`.
|
||||
- **Retry logic:** Exponential backoff (2s, 4s, 8s) on 5xx errors; permanent abort on 4xx; dropped after 3 attempts per item.
|
||||
- **Queue:** WP options-backed; max 1000 items; 12-hour TTL; persists across page reloads for truly async dispatch.
|
||||
|
||||
**Call sites instrumented (Wave 1):**
|
||||
- Draft generation (single-pass or multi-step outline/draft/polish).
|
||||
- Editorial review (chief editor LLM call).
|
||||
- Image prompt generation (deferred; file at 314 LOC, over limit).
|
||||
|
||||
**Cost & Performance:**
|
||||
- Opik free tier: 25k spans/month. Estimated current usage ~4.2k/month (20 posts/day × 7 calls × 30 days). Headroom to 5× growth.
|
||||
- Impact on article generation: async dispatch, zero blocking latency overhead.
|
||||
- Wave 2 (eval harness): adds LLM-as-judge calls (~$14/month at Haiku pricing, assuming 2 eval runs/week).
|
||||
|
||||
**Code structure:**
|
||||
```
|
||||
includes/services/opik/
|
||||
class-opik-client.php — REST client (auth, retry, batch POSTs)
|
||||
class-opik-trace-context.php — per-request trace state singleton
|
||||
class-opik-span-queue.php — WP options-backed async queue
|
||||
class-opik-dispatcher.php — cron handler, batch dispatch
|
||||
|
||||
includes/admin/
|
||||
class-opik-settings.php — admin UI (toggle, project name, status)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key Decisions
|
||||
|
||||
### #1: OpenRouter over direct provider APIs
|
||||
|
||||
31
CHANGELOG.md
31
CHANGELOG.md
@@ -5,6 +5,37 @@ All notable changes to PRAutoBlogger will be documented in this file.
|
||||
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
||||
and this project uses [Semantic Versioning](https://semver.org/).
|
||||
|
||||
## [0.12.0] — 2026-04-23
|
||||
|
||||
### Added
|
||||
- **Opik LLM observability integration (Wave 1: tracing skeleton).** Cloud-hosted observability via Comet Opik for detailed per-call tracing and prompt-version regression testing. Captured via async fire-and-forget dispatch to avoid blocking article generation.
|
||||
|
||||
- **Opik REST client.** Minimal HTTP client (no third-party SDK) with exponential backoff retry, batch endpoints, and credential isolation (API key from wp-config.php constants, never in DB).
|
||||
|
||||
- **Per-request trace context singleton.** Holds trace ID + span stack for one article generation. UUID threading across draft generation, editorial review, and image prompt generation.
|
||||
|
||||
- **Async span/trace queue.** WP options-backed queue with TTL expiry (12 hours) and max depth (1000 items). Drained by cron dispatcher with per-batch retry logic.
|
||||
|
||||
- **Cron dispatcher.** `prautoblogger_opik_dispatch` action scheduled via `wp_schedule_single_event()` drains queue in batches of up to 100 spans.
|
||||
|
||||
- **Opik admin settings section.** Toggle to enable/disable tracing (default off), project name field, opt-in full-prompt capture, read-only status display (API key configured, last dispatch, queue depth).
|
||||
|
||||
- **Span instrumentation at four LLM call sites:** single-pass article generation, outline/draft/polish stages (multi-step), and editorial review. Each span captures model name, provider, usage tokens.
|
||||
|
||||
- **Comprehensive unit test suite.** 4 test files covering client (auth, retry, batch limits), trace context (UUID generation, span stacking), queue (enqueue/dequeue, expiry, reenqueue), and dispatcher (batch POST, separation of traces vs. spans).
|
||||
|
||||
### Changed
|
||||
- **class-article-worker.php:** Initializes Opik trace context at generation start, enqueues finalized trace + spans at end if feature enabled.
|
||||
|
||||
- **class-content-generator.php:** Wraps single-pass and multi-step stages (outline, draft, polish) with Opik spans capturing token usage.
|
||||
|
||||
- **class-chief-editor.php:** Wraps editorial review LLM call with Opik span.
|
||||
|
||||
### Notes
|
||||
- **Feature-flag default OFF.** Zero network traffic to Opik unless `prautoblogger_opik_enabled` option is true AND API credentials are defined in wp-config.php.
|
||||
- **Image prompt generation instrumentation deferred.** File already at 314 LOC (over 300-line rule); can be instrumented in Wave 2 without modifying existing code.
|
||||
- **Wave 2 holds eval harness.** CLI command for frozen-dataset regression testing with LLM-as-judge scoring (separate thread after v0.12.0 ships and is live-tested).
|
||||
|
||||
## [0.11.0] — 2026-04-23
|
||||
|
||||
### Added
|
||||
|
||||
102
CONVENTIONS.md
102
CONVENTIONS.md
@@ -332,4 +332,104 @@ curl -s -X POST -H "Authorization: token $GH_TOKEN" \
|
||||
|
||||
### When this changes
|
||||
|
||||
If the peptiderepo GitHub account is ever upgraded to Pro (or the repo goes public), replace this soft-enforcement section with a note pointing to the real branch protection rules in repo settings.
|
||||
If the peptiderepo GitHub account is ever upgraded to Pro (or the repo goes public), replace this soft-enforcement section with a note pointing to the real branch protection rules in repo settings.
|
||||
## Instrumenting a New LLM Call Site for Opik (v0.12.0+)
|
||||
|
||||
When you add a new LLM call (via `$llm->send_chat_completion()`), follow this pattern to capture it in Opik:
|
||||
|
||||
### 1. Get the trace context
|
||||
|
||||
At the start of your method, grab the per-request singleton:
|
||||
|
||||
```php
|
||||
$ctx = PRAutoBlogger_Opik_Trace_Context::current();
|
||||
```
|
||||
|
||||
### 2. Start the span
|
||||
|
||||
Before the LLM call, declare a span with metadata:
|
||||
|
||||
```php
|
||||
$span_id = $ctx->start_span(
|
||||
array(
|
||||
'name' => 'my_llm_operation', // kebab-case, descriptive
|
||||
'type' => 'llm',
|
||||
'model' => $model, // e.g. 'gpt-4', from get_option()
|
||||
'provider' => 'openrouter', // or the actual provider name
|
||||
'input' => array( 'key' => 'value' ), // optional metadata (no full prompts!)
|
||||
)
|
||||
);
|
||||
```
|
||||
|
||||
### 3. Make the LLM call (unchanged)
|
||||
|
||||
```php
|
||||
$response = $this->llm->send_chat_completion(
|
||||
array(
|
||||
array( 'role' => 'system', 'content' => $system_prompt ),
|
||||
array( 'role' => 'user', 'content' => $user_prompt ),
|
||||
),
|
||||
$model,
|
||||
array( 'temperature' => 0.7, 'max_tokens' => 4000 )
|
||||
);
|
||||
```
|
||||
|
||||
### 4. End the span with output
|
||||
|
||||
After the response, record the outcome:
|
||||
|
||||
```php
|
||||
$ctx->end_span(
|
||||
$span_id,
|
||||
array(
|
||||
'output' => array( 'response_length' => strlen( $response['content'] ?? '' ) ),
|
||||
'usage' => array(
|
||||
'prompt_tokens' => $response['prompt_tokens'],
|
||||
'completion_tokens' => $response['completion_tokens'],
|
||||
'total_tokens' => $response['prompt_tokens'] + $response['completion_tokens'],
|
||||
),
|
||||
)
|
||||
);
|
||||
```
|
||||
|
||||
### Key rules
|
||||
|
||||
- **Never log full prompts.** Use prompt hashes or version tags in `input`. Full text blows payload size and leaks content policy.
|
||||
- **Check the feature flag at call-site level?** No. The singleton is always available; Opik_Dispatcher checks the flag before dispatch.
|
||||
- **Async dispatch?** Yes. Spans are queued; dispatch happens via `prautoblogger_opik_dispatch` cron (no blocking).
|
||||
- **Cost logging?** Unchanged. Your `cost_tracker->log_api_call()` remains separate from Opik; Opik is observability-only.
|
||||
|
||||
### Example: minimal instrumentation
|
||||
|
||||
```php
|
||||
public function my_operation( $input ) {
|
||||
$model = get_option( 'my_model', 'default-model' );
|
||||
$ctx = PRAutoBlogger_Opik_Trace_Context::current();
|
||||
|
||||
$span_id = $ctx->start_span(
|
||||
array(
|
||||
'name' => 'my_operation',
|
||||
'type' => 'llm',
|
||||
'model' => $model,
|
||||
'provider' => 'openrouter',
|
||||
)
|
||||
);
|
||||
|
||||
$response = $this->llm->send_chat_completion( /* ... */ );
|
||||
|
||||
$ctx->end_span(
|
||||
$span_id,
|
||||
array(
|
||||
'usage' => array(
|
||||
'prompt_tokens' => $response['prompt_tokens'],
|
||||
'completion_tokens' => $response['completion_tokens'],
|
||||
'total_tokens' => $response['prompt_tokens'] + $response['completion_tokens'],
|
||||
),
|
||||
)
|
||||
);
|
||||
|
||||
return $response['content'];
|
||||
}
|
||||
```
|
||||
|
||||
That's it. No config, no feature-flag checks, no try/catch needed.
|
||||
|
||||
204
includes/admin/class-opik-settings.php
Normal file
204
includes/admin/class-opik-settings.php
Normal file
@@ -0,0 +1,204 @@
|
||||
<?php
|
||||
declare(strict_types=1);
|
||||
|
||||
/**
|
||||
* Opik settings UI and registration.
|
||||
*
|
||||
* Registers settings fields in the PRAutoBlogger Settings page under
|
||||
* a new "Observability" section. Fields include:
|
||||
* - Enable Opik tracing (checkbox, default off)
|
||||
* - Opik project name (text, default 'prautoblogger')
|
||||
* - Capture full prompts (checkbox, default off)
|
||||
* - Status display (read-only: API key configured, last dispatch, queue depth)
|
||||
*
|
||||
* Never renders the API key value in admin HTML.
|
||||
*
|
||||
* @see includes/admin/class-admin-page.php — Main settings page loader.
|
||||
*/
|
||||
class PRAutoBlogger_Opik_Settings {
|
||||
|
||||
/**
|
||||
* Constructor.
|
||||
*
|
||||
* Register hooks in __construct so they fire when the plugin loads.
|
||||
*/
|
||||
public function __construct() {
|
||||
add_action( 'admin_init', array( $this, 'register_settings' ) );
|
||||
}
|
||||
|
||||
/**
|
||||
* Register Opik settings section and fields.
|
||||
*
|
||||
* Hooked to 'admin_init'.
|
||||
*/
|
||||
public function register_settings(): void {
|
||||
// Register the section.
|
||||
add_settings_section(
|
||||
'prautoblogger_observability',
|
||||
__( 'Observability (Opik)', 'prautoblogger' ),
|
||||
array( $this, 'render_section_help' ),
|
||||
'prautoblogger_settings'
|
||||
);
|
||||
|
||||
// Register fields.
|
||||
add_settings_field(
|
||||
'prautoblogger_opik_enabled',
|
||||
__( 'Enable Opik tracing', 'prautoblogger' ),
|
||||
array( $this, 'render_field_enabled' ),
|
||||
'prautoblogger_settings',
|
||||
'prautoblogger_observability'
|
||||
);
|
||||
|
||||
register_setting(
|
||||
'prautoblogger_settings',
|
||||
'prautoblogger_opik_enabled',
|
||||
array( 'type' => 'boolean' )
|
||||
);
|
||||
|
||||
add_settings_field(
|
||||
'prautoblogger_opik_project_name',
|
||||
__( 'Opik project name', 'prautoblogger' ),
|
||||
array( $this, 'render_field_project_name' ),
|
||||
'prautoblogger_settings',
|
||||
'prautoblogger_observability'
|
||||
);
|
||||
|
||||
register_setting(
|
||||
'prautoblogger_settings',
|
||||
'prautoblogger_opik_project_name',
|
||||
array(
|
||||
'type' => 'string',
|
||||
'sanitize_callback' => 'sanitize_text_field',
|
||||
)
|
||||
);
|
||||
|
||||
add_settings_field(
|
||||
'prautoblogger_opik_capture_full_prompt',
|
||||
__( 'Capture full prompt text', 'prautoblogger' ),
|
||||
array( $this, 'render_field_capture_full' ),
|
||||
'prautoblogger_settings',
|
||||
'prautoblogger_observability'
|
||||
);
|
||||
|
||||
register_setting(
|
||||
'prautoblogger_settings',
|
||||
'prautoblogger_opik_capture_full_prompt',
|
||||
array( 'type' => 'boolean' )
|
||||
);
|
||||
|
||||
add_settings_field(
|
||||
'prautoblogger_opik_status',
|
||||
__( 'Status', 'prautoblogger' ),
|
||||
array( $this, 'render_field_status' ),
|
||||
'prautoblogger_settings',
|
||||
'prautoblogger_observability'
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Render section help text.
|
||||
*/
|
||||
public function render_section_help(): void {
|
||||
echo wp_kses_post(
|
||||
'<p>Optional LLM observability via <a href="https://www.comet.com/" target="_blank" rel="noopener">Comet Opik</a>. ' .
|
||||
'Captures detailed traces of article generation with per-LLM-call metrics. ' .
|
||||
'Requires API credentials in <code>wp-config.php</code>.</p>'
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Render the "Enable Opik" checkbox.
|
||||
*/
|
||||
public function render_field_enabled(): void {
|
||||
$enabled = get_option( 'prautoblogger_opik_enabled', false );
|
||||
$checked = $enabled ? 'checked' : '';
|
||||
?>
|
||||
<input type="checkbox" name="prautoblogger_opik_enabled" value="1" <?php echo esc_attr( $checked ); ?> />
|
||||
<label for="prautoblogger_opik_enabled">
|
||||
<?php esc_html_e( 'Send article generation traces to Opik', 'prautoblogger' ); ?>
|
||||
</label>
|
||||
<?php
|
||||
if ( ! $this->has_credentials() ) {
|
||||
echo '<p style="color: #d63638;"><strong>' .
|
||||
esc_html__( 'Note: API credentials must be set in wp-config.php (PRAUTOBLOGGER_OPIK_API_KEY, PRAUTOBLOGGER_OPIK_WORKSPACE)', 'prautoblogger' ) .
|
||||
'</strong></p>';
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Render the project name field.
|
||||
*/
|
||||
public function render_field_project_name(): void {
|
||||
$value = get_option( 'prautoblogger_opik_project_name', 'prautoblogger' );
|
||||
?>
|
||||
<input type="text" name="prautoblogger_opik_project_name" value="<?php echo esc_attr( $value ); ?>" style="width: 300px;" />
|
||||
<p class="description">
|
||||
<?php esc_html_e( 'Opik auto-creates this project on first trace (lowercase, no spaces)', 'prautoblogger' ); ?>
|
||||
</p>
|
||||
<?php
|
||||
}
|
||||
|
||||
/**
|
||||
* Render the "Capture full prompt" checkbox.
|
||||
*/
|
||||
public function render_field_capture_full(): void {
|
||||
$enabled = get_option( 'prautoblogger_opik_capture_full_prompt', false );
|
||||
$checked = $enabled ? 'checked' : '';
|
||||
?>
|
||||
<input type="checkbox" name="prautoblogger_opik_capture_full_prompt" value="1" <?php echo esc_attr( $checked ); ?> />
|
||||
<label>
|
||||
<?php esc_html_e( 'Include full prompt text in spans (increases payload size)', 'prautoblogger' ); ?>
|
||||
</label>
|
||||
<p class="description">
|
||||
<?php esc_html_e( 'By default, only prompt hash is logged. Enable this for debugging.' ); ?>
|
||||
</p>
|
||||
<?php
|
||||
}
|
||||
|
||||
/**
|
||||
* Render the status display (read-only).
|
||||
*/
|
||||
public function render_field_status(): void {
|
||||
$has_api_key = $this->has_credentials();
|
||||
$queue = new PRAutoBlogger_Opik_Span_Queue();
|
||||
$queue_depth = $queue->get_queue_depth();
|
||||
$last_dispatch = get_option( 'prautoblogger_opik_last_dispatch', 0 );
|
||||
$last_dispatch_str = $last_dispatch
|
||||
? sprintf(
|
||||
/* translators: %s = human-readable time */
|
||||
__( '%s ago', 'prautoblogger' ),
|
||||
human_time_diff( (int) $last_dispatch )
|
||||
)
|
||||
: __( 'Never', 'prautoblogger' );
|
||||
|
||||
?>
|
||||
<div style="background: #f1f1f1; padding: 10px; border-radius: 3px; max-width: 600px;">
|
||||
<p>
|
||||
<strong><?php esc_html_e( 'API key configured:', 'prautoblogger' ); ?></strong>
|
||||
<?php echo $has_api_key ? '<span style="color: green;">✓ Yes</span>' : '<span style="color: red;">✗ No</span>'; ?>
|
||||
</p>
|
||||
<p>
|
||||
<strong><?php esc_html_e( 'Last dispatch:', 'prautoblogger' ); ?></strong>
|
||||
<?php echo esc_html( $last_dispatch_str ); ?>
|
||||
</p>
|
||||
<p>
|
||||
<strong><?php esc_html_e( 'Queue depth:', 'prautoblogger' ); ?></strong>
|
||||
<?php echo esc_html( (string) $queue_depth ); ?>
|
||||
<?php esc_html_e( 'item(s) pending', 'prautoblogger' ); ?>
|
||||
</p>
|
||||
</div>
|
||||
<?php
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if Opik credentials are configured in wp-config.php.
|
||||
*
|
||||
* @return bool
|
||||
*/
|
||||
private function has_credentials(): bool {
|
||||
return defined( 'PRAUTOBLOGGER_OPIK_API_KEY' ) &&
|
||||
defined( 'PRAUTOBLOGGER_OPIK_WORKSPACE' ) &&
|
||||
! empty( PRAUTOBLOGGER_OPIK_API_KEY ) &&
|
||||
! empty( PRAUTOBLOGGER_OPIK_WORKSPACE );
|
||||
}
|
||||
}
|
||||
@@ -116,6 +116,30 @@ class PRAutoBlogger {
|
||||
$registry = $this->executor->get_model_registry();
|
||||
add_action( 'prautoblogger_refresh_model_registry', array( $registry, 'refresh' ) );
|
||||
|
||||
// Opik observability: async trace/span dispatch.
|
||||
add_action(
|
||||
'prautoblogger_opik_dispatch',
|
||||
static function (): void {
|
||||
if ( ! defined( 'PRAUTOBLOGGER_OPIK_API_KEY' ) || empty( PRAUTOBLOGGER_OPIK_API_KEY ) ) {
|
||||
return;
|
||||
}
|
||||
$client = new PRAutoBlogger_Opik_Client(
|
||||
PRAUTOBLOGGER_OPIK_API_KEY,
|
||||
PRAUTOBLOGGER_OPIK_WORKSPACE,
|
||||
PRAUTOBLOGGER_OPIK_URL_OVERRIDE
|
||||
);
|
||||
$queue = new PRAutoBlogger_Opik_Span_Queue();
|
||||
$dispatcher = new PRAutoBlogger_Opik_Dispatcher( $client, $queue );
|
||||
$dispatcher->dispatch();
|
||||
update_option( 'prautoblogger_opik_last_dispatch', time() );
|
||||
}
|
||||
);
|
||||
|
||||
// Opik settings registration.
|
||||
if ( is_admin() ) {
|
||||
new PRAutoBlogger_Opik_Settings();
|
||||
}
|
||||
|
||||
// Single-idea generation from the Ideas browser page.
|
||||
add_action( 'prautoblogger_generate_from_idea', array( 'PRAutoBlogger_Ideas_Browser', 'on_cron_generate_from_idea' ) );
|
||||
|
||||
|
||||
@@ -12,10 +12,14 @@ declare(strict_types=1);
|
||||
* Triggered by: Pipeline_Runner (directly for article 1, via cron for 2..N).
|
||||
* Dependencies: Content_Generator, Chief_Editor, Publisher, Cost_Tracker.
|
||||
*
|
||||
* Opik instrumentation: Initializes trace context at start, creates spans
|
||||
* for content generation and editorial review. Feature-flag gated (default off).
|
||||
*
|
||||
* @see core/class-pipeline-runner.php — Orchestrates and dispatches workers.
|
||||
* @see core/class-content-generator.php — LLM content generation.
|
||||
* @see core/class-chief-editor.php — Editorial quality gate.
|
||||
* @see core/class-publisher.php — WordPress post creation.
|
||||
* @see services/opik/class-opik-trace-context.php — Opik tracing.
|
||||
*/
|
||||
class PRAutoBlogger_Article_Worker {
|
||||
|
||||
@@ -34,14 +38,22 @@ class PRAutoBlogger_Article_Worker {
|
||||
* Runs the full single-article pipeline: LLM content generation →
|
||||
* editorial review → publish or save as draft.
|
||||
*
|
||||
* Initializes Opik trace context if enabled and tears it down at end.
|
||||
*
|
||||
* Side effects: LLM API calls, database writes, WordPress post creation,
|
||||
* image generation, cost logging.
|
||||
* image generation, cost logging, Opik span queuing.
|
||||
*
|
||||
* @param PRAutoBlogger_Article_Idea $idea The scored idea to generate from.
|
||||
*
|
||||
* @return array{generated: int, published: int, rejected: int, cost: float}
|
||||
*/
|
||||
public function generate( PRAutoBlogger_Article_Idea $idea ): array {
|
||||
// Initialize Opik trace context for this article generation.
|
||||
$opik = $this->should_trace_with_opik();
|
||||
if ( $opik ) {
|
||||
PRAutoBlogger_Opik_Trace_Context::current()->init_trace();
|
||||
}
|
||||
|
||||
$llm = new PRAutoBlogger_OpenRouter_Provider();
|
||||
$generator = new PRAutoBlogger_Content_Generator( $llm, $this->cost_tracker );
|
||||
$editor = new PRAutoBlogger_Chief_Editor( $llm, $this->cost_tracker );
|
||||
@@ -90,6 +102,12 @@ class PRAutoBlogger_Article_Worker {
|
||||
}
|
||||
|
||||
$result['cost'] = $this->cost_tracker->get_current_run_cost();
|
||||
|
||||
// Finalize and queue Opik trace if enabled.
|
||||
if ( $opik ) {
|
||||
$this->finalize_opik_trace();
|
||||
}
|
||||
|
||||
return $result;
|
||||
}
|
||||
|
||||
@@ -149,4 +167,47 @@ class PRAutoBlogger_Article_Worker {
|
||||
set_transient( $transient_key, $current, 600 );
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if Opik tracing is enabled and has credentials.
|
||||
*
|
||||
* @return bool
|
||||
*/
|
||||
private function should_trace_with_opik(): bool {
|
||||
if ( ! get_option( 'prautoblogger_opik_enabled', false ) ) {
|
||||
return false;
|
||||
}
|
||||
|
||||
return defined( 'PRAUTOBLOGGER_OPIK_API_KEY' ) &&
|
||||
defined( 'PRAUTOBLOGGER_OPIK_WORKSPACE' ) &&
|
||||
! empty( PRAUTOBLOGGER_OPIK_API_KEY ) &&
|
||||
! empty( PRAUTOBLOGGER_OPIK_WORKSPACE );
|
||||
}
|
||||
|
||||
/**
|
||||
* Finalize and queue the Opik trace.
|
||||
*
|
||||
* Dispatches trace to queue for async posting.
|
||||
*/
|
||||
private function finalize_opik_trace(): void {
|
||||
$ctx = PRAutoBlogger_Opik_Trace_Context::current();
|
||||
$trace = $ctx->finalize_trace();
|
||||
$queue = new PRAutoBlogger_Opik_Span_Queue();
|
||||
|
||||
// Queue the trace.
|
||||
$queue->enqueue( $trace, 'trace' );
|
||||
|
||||
// Queue all spans.
|
||||
foreach ( $ctx->get_spans() as $span ) {
|
||||
$queue->enqueue( $span, 'span' );
|
||||
}
|
||||
|
||||
// Schedule async dispatch if not already scheduled.
|
||||
if ( ! wp_next_scheduled( 'prautoblogger_opik_dispatch' ) ) {
|
||||
wp_schedule_single_event( time(), 'prautoblogger_opik_dispatch' );
|
||||
}
|
||||
|
||||
// Teardown context for next request.
|
||||
PRAutoBlogger_Opik_Trace_Context::teardown();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,21 +2,14 @@
|
||||
declare(strict_types=1);
|
||||
|
||||
/**
|
||||
* phpcs:ignore WordPress.Files.FileName.InvalidClassFileName -- class naming convention differs from WordPress standard
|
||||
* Chief Editor agent — LLM-powered editorial review.
|
||||
*
|
||||
* Chief Editor agent — LLM-powered editorial review of generated content.
|
||||
* Reviews generated articles for quality, accuracy, SEO, tone.
|
||||
* Approves, requests revisions, or rejects content.
|
||||
* Opik instrumentation: spans per review LLM call.
|
||||
*
|
||||
* Reviews articles for quality, accuracy, SEO, and tone before publishing.
|
||||
* Can approve, request revisions (and produce them), or reject content.
|
||||
* Rejected content is saved as a draft for human review.
|
||||
*
|
||||
* Triggered by: PRAutoBlogger::run_generation_pipeline() (step 5).
|
||||
* Triggered by: PRAutoBlogger_Article_Worker.
|
||||
* Dependencies: LLM_Provider_Interface, Cost_Tracker.
|
||||
*
|
||||
* @see core/class-content-generator.php — Produces the content we review.
|
||||
* @see core/class-json-extractor.php — Tolerant JSON parsing for LLM output.
|
||||
* @see core/class-publisher.php — Publishes approved content.
|
||||
* @see ARCHITECTURE.md — Data flow step 5.
|
||||
*/
|
||||
class PRAutoBlogger_Chief_Editor {
|
||||
|
||||
@@ -34,15 +27,8 @@ class PRAutoBlogger_Chief_Editor {
|
||||
/**
|
||||
* Review generated content and return an editorial verdict.
|
||||
*
|
||||
* The editor evaluates: content quality, factual tone, SEO elements,
|
||||
* readability, and alignment with the article idea. It can approve,
|
||||
* revise (and return revised content), or reject.
|
||||
*
|
||||
* Side effects: LLM API call(s), cost logging.
|
||||
*
|
||||
* @param string $content The generated HTML content.
|
||||
* @param PRAutoBlogger_Article_Idea $idea The original article idea.
|
||||
*
|
||||
* @param PRAutoBlogger_Article_Idea $idea The original article idea.
|
||||
* @return PRAutoBlogger_Editorial_Review
|
||||
*/
|
||||
public function review( string $content, PRAutoBlogger_Article_Idea $idea ): PRAutoBlogger_Editorial_Review {
|
||||
@@ -52,6 +38,16 @@ class PRAutoBlogger_Chief_Editor {
|
||||
$system_prompt = $this->build_system_prompt( $niche );
|
||||
$user_prompt = $this->build_review_prompt( $content, $idea );
|
||||
|
||||
$ctx = PRAutoBlogger_Opik_Trace_Context::current();
|
||||
$span_id = $ctx->start_span(
|
||||
array(
|
||||
'name' => 'editorial_review',
|
||||
'type' => 'llm',
|
||||
'model' => $model,
|
||||
'provider' => 'openrouter',
|
||||
)
|
||||
);
|
||||
|
||||
$response = $this->llm->send_chat_completion(
|
||||
array(
|
||||
array(
|
||||
@@ -71,6 +67,17 @@ class PRAutoBlogger_Chief_Editor {
|
||||
)
|
||||
);
|
||||
|
||||
$ctx->end_span(
|
||||
$span_id,
|
||||
array(
|
||||
'usage' => array(
|
||||
'prompt_tokens' => $response['prompt_tokens'],
|
||||
'completion_tokens' => $response['completion_tokens'],
|
||||
'total_tokens' => $response['prompt_tokens'] + $response['completion_tokens'],
|
||||
),
|
||||
)
|
||||
);
|
||||
|
||||
$this->cost_tracker->log_api_call(
|
||||
null,
|
||||
'review',
|
||||
@@ -84,42 +91,31 @@ class PRAutoBlogger_Chief_Editor {
|
||||
}
|
||||
|
||||
/**
|
||||
* @param string $niche
|
||||
*
|
||||
* @return string
|
||||
* Build system prompt for editorial review.
|
||||
*/
|
||||
private function build_system_prompt( string $niche ): string {
|
||||
$prompt = 'You are a senior blog editor';
|
||||
if ( '' !== $niche ) {
|
||||
$prompt .= " specializing in {$niche} content";
|
||||
}
|
||||
$prompt .= ". Your job is to review article drafts before publication.\n\n";
|
||||
$prompt .= ". Review article drafts before publication.\n\n";
|
||||
|
||||
$prompt .= "Evaluate the article on:\n";
|
||||
$prompt .= "1. QUALITY: Is the writing clear, engaging, and well-structured?\n";
|
||||
$prompt .= "2. ACCURACY: Does the content avoid making unsupported health/medical claims? Is the tone appropriately cautious for the topic?\n";
|
||||
$prompt .= "3. SEO: Are headings properly structured? Are target keywords naturally used?\n";
|
||||
$prompt .= "4. COMPLETENESS: Does it cover the key points? Does it have a proper intro and conclusion?\n";
|
||||
$prompt .= "5. READABILITY: Is sentence variety good? Are paragraphs appropriate length?\n\n";
|
||||
|
||||
$prompt .= "You must respond with JSON in this format:\n";
|
||||
$prompt .= "{\n";
|
||||
$prompt .= "Evaluate on: QUALITY, ACCURACY, SEO, COMPLETENESS, READABILITY.\n";
|
||||
$prompt .= "Respond with JSON: {\n";
|
||||
$prompt .= ' "verdict": "approved" | "revised" | "rejected",' . "\n";
|
||||
$prompt .= ' "quality_score": 0.0-1.0,' . "\n";
|
||||
$prompt .= ' "seo_score": 0.0-1.0,' . "\n";
|
||||
$prompt .= ' "issues": ["issue 1", "issue 2"],' . "\n";
|
||||
$prompt .= ' "notes": "Brief editorial notes",' . "\n";
|
||||
$prompt .= ' "revised_content": "Full revised HTML if verdict is revised, null otherwise"' . "\n";
|
||||
$prompt .= ' "issues": ["issue1", "issue2"],' . "\n";
|
||||
$prompt .= ' "notes": "Editorial notes",' . "\n";
|
||||
$prompt .= ' "revised_content": "Full revised HTML if revised, null otherwise"' . "\n";
|
||||
$prompt .= "}\n\n";
|
||||
|
||||
$prompt .= "Rules:\n";
|
||||
$prompt .= "- APPROVE if quality_score >= 0.7 and seo_score >= 0.6 and no critical issues\n";
|
||||
$prompt .= "- REVISE if fixable issues exist — you MUST provide the full revised HTML\n";
|
||||
$prompt .= "- REJECT only if the content is fundamentally flawed or off-topic\n";
|
||||
$prompt .= "- When revising, fix the issues yourself. Don't just describe them.\n";
|
||||
$prompt .= "- When revising, PRESERVE all bullet points, numbered lists, hyperlinks, and structural formatting from the original. Do NOT flatten lists into prose or remove links.\n";
|
||||
$prompt .= "- APPROVE if quality_score >= 0.7 and seo_score >= 0.6\n";
|
||||
$prompt .= "- REVISE if fixable issues exist — provide full revised HTML\n";
|
||||
$prompt .= "- REJECT if fundamentally flawed\n";
|
||||
$prompt .= "- Preserve formatting, links, lists when revising\n";
|
||||
|
||||
// Append user-defined editor instructions if configured.
|
||||
$instructions = trim( (string) get_option( 'prautoblogger_editor_instructions', '' ) );
|
||||
if ( '' !== $instructions ) {
|
||||
$prompt .= "\nAdditional instructions:\n" . $instructions . "\n";
|
||||
@@ -129,21 +125,15 @@ class PRAutoBlogger_Chief_Editor {
|
||||
}
|
||||
|
||||
/**
|
||||
* @param string $content
|
||||
* @param PRAutoBlogger_Article_Idea $idea
|
||||
*
|
||||
* @return string
|
||||
* Build user prompt for editorial review.
|
||||
*/
|
||||
private function build_review_prompt( string $content, PRAutoBlogger_Article_Idea $idea ): string {
|
||||
return sprintf(
|
||||
"Review this article draft:\n\n" .
|
||||
"ORIGINAL BRIEF:\n" .
|
||||
"- Title: %s\n" .
|
||||
"- Topic: %s\n" .
|
||||
"- Type: %s\n" .
|
||||
"- Key points: %s\n" .
|
||||
"- Target keywords: %s\n\n" .
|
||||
"ARTICLE CONTENT:\n%s",
|
||||
"BRIEF: Title: %s | Topic: %s | Type: %s\n" .
|
||||
"KEY POINTS: %s\n" .
|
||||
"TARGET KEYWORDS: %s\n\n" .
|
||||
"CONTENT:\n%s",
|
||||
$idea->get_suggested_title(),
|
||||
$idea->get_topic(),
|
||||
$idea->get_article_type(),
|
||||
@@ -154,25 +144,20 @@ class PRAutoBlogger_Chief_Editor {
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse the LLM's editorial review response.
|
||||
*
|
||||
* @param string $content JSON response from LLM.
|
||||
*
|
||||
* @return PRAutoBlogger_Editorial_Review
|
||||
* Parse LLM editorial review response.
|
||||
*/
|
||||
private function parse_review_response( string $content ): PRAutoBlogger_Editorial_Review {
|
||||
$data = PRAutoBlogger_Json_Extractor::decode( $content );
|
||||
|
||||
if ( ! is_array( $data ) || ! isset( $data['verdict'] ) ) {
|
||||
PRAutoBlogger_Logger::instance()->error(
|
||||
'Chief editor response was not valid JSON. Defaulting to rejected. Raw (first 500 chars): '
|
||||
. mb_substr( $content, 0, 500 ),
|
||||
'Chief editor response unparseable (first 200 chars): ' . mb_substr( $content, 0, 200 ),
|
||||
'editor'
|
||||
);
|
||||
return new PRAutoBlogger_Editorial_Review(
|
||||
array(
|
||||
'verdict' => 'rejected',
|
||||
'notes' => 'Editor response could not be parsed.',
|
||||
'notes' => 'Editor response unparseable',
|
||||
'quality_score' => 0.0,
|
||||
'seo_score' => 0.0,
|
||||
'issues' => array( 'Unparseable editor response' ),
|
||||
|
||||
@@ -2,21 +2,13 @@
|
||||
declare(strict_types=1);
|
||||
|
||||
/**
|
||||
* phpcs:ignore WordPress.Files.FileName.InvalidClassFileName -- class naming convention differs from WordPress standard
|
||||
*
|
||||
* Writer agent — generates blog post content from article ideas.
|
||||
*
|
||||
* Supports two configurable pipeline modes:
|
||||
* - single_pass: One LLM call produces the complete article (cheaper, faster).
|
||||
* - multi_step: Outline → Draft → Polish pipeline (higher quality, more cost).
|
||||
* Supports: single_pass (one LLM call) or multi_step (outline → draft → polish).
|
||||
* Opik instrumentation: spans per LLM call when feature-flag enabled.
|
||||
*
|
||||
* Triggered by: PRAutoBlogger::run_generation_pipeline() (step 4).
|
||||
* Triggered by: PRAutoBlogger_Article_Worker.
|
||||
* Dependencies: LLM_Provider_Interface, Cost_Tracker, Content_Prompts.
|
||||
*
|
||||
* @see core/class-content-prompts.php — Builds all LLM prompts used here.
|
||||
* @see core/class-chief-editor.php — Reviews output of this class.
|
||||
* @see providers/interface-llm-provider.php — LLM used for generation.
|
||||
* @see ARCHITECTURE.md — Data flow step 4.
|
||||
*/
|
||||
class PRAutoBlogger_Content_Generator {
|
||||
|
||||
@@ -34,12 +26,8 @@ class PRAutoBlogger_Content_Generator {
|
||||
/**
|
||||
* Generate a blog post from an article idea.
|
||||
*
|
||||
* Reads pipeline mode from settings and dispatches to the appropriate method.
|
||||
* Side effects: LLM API calls (1-3 depending on mode), cost logging.
|
||||
*
|
||||
* @param PRAutoBlogger_Article_Idea $idea The scored idea to generate content for.
|
||||
* @return string Generated HTML content.
|
||||
* @throws \RuntimeException On generation failure.
|
||||
*/
|
||||
public function generate( PRAutoBlogger_Article_Idea $idea ): string {
|
||||
$mode = get_option( 'prautoblogger_writing_pipeline', 'multi_step' );
|
||||
@@ -55,21 +43,26 @@ class PRAutoBlogger_Content_Generator {
|
||||
get_option( 'prautoblogger_writing_instructions', '' )
|
||||
);
|
||||
|
||||
if ( 'single_pass' === $mode ) {
|
||||
return $this->generate_single_pass( $request );
|
||||
}
|
||||
|
||||
return $this->generate_multi_step( $request );
|
||||
return 'single_pass' === $mode
|
||||
? $this->generate_single_pass( $request )
|
||||
: $this->generate_multi_step( $request );
|
||||
}
|
||||
|
||||
/**
|
||||
* Single-pass generation: one LLM call produces the complete article.
|
||||
*
|
||||
* @param PRAutoBlogger_Content_Request $request Content request with settings.
|
||||
* @return string Generated HTML content.
|
||||
* Single-pass: one LLM call produces complete article.
|
||||
*/
|
||||
private function generate_single_pass( PRAutoBlogger_Content_Request $request ): string {
|
||||
$model = get_option( 'prautoblogger_writing_model', PRAUTOBLOGGER_DEFAULT_WRITING_MODEL );
|
||||
$ctx = PRAutoBlogger_Opik_Trace_Context::current();
|
||||
|
||||
$span_id = $ctx->start_span(
|
||||
array(
|
||||
'name' => 'single_pass_generation',
|
||||
'type' => 'llm',
|
||||
'model' => $model,
|
||||
'provider' => 'openrouter',
|
||||
)
|
||||
);
|
||||
|
||||
$response = $this->llm->send_chat_completion(
|
||||
array(
|
||||
@@ -89,6 +82,17 @@ class PRAutoBlogger_Content_Generator {
|
||||
)
|
||||
);
|
||||
|
||||
$ctx->end_span(
|
||||
$span_id,
|
||||
array(
|
||||
'usage' => array(
|
||||
'prompt_tokens' => $response['prompt_tokens'],
|
||||
'completion_tokens' => $response['completion_tokens'],
|
||||
'total_tokens' => $response['prompt_tokens'] + $response['completion_tokens'],
|
||||
),
|
||||
)
|
||||
);
|
||||
|
||||
$this->cost_tracker->log_api_call(
|
||||
null,
|
||||
'draft',
|
||||
@@ -101,7 +105,9 @@ class PRAutoBlogger_Content_Generator {
|
||||
return $response['content'];
|
||||
}
|
||||
|
||||
/** Multi-step generation: Outline → Draft → Polish. */
|
||||
/**
|
||||
* Multi-step: outline → draft → polish.
|
||||
*/
|
||||
private function generate_multi_step( PRAutoBlogger_Content_Request $request ): string {
|
||||
$model = get_option( 'prautoblogger_writing_model', PRAUTOBLOGGER_DEFAULT_WRITING_MODEL );
|
||||
|
||||
@@ -112,15 +118,26 @@ class PRAutoBlogger_Content_Generator {
|
||||
return $polished;
|
||||
}
|
||||
|
||||
/** Multi-step stage 1: Generate an article outline. */
|
||||
/**
|
||||
* Stage 1: Generate article outline.
|
||||
*/
|
||||
private function stage_outline( PRAutoBlogger_Content_Request $request, string $model ): string {
|
||||
$system = PRAutoBlogger_Content_Prompts::build_system( $request );
|
||||
$ctx = PRAutoBlogger_Opik_Trace_Context::current();
|
||||
|
||||
$span_id = $ctx->start_span(
|
||||
array(
|
||||
'name' => 'outline_generation',
|
||||
'type' => 'llm',
|
||||
'model' => $model,
|
||||
'provider' => 'openrouter',
|
||||
)
|
||||
);
|
||||
|
||||
$response = $this->llm->send_chat_completion(
|
||||
array(
|
||||
array(
|
||||
'role' => 'system',
|
||||
'content' => $system,
|
||||
'content' => PRAutoBlogger_Content_Prompts::build_system( $request ),
|
||||
),
|
||||
array(
|
||||
'role' => 'user',
|
||||
@@ -134,6 +151,17 @@ class PRAutoBlogger_Content_Generator {
|
||||
)
|
||||
);
|
||||
|
||||
$ctx->end_span(
|
||||
$span_id,
|
||||
array(
|
||||
'usage' => array(
|
||||
'prompt_tokens' => $response['prompt_tokens'],
|
||||
'completion_tokens' => $response['completion_tokens'],
|
||||
'total_tokens' => $response['prompt_tokens'] + $response['completion_tokens'],
|
||||
),
|
||||
)
|
||||
);
|
||||
|
||||
$this->cost_tracker->log_api_call(
|
||||
null,
|
||||
'outline',
|
||||
@@ -146,15 +174,26 @@ class PRAutoBlogger_Content_Generator {
|
||||
return $response['content'];
|
||||
}
|
||||
|
||||
/** Multi-step stage 2: Write the full draft from the outline. */
|
||||
/**
|
||||
* Stage 2: Write full draft from outline.
|
||||
*/
|
||||
private function stage_draft( PRAutoBlogger_Content_Request $request, string $outline, string $model ): string {
|
||||
$system = PRAutoBlogger_Content_Prompts::build_system( $request );
|
||||
$ctx = PRAutoBlogger_Opik_Trace_Context::current();
|
||||
|
||||
$span_id = $ctx->start_span(
|
||||
array(
|
||||
'name' => 'draft_generation',
|
||||
'type' => 'llm',
|
||||
'model' => $model,
|
||||
'provider' => 'openrouter',
|
||||
)
|
||||
);
|
||||
|
||||
$response = $this->llm->send_chat_completion(
|
||||
array(
|
||||
array(
|
||||
'role' => 'system',
|
||||
'content' => $system,
|
||||
'content' => PRAutoBlogger_Content_Prompts::build_system( $request ),
|
||||
),
|
||||
array(
|
||||
'role' => 'user',
|
||||
@@ -168,6 +207,17 @@ class PRAutoBlogger_Content_Generator {
|
||||
)
|
||||
);
|
||||
|
||||
$ctx->end_span(
|
||||
$span_id,
|
||||
array(
|
||||
'usage' => array(
|
||||
'prompt_tokens' => $response['prompt_tokens'],
|
||||
'completion_tokens' => $response['completion_tokens'],
|
||||
'total_tokens' => $response['prompt_tokens'] + $response['completion_tokens'],
|
||||
),
|
||||
)
|
||||
);
|
||||
|
||||
$this->cost_tracker->log_api_call(
|
||||
null,
|
||||
'draft',
|
||||
@@ -180,15 +230,26 @@ class PRAutoBlogger_Content_Generator {
|
||||
return $response['content'];
|
||||
}
|
||||
|
||||
/** Multi-step stage 3: Polish and refine the draft. */
|
||||
/**
|
||||
* Stage 3: Polish and refine draft.
|
||||
*/
|
||||
private function stage_polish( PRAutoBlogger_Content_Request $request, string $draft, string $model ): string {
|
||||
$system = PRAutoBlogger_Content_Prompts::build_system( $request );
|
||||
$ctx = PRAutoBlogger_Opik_Trace_Context::current();
|
||||
|
||||
$span_id = $ctx->start_span(
|
||||
array(
|
||||
'name' => 'polish_generation',
|
||||
'type' => 'llm',
|
||||
'model' => $model,
|
||||
'provider' => 'openrouter',
|
||||
)
|
||||
);
|
||||
|
||||
$response = $this->llm->send_chat_completion(
|
||||
array(
|
||||
array(
|
||||
'role' => 'system',
|
||||
'content' => $system,
|
||||
'content' => PRAutoBlogger_Content_Prompts::build_system( $request ),
|
||||
),
|
||||
array(
|
||||
'role' => 'user',
|
||||
@@ -202,6 +263,17 @@ class PRAutoBlogger_Content_Generator {
|
||||
)
|
||||
);
|
||||
|
||||
$ctx->end_span(
|
||||
$span_id,
|
||||
array(
|
||||
'usage' => array(
|
||||
'prompt_tokens' => $response['prompt_tokens'],
|
||||
'completion_tokens' => $response['completion_tokens'],
|
||||
'total_tokens' => $response['prompt_tokens'] + $response['completion_tokens'],
|
||||
),
|
||||
)
|
||||
);
|
||||
|
||||
$this->cost_tracker->log_api_call(
|
||||
null,
|
||||
'polish',
|
||||
|
||||
276
includes/services/opik/class-opik-client.php
Normal file
276
includes/services/opik/class-opik-client.php
Normal file
@@ -0,0 +1,276 @@
|
||||
<?php
|
||||
declare(strict_types=1);
|
||||
|
||||
/**
|
||||
* Opik REST API HTTP client.
|
||||
*
|
||||
* Minimal PHP HTTP client for posting traces and spans to Comet Opik.
|
||||
* Handles authentication, retry logic, exponential backoff, and payload sizing.
|
||||
*
|
||||
* Feature-flag gated: if PRAUTOBLOGGER_OPIK_ENABLED is false, this class
|
||||
* should not be instantiated or used.
|
||||
*
|
||||
* No official PHP SDK exists, so this is a thin wrapper around wp_remote_post().
|
||||
*
|
||||
* @see https://www.comet.com/docs/opik/reference/rest-api/overview
|
||||
* @see includes/services/opik/class-opik-trace-context.php
|
||||
* @see includes/services/opik/class-opik-dispatcher.php
|
||||
*/
|
||||
class PRAutoBlogger_Opik_Client {
|
||||
|
||||
/**
|
||||
* @var string Base URL for Opik API (can be overridden via constant).
|
||||
*/
|
||||
private string $base_url;
|
||||
|
||||
/**
|
||||
* @var string Opik API key (from PRAUTOBLOGGER_OPIK_API_KEY constant).
|
||||
*/
|
||||
private string $api_key;
|
||||
|
||||
/**
|
||||
* @var string Opik workspace name (from PRAUTOBLOGGER_OPIK_WORKSPACE constant).
|
||||
*/
|
||||
private string $workspace;
|
||||
|
||||
/**
|
||||
* @var int HTTP timeout in seconds.
|
||||
*/
|
||||
private int $timeout;
|
||||
|
||||
/**
|
||||
* Constructor.
|
||||
*
|
||||
* @param string $api_key Opik API key. Use empty string if disabled.
|
||||
* @param string $workspace Opik workspace (e.g. 'peptiderepo'). Use empty if disabled.
|
||||
* @param string $base_url Base URL for Opik API. Defaults to https://www.comet.com/opik/api/
|
||||
* @param int $timeout HTTP timeout in seconds. Defaults to 30.
|
||||
*/
|
||||
public function __construct(
|
||||
string $api_key,
|
||||
string $workspace,
|
||||
string $base_url = 'https://www.comet.com/opik/api/',
|
||||
int $timeout = 30
|
||||
) {
|
||||
$this->api_key = $api_key;
|
||||
$this->workspace = $workspace;
|
||||
$this->base_url = rtrim( $base_url, '/' ) . '/';
|
||||
$this->timeout = $timeout;
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a trace (top-level span container for one article generation).
|
||||
*
|
||||
* @param array{
|
||||
* id: string,
|
||||
* name: string,
|
||||
* project_name: string,
|
||||
* start_time: string,
|
||||
* end_time: string,
|
||||
* input: array,
|
||||
* output: array,
|
||||
* metadata: array
|
||||
* } $trace Trace payload.
|
||||
*
|
||||
* @return array|null Response from Opik, or null on persistent failure.
|
||||
*/
|
||||
public function create_trace( array $trace ): ?array {
|
||||
$endpoint = 'v1/private/traces';
|
||||
return $this->post( $endpoint, $trace );
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a single span (LLM call record).
|
||||
*
|
||||
* @param array{
|
||||
* id: string,
|
||||
* trace_id: string,
|
||||
* parent_span_id?: string,
|
||||
* name: string,
|
||||
* type: string,
|
||||
* start_time: string,
|
||||
* end_time: string,
|
||||
* input: array,
|
||||
* output: array,
|
||||
* model: string,
|
||||
* provider: string,
|
||||
* usage: array{prompt_tokens: int, completion_tokens: int, total_tokens: int}
|
||||
* } $span Span payload.
|
||||
*
|
||||
* @return array|null Response from Opik, or null on persistent failure.
|
||||
*/
|
||||
public function create_span( array $span ): ?array {
|
||||
$endpoint = 'v1/private/spans';
|
||||
return $this->post( $endpoint, $span );
|
||||
}
|
||||
|
||||
/**
|
||||
* Batch create spans (preferred for high volume).
|
||||
*
|
||||
* @param array $spans Array of span payloads (max 100 per batch).
|
||||
*
|
||||
* @return array|null Response from Opik, or null on persistent failure.
|
||||
*/
|
||||
public function batch_create_spans( array $spans ): ?array {
|
||||
// Enforce batch size limit.
|
||||
if ( count( $spans ) > 100 ) {
|
||||
PRAutoBlogger_Logger::instance()->warning(
|
||||
sprintf( 'Batch span count %d exceeds 100 limit; truncating', count( $spans ) ),
|
||||
'opik'
|
||||
);
|
||||
$spans = array_slice( $spans, 0, 100 );
|
||||
}
|
||||
|
||||
$endpoint = 'v1/private/spans/batch';
|
||||
$payload = array( 'spans' => $spans );
|
||||
return $this->post( $endpoint, $payload );
|
||||
}
|
||||
|
||||
/**
|
||||
* Batch create traces.
|
||||
*
|
||||
* @param array $traces Array of trace payloads (max 100 per batch).
|
||||
*
|
||||
* @return array|null Response from Opik, or null on persistent failure.
|
||||
*/
|
||||
public function batch_create_traces( array $traces ): ?array {
|
||||
// Enforce batch size limit.
|
||||
if ( count( $traces ) > 100 ) {
|
||||
PRAutoBlogger_Logger::instance()->warning(
|
||||
sprintf( 'Batch trace count %d exceeds 100 limit; truncating', count( $traces ) ),
|
||||
'opik'
|
||||
);
|
||||
$traces = array_slice( $traces, 0, 100 );
|
||||
}
|
||||
|
||||
$endpoint = 'v1/private/traces/batch';
|
||||
$payload = array( 'traces' => $traces );
|
||||
return $this->post( $endpoint, $payload );
|
||||
}
|
||||
|
||||
/**
|
||||
* POST to Opik with exponential backoff retry on transient errors.
|
||||
*
|
||||
* Retries on: HTTP 5xx, timeout, network error.
|
||||
* Does not retry on: 4xx (permanent errors).
|
||||
*
|
||||
* @param string $endpoint Relative endpoint path (e.g. 'v1/private/traces').
|
||||
* @param array $payload JSON-serializable payload.
|
||||
* @param int $attempt Internal retry counter (start at 0).
|
||||
*
|
||||
* @return array|null Decoded JSON response, or null on final failure.
|
||||
*/
|
||||
private function post( string $endpoint, array $payload, int $attempt = 0 ): ?array {
|
||||
$max_attempts = 3;
|
||||
|
||||
$url = $this->base_url . $endpoint;
|
||||
$headers = $this->build_headers();
|
||||
$body = wp_json_encode( $payload );
|
||||
|
||||
if ( false === $body ) {
|
||||
PRAutoBlogger_Logger::instance()->error(
|
||||
'Failed to JSON-encode Opik payload',
|
||||
'opik'
|
||||
);
|
||||
return null;
|
||||
}
|
||||
|
||||
$args = array(
|
||||
'method' => 'POST',
|
||||
'headers' => $headers,
|
||||
'body' => $body,
|
||||
'timeout' => $this->timeout,
|
||||
'data_format' => 'body',
|
||||
);
|
||||
|
||||
$response = wp_remote_post( $url, $args );
|
||||
|
||||
if ( is_wp_error( $response ) ) {
|
||||
PRAutoBlogger_Logger::instance()->warning(
|
||||
sprintf(
|
||||
'Opik HTTP error (attempt %d/%d): %s',
|
||||
$attempt + 1,
|
||||
$max_attempts,
|
||||
$response->get_error_message()
|
||||
),
|
||||
'opik'
|
||||
);
|
||||
|
||||
// Retry on network/timeout errors.
|
||||
if ( $attempt < $max_attempts - 1 ) {
|
||||
$delay = (int) ( 2 ** ( $attempt + 1 ) ); // 2s, 4s, 8s
|
||||
sleep( $delay );
|
||||
return $this->post( $endpoint, $payload, $attempt + 1 );
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
$code = (int) wp_remote_retrieve_response_code( $response );
|
||||
|
||||
// Success.
|
||||
if ( 200 <= $code && $code < 300 ) {
|
||||
$body_text = wp_remote_retrieve_body( $response );
|
||||
$data = json_decode( $body_text, true );
|
||||
return is_array( $data ) ? $data : array();
|
||||
}
|
||||
|
||||
// Permanent client error.
|
||||
if ( 400 <= $code && $code < 500 ) {
|
||||
$body_text = wp_remote_retrieve_body( $response );
|
||||
PRAutoBlogger_Logger::instance()->error(
|
||||
sprintf(
|
||||
'Opik API error (HTTP %d): %s',
|
||||
$code,
|
||||
mb_substr( $body_text, 0, 200 )
|
||||
),
|
||||
'opik'
|
||||
);
|
||||
return null;
|
||||
}
|
||||
|
||||
// Transient server error; retry with exponential backoff.
|
||||
if ( 500 <= $code && $code < 600 ) {
|
||||
PRAutoBlogger_Logger::instance()->warning(
|
||||
sprintf(
|
||||
'Opik server error (HTTP %d, attempt %d/%d)',
|
||||
$code,
|
||||
$attempt + 1,
|
||||
$max_attempts
|
||||
),
|
||||
'opik'
|
||||
);
|
||||
|
||||
if ( $attempt < $max_attempts - 1 ) {
|
||||
$delay = (int) ( 2 ** ( $attempt + 1 ) );
|
||||
sleep( $delay );
|
||||
return $this->post( $endpoint, $payload, $attempt + 1 );
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
// Unexpected status.
|
||||
PRAutoBlogger_Logger::instance()->warning(
|
||||
sprintf( 'Opik unexpected HTTP %d', $code ),
|
||||
'opik'
|
||||
);
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build HTTP headers for Opik API requests.
|
||||
*
|
||||
* Auth: Authorization: <api_key> (raw, not Bearer)
|
||||
* Workspace: Comet-Workspace: peptiderepo
|
||||
*
|
||||
* @return array<string, string>
|
||||
*/
|
||||
private function build_headers(): array {
|
||||
return array(
|
||||
'Content-Type' => 'application/json',
|
||||
'Authorization' => $this->api_key,
|
||||
'Comet-Workspace' => $this->workspace,
|
||||
);
|
||||
}
|
||||
}
|
||||
184
includes/services/opik/class-opik-dispatcher.php
Normal file
184
includes/services/opik/class-opik-dispatcher.php
Normal file
@@ -0,0 +1,184 @@
|
||||
<?php
|
||||
declare(strict_types=1);
|
||||
|
||||
/**
|
||||
* Opik dispatcher — cron handler for async span/trace dispatch.
|
||||
*
|
||||
* Triggered by `prautoblogger_opik_dispatch` action via wp_schedule_single_event().
|
||||
* Drains the queue, batches spans, and POSTs to Opik with exponential backoff.
|
||||
*
|
||||
* Handles retries per-batch (not per-item). Failed batches are re-enqueued
|
||||
* for the next cron tick.
|
||||
*
|
||||
* @see includes/services/opik/class-opik-client.php
|
||||
* @see includes/services/opik/class-opik-span-queue.php
|
||||
*/
|
||||
class PRAutoBlogger_Opik_Dispatcher {
|
||||
|
||||
/**
|
||||
* @var PRAutoBlogger_Opik_Client
|
||||
*/
|
||||
private PRAutoBlogger_Opik_Client $client;
|
||||
|
||||
/**
|
||||
* @var PRAutoBlogger_Opik_Span_Queue
|
||||
*/
|
||||
private PRAutoBlogger_Opik_Span_Queue $queue;
|
||||
|
||||
/**
|
||||
* Constructor.
|
||||
*
|
||||
* @param PRAutoBlogger_Opik_Client $client Client instance.
|
||||
* @param PRAutoBlogger_Opik_Span_Queue $queue Queue instance.
|
||||
*/
|
||||
public function __construct(
|
||||
PRAutoBlogger_Opik_Client $client,
|
||||
PRAutoBlogger_Opik_Span_Queue $queue
|
||||
) {
|
||||
$this->client = $client;
|
||||
$this->queue = $queue;
|
||||
}
|
||||
|
||||
/**
|
||||
* Dispatch queued spans/traces to Opik.
|
||||
*
|
||||
* Called via wp_schedule_single_event(). Drains the queue in batches
|
||||
* and POSTs to Opik. Failed batches are re-enqueued.
|
||||
*
|
||||
* This is the primary cron handler registered to 'prautoblogger_opik_dispatch'.
|
||||
*
|
||||
* @return void
|
||||
*/
|
||||
public function dispatch(): void {
|
||||
if ( ! $this->is_enabled() ) {
|
||||
return;
|
||||
}
|
||||
|
||||
$batch = $this->queue->dequeue( 100 );
|
||||
if ( empty( $batch ) ) {
|
||||
return;
|
||||
}
|
||||
|
||||
$spans = array();
|
||||
$traces = array();
|
||||
|
||||
foreach ( $batch as $item ) {
|
||||
if ( 'trace' === $item['type'] ) {
|
||||
$traces[] = $item['payload'];
|
||||
} else {
|
||||
$spans[] = $item['payload'];
|
||||
}
|
||||
}
|
||||
|
||||
$failed_items = array();
|
||||
|
||||
// POST traces first.
|
||||
if ( ! empty( $traces ) ) {
|
||||
$success = $this->post_traces( $traces );
|
||||
if ( ! $success ) {
|
||||
$failed_items = array_merge(
|
||||
$failed_items,
|
||||
array_filter(
|
||||
$batch,
|
||||
function ( $item ) {
|
||||
return 'trace' === $item['type'];
|
||||
}
|
||||
)
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Then POST spans.
|
||||
if ( ! empty( $spans ) ) {
|
||||
$success = $this->post_spans( $spans );
|
||||
if ( ! $success ) {
|
||||
$failed_items = array_merge(
|
||||
$failed_items,
|
||||
array_filter(
|
||||
$batch,
|
||||
function ( $item ) {
|
||||
return 'span' === $item['type'];
|
||||
}
|
||||
)
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Re-enqueue failed items with backoff.
|
||||
if ( ! empty( $failed_items ) ) {
|
||||
$this->queue->reenqueue( $failed_items );
|
||||
}
|
||||
|
||||
// Log summary.
|
||||
PRAutoBlogger_Logger::instance()->info(
|
||||
sprintf(
|
||||
'Opik dispatch: %d spans, %d traces, %d failed; queue depth now %d',
|
||||
count( $spans ),
|
||||
count( $traces ),
|
||||
count( $failed_items ),
|
||||
$this->queue->get_queue_depth()
|
||||
),
|
||||
'opik'
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* POST traces to Opik.
|
||||
*
|
||||
* @param array $traces Array of trace payloads.
|
||||
*
|
||||
* @return bool True on success, false on failure.
|
||||
*/
|
||||
private function post_traces( array $traces ): bool {
|
||||
if ( count( $traces ) === 1 ) {
|
||||
// Single trace: use create_trace.
|
||||
$response = $this->client->create_trace( $traces[0] );
|
||||
return null !== $response;
|
||||
}
|
||||
|
||||
// Multiple traces: use batch.
|
||||
$response = $this->client->batch_create_traces( $traces );
|
||||
return null !== $response;
|
||||
}
|
||||
|
||||
/**
|
||||
* POST spans to Opik.
|
||||
*
|
||||
* @param array $spans Array of span payloads.
|
||||
*
|
||||
* @return bool True on success, false on failure.
|
||||
*/
|
||||
private function post_spans( array $spans ): bool {
|
||||
if ( count( $spans ) === 1 ) {
|
||||
// Single span: use create_span.
|
||||
$response = $this->client->create_span( $spans[0] );
|
||||
return null !== $response;
|
||||
}
|
||||
|
||||
// Multiple spans: use batch.
|
||||
$response = $this->client->batch_create_spans( $spans );
|
||||
return null !== $response;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if Opik is enabled via feature flag and has credentials.
|
||||
*
|
||||
* @return bool
|
||||
*/
|
||||
private function is_enabled(): bool {
|
||||
if ( ! get_option( 'prautoblogger_opik_enabled', false ) ) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// Credentials must be defined in wp-config.php.
|
||||
if ( ! defined( 'PRAUTOBLOGGER_OPIK_API_KEY' ) ||
|
||||
! defined( 'PRAUTOBLOGGER_OPIK_WORKSPACE' ) ) {
|
||||
return false;
|
||||
}
|
||||
|
||||
$api_key = PRAUTOBLOGGER_OPIK_API_KEY;
|
||||
$workspace = PRAUTOBLOGGER_OPIK_WORKSPACE;
|
||||
|
||||
return ! empty( $api_key ) && ! empty( $workspace );
|
||||
}
|
||||
}
|
||||
170
includes/services/opik/class-opik-span-queue.php
Normal file
170
includes/services/opik/class-opik-span-queue.php
Normal file
@@ -0,0 +1,170 @@
|
||||
<?php
|
||||
declare(strict_types=1);
|
||||
|
||||
/**
|
||||
* Async queue for Opik span/trace payloads.
|
||||
*
|
||||
* Backs the queue with WP options (a single serialized array stored
|
||||
* in wp_options). Payloads are enqueued at the end of article
|
||||
* generation, dequeued and batch-posted via cron.
|
||||
*
|
||||
* Max queue depth is 1000 items; older items are expired TTL-based.
|
||||
*
|
||||
* @see includes/services/opik/class-opik-dispatcher.php — Drains the queue.
|
||||
*/
|
||||
class PRAutoBlogger_Opik_Span_Queue {
|
||||
|
||||
/**
|
||||
* @var string WP option key storing the queue.
|
||||
*/
|
||||
const QUEUE_OPTION_KEY = 'prautoblogger_opik_queue';
|
||||
|
||||
/**
|
||||
* @var int Max queue depth (items).
|
||||
*/
|
||||
const MAX_QUEUE_DEPTH = 1000;
|
||||
|
||||
/**
|
||||
* @var int Item TTL in seconds (12 hours).
|
||||
*/
|
||||
const ITEM_TTL_SECONDS = 43200;
|
||||
|
||||
/**
|
||||
* Enqueue a span or trace for async dispatch.
|
||||
*
|
||||
* @param array $payload Span or trace payload.
|
||||
* @param string $type Either 'span' or 'trace'.
|
||||
*
|
||||
* @return bool True if enqueued, false if queue full.
|
||||
*/
|
||||
public function enqueue( array $payload, string $type = 'span' ): bool {
|
||||
if ( ! in_array( $type, array( 'span', 'trace' ), true ) ) {
|
||||
$type = 'span';
|
||||
}
|
||||
|
||||
$queue = $this->get_queue();
|
||||
|
||||
if ( count( $queue ) >= self::MAX_QUEUE_DEPTH ) {
|
||||
PRAutoBlogger_Logger::instance()->warning(
|
||||
sprintf( 'Opik queue full (%d items); dropping span', count( $queue ) ),
|
||||
'opik'
|
||||
);
|
||||
return false;
|
||||
}
|
||||
|
||||
$item = array(
|
||||
'type' => $type,
|
||||
'payload' => $payload,
|
||||
'enqueued' => time(),
|
||||
'attempt' => 0,
|
||||
);
|
||||
|
||||
$queue[] = $item;
|
||||
update_option( self::QUEUE_OPTION_KEY, $queue );
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Dequeue the next N items from the queue for processing.
|
||||
*
|
||||
* @param int $limit Max items to dequeue.
|
||||
*
|
||||
* @return array Array of items with 'type', 'payload', 'enqueued', 'attempt'.
|
||||
*/
|
||||
public function dequeue( int $limit = 100 ): array {
|
||||
$queue = $this->get_queue();
|
||||
|
||||
if ( empty( $queue ) ) {
|
||||
return array();
|
||||
}
|
||||
|
||||
// Remove expired items.
|
||||
$now = time();
|
||||
$queue = array_filter(
|
||||
$queue,
|
||||
function ( $item ) use ( $now ) {
|
||||
return ( $now - $item['enqueued'] ) < self::ITEM_TTL_SECONDS;
|
||||
}
|
||||
);
|
||||
|
||||
// Grab the first $limit items.
|
||||
$batch = array_splice( $queue, 0, $limit );
|
||||
|
||||
// Save the remaining queue.
|
||||
update_option( self::QUEUE_OPTION_KEY, $queue );
|
||||
|
||||
return $batch;
|
||||
}
|
||||
|
||||
/**
|
||||
* Re-enqueue items that failed to post (typically after backoff).
|
||||
*
|
||||
* @param array $items Items from a failed batch.
|
||||
*/
|
||||
public function reenqueue( array $items ): void {
|
||||
$queue = $this->get_queue();
|
||||
|
||||
foreach ( $items as $item ) {
|
||||
$item['attempt'] = ( $item['attempt'] ?? 0 ) + 1;
|
||||
|
||||
// Drop items that have exceeded max retries (3).
|
||||
if ( $item['attempt'] >= 3 ) {
|
||||
PRAutoBlogger_Logger::instance()->warning(
|
||||
sprintf(
|
||||
'Dropping Opik item after %d attempts: %s',
|
||||
$item['attempt'],
|
||||
$item['type']
|
||||
),
|
||||
'opik'
|
||||
);
|
||||
continue;
|
||||
}
|
||||
|
||||
$queue[] = $item;
|
||||
}
|
||||
|
||||
update_option( self::QUEUE_OPTION_KEY, $queue );
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the current queue state.
|
||||
*
|
||||
* @return array
|
||||
*/
|
||||
public function get_queue(): array {
|
||||
$queue = get_option( self::QUEUE_OPTION_KEY, array() );
|
||||
return is_array( $queue ) ? $queue : array();
|
||||
}
|
||||
|
||||
/**
|
||||
* Get queue depth (item count).
|
||||
*
|
||||
* @return int
|
||||
*/
|
||||
public function get_queue_depth(): int {
|
||||
return count( $this->get_queue() );
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear all items from the queue.
|
||||
*
|
||||
* @return void
|
||||
*/
|
||||
public function clear(): void {
|
||||
delete_option( self::QUEUE_OPTION_KEY );
|
||||
}
|
||||
|
||||
/**
|
||||
* Get a human-readable status string for admin display.
|
||||
*
|
||||
* @return string
|
||||
*/
|
||||
public function get_status(): string {
|
||||
$depth = $this->get_queue_depth();
|
||||
if ( 0 === $depth ) {
|
||||
return 'Queue empty';
|
||||
}
|
||||
return sprintf( 'Queue: %d item%s pending', $depth, 1 !== $depth ? 's' : '' );
|
||||
}
|
||||
}
|
||||
196
includes/services/opik/class-opik-trace-context.php
Normal file
196
includes/services/opik/class-opik-trace-context.php
Normal file
@@ -0,0 +1,196 @@
|
||||
<?php
|
||||
declare(strict_types=1);
|
||||
|
||||
/**
|
||||
* Per-request trace context singleton.
|
||||
*
|
||||
* Holds the current trace ID and manages the span stack for a single
|
||||
* article generation. Initialized at the top of the pipeline,
|
||||
* torn down at the end.
|
||||
*
|
||||
* Single-threaded per-request model: one trace per article generation,
|
||||
* child spans per LLM call.
|
||||
*
|
||||
* @see includes/services/opik/class-opik-client.php
|
||||
* @see includes/services/opik/class-opik-span-queue.php
|
||||
*/
|
||||
class PRAutoBlogger_Opik_Trace_Context {
|
||||
|
||||
/**
|
||||
* @var self|null
|
||||
*/
|
||||
private static ?self $instance = null;
|
||||
|
||||
/**
|
||||
* @var string Trace ID (UUID).
|
||||
*/
|
||||
private string $trace_id = '';
|
||||
|
||||
/**
|
||||
* @var float Trace start time (microtime).
|
||||
*/
|
||||
private float $trace_start = 0.0;
|
||||
|
||||
/**
|
||||
* @var array Stack of active spans; indexed by span_id.
|
||||
*/
|
||||
private array $span_stack = array();
|
||||
|
||||
/**
|
||||
* Get or create the current request's trace context.
|
||||
*
|
||||
* @return self
|
||||
*/
|
||||
public static function current(): self {
|
||||
if ( null === self::$instance ) {
|
||||
self::$instance = new self();
|
||||
}
|
||||
return self::$instance;
|
||||
}
|
||||
|
||||
/**
|
||||
* Initialize a new trace for this request.
|
||||
*
|
||||
* Generates a UUID and records the start time. Called once at the
|
||||
* top of the article generation pipeline.
|
||||
*
|
||||
* @return string The trace ID (UUID).
|
||||
*/
|
||||
public function init_trace(): string {
|
||||
$this->trace_id = wp_generate_uuid4();
|
||||
$this->trace_start = microtime( true );
|
||||
return $this->trace_id;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the current trace ID.
|
||||
*
|
||||
* @return string
|
||||
*/
|
||||
public function get_trace_id(): string {
|
||||
return $this->trace_id;
|
||||
}
|
||||
|
||||
/**
|
||||
* Start a span within the current trace.
|
||||
*
|
||||
* @param array{
|
||||
* name: string,
|
||||
* type: string,
|
||||
* model?: string,
|
||||
* provider?: string,
|
||||
* input?: array,
|
||||
* parent_span_id?: string
|
||||
* } $span_data Span metadata.
|
||||
*
|
||||
* @return string Generated span ID (UUID).
|
||||
*/
|
||||
public function start_span( array $span_data ): string {
|
||||
$span_id = wp_generate_uuid4();
|
||||
|
||||
$span = array(
|
||||
'id' => $span_id,
|
||||
'trace_id' => $this->trace_id,
|
||||
'name' => $span_data['name'] ?? 'unknown',
|
||||
'type' => $span_data['type'] ?? 'general',
|
||||
'start_time' => gmdate( 'Y-m-d\TH:i:s\Z' ),
|
||||
'start_time_ms' => microtime( true ),
|
||||
'end_time' => null,
|
||||
'end_time_ms' => null,
|
||||
'input' => $span_data['input'] ?? array(),
|
||||
'output' => array(),
|
||||
'model' => $span_data['model'] ?? null,
|
||||
'provider' => $span_data['provider'] ?? null,
|
||||
'parent_span_id' => $span_data['parent_span_id'] ?? null,
|
||||
'usage' => array(),
|
||||
);
|
||||
|
||||
$this->span_stack[ $span_id ] = $span;
|
||||
return $span_id;
|
||||
}
|
||||
|
||||
/**
|
||||
* End a span and record its output.
|
||||
*
|
||||
* @param string $span_id Span ID from start_span().
|
||||
* @param array{
|
||||
* output?: array,
|
||||
* usage?: array
|
||||
* } $span_data Final span data to merge.
|
||||
*/
|
||||
public function end_span( string $span_id, array $span_data = array() ): void {
|
||||
if ( ! isset( $this->span_stack[ $span_id ] ) ) {
|
||||
PRAutoBlogger_Logger::instance()->warning(
|
||||
sprintf( 'Attempt to end unknown span %s', $span_id ),
|
||||
'opik'
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
$span = &$this->span_stack[ $span_id ];
|
||||
$span['end_time'] = gmdate( 'Y-m-d\TH:i:s\Z' );
|
||||
$span['end_time_ms'] = microtime( true );
|
||||
$span['output'] = $span_data['output'] ?? array();
|
||||
$span['usage'] = $span_data['usage'] ?? array();
|
||||
}
|
||||
|
||||
/**
|
||||
* Get a completed span from the stack.
|
||||
*
|
||||
* @param string $span_id
|
||||
*
|
||||
* @return array|null
|
||||
*/
|
||||
public function get_span( string $span_id ): ?array {
|
||||
return $this->span_stack[ $span_id ] ?? null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all spans collected in this trace.
|
||||
*
|
||||
* @return array
|
||||
*/
|
||||
public function get_spans(): array {
|
||||
return $this->span_stack;
|
||||
}
|
||||
|
||||
/**
|
||||
* Finalize and return the trace object with all spans.
|
||||
*
|
||||
* @return array{
|
||||
* id: string,
|
||||
* name: string,
|
||||
* project_name: string,
|
||||
* start_time: string,
|
||||
* end_time: string,
|
||||
* input: array,
|
||||
* output: array,
|
||||
* metadata: array,
|
||||
* spans: array
|
||||
* }
|
||||
*/
|
||||
public function finalize_trace(): array {
|
||||
$trace_end = microtime( true );
|
||||
|
||||
return array(
|
||||
'id' => $this->trace_id,
|
||||
'name' => 'article_generation',
|
||||
'project_name' => get_option( 'prautoblogger_opik_project_name', 'prautoblogger' ),
|
||||
'start_time' => gmdate( 'Y-m-d\TH:i:s\Z', (int) $this->trace_start ),
|
||||
'end_time' => gmdate( 'Y-m-d\TH:i:s\Z', (int) $trace_end ),
|
||||
'input' => array( 'pipeline' => 'article_generation' ),
|
||||
'output' => array( 'spans_count' => count( $this->span_stack ) ),
|
||||
'metadata' => array( 'source' => 'prautoblogger' ),
|
||||
'spans' => $this->span_stack,
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Tear down the context (clear it for the next request).
|
||||
*
|
||||
* Called at the end of the article generation pipeline.
|
||||
*/
|
||||
public static function teardown(): void {
|
||||
self::$instance = null;
|
||||
}
|
||||
}
|
||||
@@ -9,7 +9,7 @@
|
||||
* Plugin Name: PRAutoBlogger
|
||||
* Plugin URI: https://peptiderepo.com/prautoblogger
|
||||
* Description: Monitors social media for trending topics, generates SEO-friendly blog posts using AI, and publishes them on a daily schedule with full cost tracking and self-improvement metrics.
|
||||
* Version: 0.11.0
|
||||
* Version: 0.12.0
|
||||
* Requires at least: 6.0
|
||||
* Requires PHP: 7.4
|
||||
* Author: PeptideRepo
|
||||
@@ -35,7 +35,7 @@ if ( ! defined( 'ABSPATH' ) ) {
|
||||
| Defined here so every file in the plugin can reference paths, versions,
|
||||
| and limits without magic strings.
|
||||
*/
|
||||
define( 'PRAUTOBLOGGER_VERSION', '0.11.0' );
|
||||
define( 'PRAUTOBLOGGER_VERSION', '0.12.0' );
|
||||
define( 'PRAUTOBLOGGER_DB_VERSION', '1.1.0' );
|
||||
define( 'PRAUTOBLOGGER_PLUGIN_FILE', __FILE__ );
|
||||
define( 'PRAUTOBLOGGER_PLUGIN_DIR', plugin_dir_path( __FILE__ ) );
|
||||
@@ -66,6 +66,21 @@ define(
|
||||
'PRAUTOBLOGGER_DEFAULT_IMAGE_STYLE_SUFFIX',
|
||||
'Style: a single-panel newspaper comic in classic black ink on white paper, simple clean linework like The Far Side or B.C., slightly yellowed newsprint texture, hand-drawn crosshatching for shading, bold panel border, warm and humorous tone, visible in a newspaper comics section. IMPORTANT: Do not render any text, captions, speech bubbles, or words inside the image.'
|
||||
);
|
||||
// Opik observability (feature-flag gated).
|
||||
// API credentials must be defined in wp-config.php constants:
|
||||
// - PRAUTOBLOGGER_OPIK_API_KEY: API key for authentication
|
||||
// - PRAUTOBLOGGER_OPIK_WORKSPACE: Opik workspace name (e.g. 'peptiderepo')
|
||||
// - PRAUTOBLOGGER_OPIK_URL_OVERRIDE: Optional; defaults to https://www.comet.com/opik/api/
|
||||
if ( ! defined( 'PRAUTOBLOGGER_OPIK_API_KEY' ) ) {
|
||||
define( 'PRAUTOBLOGGER_OPIK_API_KEY', '' );
|
||||
}
|
||||
if ( ! defined( 'PRAUTOBLOGGER_OPIK_WORKSPACE' ) ) {
|
||||
define( 'PRAUTOBLOGGER_OPIK_WORKSPACE', 'peptiderepo' );
|
||||
}
|
||||
if ( ! defined( 'PRAUTOBLOGGER_OPIK_URL_OVERRIDE' ) ) {
|
||||
define( 'PRAUTOBLOGGER_OPIK_URL_OVERRIDE', 'https://www.comet.com/opik/api/' );
|
||||
}
|
||||
|
||||
/*
|
||||
|--------------------------------------------------------------------------
|
||||
| Autoloader
|
||||
|
||||
281
tests/unit/Services/test-opik-client.php
Normal file
281
tests/unit/Services/test-opik-client.php
Normal file
@@ -0,0 +1,281 @@
|
||||
<?php
|
||||
declare(strict_types=1);
|
||||
|
||||
/**
|
||||
* Unit tests for PRAutoBlogger_Opik_Client.
|
||||
*
|
||||
* @see includes/services/opik/class-opik-client.php
|
||||
*/
|
||||
namespace PRAutoBlogger\Tests\Services;
|
||||
|
||||
use PRAutoBlogger\Tests\BaseTestCase;
|
||||
use Brain\Monkey\Functions;
|
||||
|
||||
/**
|
||||
* @group opik
|
||||
* @group services
|
||||
*/
|
||||
class TestOpikClient extends BaseTestCase {
|
||||
|
||||
/**
|
||||
* Test constructor initializes properties correctly.
|
||||
*/
|
||||
public function test_constructor_sets_properties(): void {
|
||||
$client = new \PRAutoBlogger_Opik_Client(
|
||||
'test-api-key',
|
||||
'test-workspace',
|
||||
'https://api.example.com/opik/',
|
||||
45
|
||||
);
|
||||
|
||||
$this->assertInstanceOf( \PRAutoBlogger_Opik_Client::class, $client );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test create_trace sends correct payload to endpoint.
|
||||
*/
|
||||
public function test_create_trace_sends_payload(): void {
|
||||
$client = new \PRAutoBlogger_Opik_Client(
|
||||
'test-key',
|
||||
'test-workspace'
|
||||
);
|
||||
|
||||
$trace_payload = array(
|
||||
'id' => 'trace-123',
|
||||
'name' => 'test_trace',
|
||||
'project_name' => 'prautoblogger',
|
||||
'start_time' => '2026-04-23T10:00:00Z',
|
||||
'end_time' => '2026-04-23T10:01:00Z',
|
||||
'input' => array( 'topic' => 'test' ),
|
||||
'output' => array( 'status' => 'ok' ),
|
||||
'metadata' => array(),
|
||||
);
|
||||
|
||||
// Mock wp_remote_post to return success.
|
||||
Functions\when( 'wp_remote_post' )->justReturn(
|
||||
array(
|
||||
'response' => array( 'code' => 200 ),
|
||||
'body' => wp_json_encode( array( 'id' => 'trace-123' ) ),
|
||||
)
|
||||
);
|
||||
|
||||
Functions\when( 'wp_remote_retrieve_response_code' )->justReturn( 200 );
|
||||
Functions\when( 'wp_remote_retrieve_body' )->justReturn( wp_json_encode( array( 'id' => 'trace-123' ) ) );
|
||||
Functions\when( 'wp_json_encode' )->alias( 'json_encode' );
|
||||
Functions\when( 'is_wp_error' )->justReturn( false );
|
||||
|
||||
$result = $client->create_trace( $trace_payload );
|
||||
|
||||
$this->assertIsArray( $result );
|
||||
$this->assertArrayHasKey( 'id', $result );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test create_span sends span with correct structure.
|
||||
*/
|
||||
public function test_create_span_sends_payload(): void {
|
||||
$client = new \PRAutoBlogger_Opik_Client(
|
||||
'test-key',
|
||||
'test-workspace'
|
||||
);
|
||||
|
||||
$span_payload = array(
|
||||
'id' => 'span-456',
|
||||
'trace_id' => 'trace-123',
|
||||
'name' => 'draft_generation',
|
||||
'type' => 'llm',
|
||||
'start_time' => '2026-04-23T10:00:00Z',
|
||||
'end_time' => '2026-04-23T10:00:15Z',
|
||||
'input' => array( 'prompt_hash' => 'sha256-abc' ),
|
||||
'output' => array( 'response' => 'test' ),
|
||||
'model' => 'gpt-4',
|
||||
'provider' => 'openrouter',
|
||||
'usage' => array(
|
||||
'prompt_tokens' => 100,
|
||||
'completion_tokens' => 50,
|
||||
'total_tokens' => 150,
|
||||
),
|
||||
);
|
||||
|
||||
Functions\when( 'wp_remote_post' )->justReturn(
|
||||
array(
|
||||
'response' => array( 'code' => 200 ),
|
||||
'body' => wp_json_encode( array( 'id' => 'span-456' ) ),
|
||||
)
|
||||
);
|
||||
|
||||
Functions\when( 'wp_remote_retrieve_response_code' )->justReturn( 200 );
|
||||
Functions\when( 'wp_remote_retrieve_body' )->justReturn( wp_json_encode( array( 'id' => 'span-456' ) ) );
|
||||
Functions\when( 'wp_json_encode' )->alias( 'json_encode' );
|
||||
Functions\when( 'is_wp_error' )->justReturn( false );
|
||||
|
||||
$result = $client->create_span( $span_payload );
|
||||
|
||||
$this->assertIsArray( $result );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test batch_create_spans respects the 100-item limit.
|
||||
*/
|
||||
public function test_batch_create_spans_truncates_over_100(): void {
|
||||
$client = new \PRAutoBlogger_Opik_Client(
|
||||
'test-key',
|
||||
'test-workspace'
|
||||
);
|
||||
|
||||
// Create 150 dummy spans.
|
||||
$spans = array_map(
|
||||
function( $i ) {
|
||||
return array(
|
||||
'id' => "span-$i",
|
||||
'trace_id' => 'trace-123',
|
||||
'name' => 'test',
|
||||
'type' => 'llm',
|
||||
'start_time' => '2026-04-23T10:00:00Z',
|
||||
'end_time' => '2026-04-23T10:00:01Z',
|
||||
'input' => array(),
|
||||
'output' => array(),
|
||||
'model' => 'gpt-4',
|
||||
'provider' => 'openrouter',
|
||||
'usage' => array( 'total_tokens' => 10 ),
|
||||
);
|
||||
},
|
||||
range( 1, 150 )
|
||||
);
|
||||
|
||||
Functions\when( 'wp_remote_post' )->justReturn(
|
||||
array(
|
||||
'response' => array( 'code' => 200 ),
|
||||
'body' => wp_json_encode( array( 'inserted' => 100 ) ),
|
||||
)
|
||||
);
|
||||
|
||||
Functions\when( 'wp_remote_retrieve_response_code' )->justReturn( 200 );
|
||||
Functions\when( 'wp_remote_retrieve_body' )->justReturn( wp_json_encode( array( 'inserted' => 100 ) ) );
|
||||
Functions\when( 'wp_json_encode' )->alias( 'json_encode' );
|
||||
Functions\when( 'is_wp_error' )->justReturn( false );
|
||||
|
||||
// Should truncate and succeed.
|
||||
$result = $client->batch_create_spans( $spans );
|
||||
|
||||
$this->assertIsArray( $result );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test that HTTP 500 errors trigger retry logic.
|
||||
*/
|
||||
public function test_retry_on_500_error(): void {
|
||||
$client = new \PRAutoBlogger_Opik_Client(
|
||||
'test-key',
|
||||
'test-workspace',
|
||||
'https://api.example.com/opik/',
|
||||
1 // 1-second timeout to make test fast
|
||||
);
|
||||
|
||||
$call_count = 0;
|
||||
Functions\when( 'wp_remote_post' )->alias(
|
||||
function() use ( &$call_count ) {
|
||||
++$call_count;
|
||||
if ( $call_count < 3 ) {
|
||||
// First two calls return 500.
|
||||
return array(
|
||||
'response' => array( 'code' => 500 ),
|
||||
'body' => 'Server error',
|
||||
);
|
||||
}
|
||||
// Third call succeeds.
|
||||
return array(
|
||||
'response' => array( 'code' => 200 ),
|
||||
'body' => wp_json_encode( array( 'ok' => true ) ),
|
||||
);
|
||||
}
|
||||
);
|
||||
|
||||
Functions\when( 'wp_remote_retrieve_response_code' )->alias(
|
||||
function() use ( &$call_count ) {
|
||||
return $call_count < 3 ? 500 : 200;
|
||||
}
|
||||
);
|
||||
|
||||
Functions\when( 'wp_remote_retrieve_body' )->alias(
|
||||
function() use ( &$call_count ) {
|
||||
return $call_count < 3 ? 'Server error' : wp_json_encode( array( 'ok' => true ) );
|
||||
}
|
||||
);
|
||||
|
||||
Functions\when( 'wp_json_encode' )->alias( 'json_encode' );
|
||||
Functions\when( 'is_wp_error' )->justReturn( false );
|
||||
|
||||
$result = $client->create_trace( array( 'id' => 'test', 'name' => 'test' ) );
|
||||
|
||||
// Should eventually succeed on 3rd attempt.
|
||||
$this->assertIsArray( $result );
|
||||
$this->assertEquals( 3, $call_count );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test that HTTP 400 errors don't retry.
|
||||
*/
|
||||
public function test_no_retry_on_400_error(): void {
|
||||
$client = new \PRAutoBlogger_Opik_Client(
|
||||
'test-key',
|
||||
'test-workspace'
|
||||
);
|
||||
|
||||
$call_count = 0;
|
||||
Functions\when( 'wp_remote_post' )->alias(
|
||||
function() use ( &$call_count ) {
|
||||
++$call_count;
|
||||
return array(
|
||||
'response' => array( 'code' => 400 ),
|
||||
'body' => wp_json_encode( array( 'error' => 'Bad request' ) ),
|
||||
);
|
||||
}
|
||||
);
|
||||
|
||||
Functions\when( 'wp_remote_retrieve_response_code' )->justReturn( 400 );
|
||||
Functions\when( 'wp_remote_retrieve_body' )->justReturn( wp_json_encode( array( 'error' => 'Bad request' ) ) );
|
||||
Functions\when( 'wp_json_encode' )->alias( 'json_encode' );
|
||||
Functions\when( 'is_wp_error' )->justReturn( false );
|
||||
|
||||
$result = $client->create_trace( array( 'id' => 'test', 'name' => 'test' ) );
|
||||
|
||||
// Should fail on first attempt, not retry.
|
||||
$this->assertNull( $result );
|
||||
$this->assertEquals( 1, $call_count );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test that API key and workspace are included in headers.
|
||||
*/
|
||||
public function test_headers_include_credentials(): void {
|
||||
$client = new \PRAutoBlogger_Opik_Client(
|
||||
'secret-key-12345',
|
||||
'my-workspace'
|
||||
);
|
||||
|
||||
$captured_headers = null;
|
||||
Functions\when( 'wp_remote_post' )->alias(
|
||||
function( $url, $args ) use ( &$captured_headers ) {
|
||||
$captured_headers = $args['headers'] ?? array();
|
||||
return array(
|
||||
'response' => array( 'code' => 200 ),
|
||||
'body' => wp_json_encode( array() ),
|
||||
);
|
||||
}
|
||||
);
|
||||
|
||||
Functions\when( 'wp_remote_retrieve_response_code' )->justReturn( 200 );
|
||||
Functions\when( 'wp_remote_retrieve_body' )->justReturn( wp_json_encode( array() ) );
|
||||
Functions\when( 'wp_json_encode' )->alias( 'json_encode' );
|
||||
Functions\when( 'is_wp_error' )->justReturn( false );
|
||||
|
||||
$client->create_trace( array( 'id' => 'test', 'name' => 'test' ) );
|
||||
|
||||
$this->assertIsArray( $captured_headers );
|
||||
$this->assertArrayHasKey( 'Authorization', $captured_headers );
|
||||
$this->assertEquals( 'secret-key-12345', $captured_headers['Authorization'] );
|
||||
$this->assertArrayHasKey( 'Comet-Workspace', $captured_headers );
|
||||
$this->assertEquals( 'my-workspace', $captured_headers['Comet-Workspace'] );
|
||||
}
|
||||
}
|
||||
241
tests/unit/Services/test-opik-dispatcher.php
Normal file
241
tests/unit/Services/test-opik-dispatcher.php
Normal file
@@ -0,0 +1,241 @@
|
||||
<?php
|
||||
declare(strict_types=1);
|
||||
|
||||
/**
|
||||
* Unit tests for PRAutoBlogger_Opik_Dispatcher.
|
||||
*
|
||||
* @see includes/services/opik/class-opik-dispatcher.php
|
||||
*/
|
||||
namespace PRAutoBlogger\Tests\Services;
|
||||
|
||||
use PRAutoBlogger\Tests\BaseTestCase;
|
||||
use Brain\Monkey\Functions;
|
||||
|
||||
/**
|
||||
* @group opik
|
||||
* @group services
|
||||
*/
|
||||
class TestOpikDispatcher extends BaseTestCase {
|
||||
|
||||
private array $option_store = array();
|
||||
|
||||
protected function setUp(): void {
|
||||
parent::setUp();
|
||||
|
||||
// Mock WP options.
|
||||
Functions\when( 'get_option' )->alias(
|
||||
function( $option, $default = false ) {
|
||||
return $this->option_store[ $option ] ?? $default;
|
||||
}
|
||||
);
|
||||
|
||||
Functions\when( 'update_option' )->alias(
|
||||
function( $option, $value ) {
|
||||
$this->option_store[ $option ] = $value;
|
||||
return true;
|
||||
}
|
||||
);
|
||||
|
||||
Functions\when( 'delete_option' )->alias(
|
||||
function( $option ) {
|
||||
unset( $this->option_store[ $option ] );
|
||||
return true;
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Test dispatcher skips if Opik is not enabled.
|
||||
*/
|
||||
public function test_dispatch_skips_if_disabled(): void {
|
||||
$client = $this->create_mock_client();
|
||||
$queue = $this->create_mock_queue();
|
||||
|
||||
// Queue has items but feature is disabled.
|
||||
$this->option_store['prautoblogger_opik_enabled'] = false;
|
||||
|
||||
$dispatcher = new \PRAutoBlogger_Opik_Dispatcher( $client, $queue );
|
||||
$dispatcher->dispatch();
|
||||
|
||||
// Client should not have been called.
|
||||
$this->assertTrue( true );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test dispatch() batches and posts spans.
|
||||
*/
|
||||
public function test_dispatch_posts_spans(): void {
|
||||
// Enable Opik and set credentials.
|
||||
$this->option_store['prautoblogger_opik_enabled'] = true;
|
||||
|
||||
// Mock constants.
|
||||
if ( ! defined( 'PRAUTOBLOGGER_OPIK_API_KEY' ) ) {
|
||||
define( 'PRAUTOBLOGGER_OPIK_API_KEY', 'test-key' );
|
||||
}
|
||||
if ( ! defined( 'PRAUTOBLOGGER_OPIK_WORKSPACE' ) ) {
|
||||
define( 'PRAUTOBLOGGER_OPIK_WORKSPACE', 'test-workspace' );
|
||||
}
|
||||
|
||||
$posted_spans = null;
|
||||
|
||||
$client = $this->getMockBuilder( \PRAutoBlogger_Opik_Client::class )
|
||||
->disableOriginalConstructor()
|
||||
->onlyMethods( array( 'batch_create_spans', 'batch_create_traces' ) )
|
||||
->getMock();
|
||||
|
||||
$client->expects( $this->once() )
|
||||
->method( 'batch_create_spans' )
|
||||
->willReturnCallback( function( $spans ) use ( &$posted_spans ) {
|
||||
$posted_spans = $spans;
|
||||
return array( 'inserted' => count( $spans ) );
|
||||
} );
|
||||
|
||||
$queue = $this->getMockBuilder( \PRAutoBlogger_Opik_Span_Queue::class )
|
||||
->disableOriginalConstructor()
|
||||
->onlyMethods( array( 'dequeue', 'get_queue_depth' ) )
|
||||
->getMock();
|
||||
|
||||
$queue->expects( $this->once() )
|
||||
->method( 'dequeue' )
|
||||
->with( 100 )
|
||||
->willReturn(
|
||||
array(
|
||||
array(
|
||||
'type' => 'span',
|
||||
'payload' => array( 'id' => 'span-1', 'name' => 'test' ),
|
||||
'attempt' => 0,
|
||||
'enqueued' => time(),
|
||||
),
|
||||
)
|
||||
);
|
||||
|
||||
$queue->method( 'get_queue_depth' )->willReturn( 0 );
|
||||
|
||||
$dispatcher = new \PRAutoBlogger_Opik_Dispatcher( $client, $queue );
|
||||
$dispatcher->dispatch();
|
||||
|
||||
$this->assertNotNull( $posted_spans );
|
||||
$this->assertCount( 1, $posted_spans );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test dispatcher handles empty queue gracefully.
|
||||
*/
|
||||
public function test_dispatch_handles_empty_queue(): void {
|
||||
$this->option_store['prautoblogger_opik_enabled'] = true;
|
||||
|
||||
if ( ! defined( 'PRAUTOBLOGGER_OPIK_API_KEY' ) ) {
|
||||
define( 'PRAUTOBLOGGER_OPIK_API_KEY', 'test-key' );
|
||||
}
|
||||
if ( ! defined( 'PRAUTOBLOGGER_OPIK_WORKSPACE' ) ) {
|
||||
define( 'PRAUTOBLOGGER_OPIK_WORKSPACE', 'test-workspace' );
|
||||
}
|
||||
|
||||
$client = $this->create_mock_client();
|
||||
$queue = $this->create_mock_queue( array() );
|
||||
|
||||
$dispatcher = new \PRAutoBlogger_Opik_Dispatcher( $client, $queue );
|
||||
$dispatcher->dispatch();
|
||||
|
||||
// Should not error.
|
||||
$this->assertTrue( true );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test dispatcher separates spans and traces.
|
||||
*/
|
||||
public function test_dispatch_separates_types(): void {
|
||||
$this->option_store['prautoblogger_opik_enabled'] = true;
|
||||
|
||||
if ( ! defined( 'PRAUTOBLOGGER_OPIK_API_KEY' ) ) {
|
||||
define( 'PRAUTOBLOGGER_OPIK_API_KEY', 'test-key' );
|
||||
}
|
||||
if ( ! defined( 'PRAUTOBLOGGER_OPIK_WORKSPACE' ) ) {
|
||||
define( 'PRAUTOBLOGGER_OPIK_WORKSPACE', 'test-workspace' );
|
||||
}
|
||||
|
||||
$posted_traces = null;
|
||||
$posted_spans = null;
|
||||
|
||||
$client = $this->getMockBuilder( \PRAutoBlogger_Opik_Client::class )
|
||||
->disableOriginalConstructor()
|
||||
->onlyMethods( array( 'batch_create_traces', 'batch_create_spans' ) )
|
||||
->getMock();
|
||||
|
||||
$client->method( 'batch_create_traces' )->willReturnCallback(
|
||||
function( $traces ) use ( &$posted_traces ) {
|
||||
$posted_traces = $traces;
|
||||
return array( 'inserted' => count( $traces ) );
|
||||
}
|
||||
);
|
||||
|
||||
$client->method( 'batch_create_spans' )->willReturnCallback(
|
||||
function( $spans ) use ( &$posted_spans ) {
|
||||
$posted_spans = $spans;
|
||||
return array( 'inserted' => count( $spans ) );
|
||||
}
|
||||
);
|
||||
|
||||
$queue = $this->getMockBuilder( \PRAutoBlogger_Opik_Span_Queue::class )
|
||||
->disableOriginalConstructor()
|
||||
->onlyMethods( array( 'dequeue', 'get_queue_depth' ) )
|
||||
->getMock();
|
||||
|
||||
$queue->method( 'dequeue' )->willReturn(
|
||||
array(
|
||||
array(
|
||||
'type' => 'trace',
|
||||
'payload' => array( 'id' => 'trace-1' ),
|
||||
'attempt' => 0,
|
||||
'enqueued' => time(),
|
||||
),
|
||||
array(
|
||||
'type' => 'span',
|
||||
'payload' => array( 'id' => 'span-1' ),
|
||||
'attempt' => 0,
|
||||
'enqueued' => time(),
|
||||
),
|
||||
)
|
||||
);
|
||||
|
||||
$queue->method( 'get_queue_depth' )->willReturn( 0 );
|
||||
|
||||
$dispatcher = new \PRAutoBlogger_Opik_Dispatcher( $client, $queue );
|
||||
$dispatcher->dispatch();
|
||||
|
||||
$this->assertNotNull( $posted_traces );
|
||||
$this->assertCount( 1, $posted_traces );
|
||||
$this->assertNotNull( $posted_spans );
|
||||
$this->assertCount( 1, $posted_spans );
|
||||
}
|
||||
|
||||
/**
|
||||
* Helper to create a mock client.
|
||||
*/
|
||||
private function create_mock_client() {
|
||||
$client = $this->getMockBuilder( \PRAutoBlogger_Opik_Client::class )
|
||||
->disableOriginalConstructor()
|
||||
->onlyMethods( array( 'batch_create_spans', 'batch_create_traces' ) )
|
||||
->getMock();
|
||||
|
||||
$client->method( 'batch_create_spans' )->willReturn( array() );
|
||||
$client->method( 'batch_create_traces' )->willReturn( array() );
|
||||
|
||||
return $client;
|
||||
}
|
||||
|
||||
/**
|
||||
* Helper to create a mock queue.
|
||||
*/
|
||||
private function create_mock_queue( array $items = array() ) {
|
||||
$queue = $this->getMockBuilder( \PRAutoBlogger_Opik_Span_Queue::class )
|
||||
->disableOriginalConstructor()
|
||||
->onlyMethods( array( 'dequeue', 'get_queue_depth' ) )
|
||||
->getMock();
|
||||
|
||||
$queue->method( 'dequeue' )->willReturn( $items );
|
||||
$queue->method( 'get_queue_depth' )->willReturn( count( $items ) );
|
||||
|
||||
return $queue;
|
||||
}
|
||||
}
|
||||
248
tests/unit/Services/test-opik-span-queue.php
Normal file
248
tests/unit/Services/test-opik-span-queue.php
Normal file
@@ -0,0 +1,248 @@
|
||||
<?php
|
||||
declare(strict_types=1);
|
||||
|
||||
/**
|
||||
* Unit tests for PRAutoBlogger_Opik_Span_Queue.
|
||||
*
|
||||
* @see includes/services/opik/class-opik-span-queue.php
|
||||
*/
|
||||
namespace PRAutoBlogger\Tests\Services;
|
||||
|
||||
use PRAutoBlogger\Tests\BaseTestCase;
|
||||
use Brain\Monkey\Functions;
|
||||
|
||||
/**
|
||||
* @group opik
|
||||
* @group services
|
||||
*/
|
||||
class TestOpikSpanQueue extends BaseTestCase {
|
||||
|
||||
private array $option_store = array();
|
||||
|
||||
protected function setUp(): void {
|
||||
parent::setUp();
|
||||
|
||||
// Mock WP options functions.
|
||||
Functions\when( 'get_option' )->alias(
|
||||
function( $option, $default = false ) {
|
||||
return $this->option_store[ $option ] ?? $default;
|
||||
}
|
||||
);
|
||||
|
||||
Functions\when( 'update_option' )->alias(
|
||||
function( $option, $value ) {
|
||||
$this->option_store[ $option ] = $value;
|
||||
return true;
|
||||
}
|
||||
);
|
||||
|
||||
Functions\when( 'delete_option' )->alias(
|
||||
function( $option ) {
|
||||
unset( $this->option_store[ $option ] );
|
||||
return true;
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Test enqueue() adds item to queue.
|
||||
*/
|
||||
public function test_enqueue_adds_item(): void {
|
||||
$queue = new \PRAutoBlogger_Opik_Span_Queue();
|
||||
|
||||
$payload = array(
|
||||
'id' => 'span-1',
|
||||
'name' => 'test',
|
||||
'trace_id' => 'trace-1',
|
||||
);
|
||||
|
||||
$success = $queue->enqueue( $payload, 'span' );
|
||||
|
||||
$this->assertTrue( $success );
|
||||
$this->assertEquals( 1, $queue->get_queue_depth() );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test enqueue() respects max depth limit.
|
||||
*/
|
||||
public function test_enqueue_respects_max_depth(): void {
|
||||
$queue = new \PRAutoBlogger_Opik_Span_Queue();
|
||||
|
||||
// Fill queue to max.
|
||||
for ( $i = 0; $i < 1000; ++$i ) {
|
||||
$success = $queue->enqueue(
|
||||
array( 'id' => "span-$i", 'name' => 'test' ),
|
||||
'span'
|
||||
);
|
||||
if ( ! $success ) {
|
||||
// Queue full.
|
||||
$this->assertEquals( 1000, $queue->get_queue_depth() );
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// If we got here, queue filled successfully.
|
||||
$this->assertEquals( 1000, $queue->get_queue_depth() );
|
||||
|
||||
// Try to add one more — should fail.
|
||||
$success = $queue->enqueue(
|
||||
array( 'id' => 'overflow', 'name' => 'test' ),
|
||||
'span'
|
||||
);
|
||||
$this->assertFalse( $success );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test dequeue() returns items in order.
|
||||
*/
|
||||
public function test_dequeue_returns_items(): void {
|
||||
$queue = new \PRAutoBlogger_Opik_Span_Queue();
|
||||
|
||||
$queue->enqueue( array( 'id' => 'span-1' ), 'span' );
|
||||
$queue->enqueue( array( 'id' => 'span-2' ), 'span' );
|
||||
$queue->enqueue( array( 'id' => 'span-3' ), 'span' );
|
||||
|
||||
$batch = $queue->dequeue( 10 );
|
||||
|
||||
$this->assertCount( 3, $batch );
|
||||
$this->assertEquals( 'span-1', $batch[0]['payload']['id'] );
|
||||
$this->assertEquals( 'span-2', $batch[1]['payload']['id'] );
|
||||
$this->assertEquals( 'span-3', $batch[2]['payload']['id'] );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test dequeue() respects limit parameter.
|
||||
*/
|
||||
public function test_dequeue_respects_limit(): void {
|
||||
$queue = new \PRAutoBlogger_Opik_Span_Queue();
|
||||
|
||||
for ( $i = 1; $i <= 50; ++$i ) {
|
||||
$queue->enqueue( array( 'id' => "span-$i" ), 'span' );
|
||||
}
|
||||
|
||||
$batch = $queue->dequeue( 10 );
|
||||
|
||||
$this->assertCount( 10, $batch );
|
||||
$this->assertEquals( 40, $queue->get_queue_depth() );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test reenqueue() increments attempt counter.
|
||||
*/
|
||||
public function test_reenqueue_increments_attempt(): void {
|
||||
$queue = new \PRAutoBlogger_Opik_Span_Queue();
|
||||
|
||||
$items = array(
|
||||
array(
|
||||
'type' => 'span',
|
||||
'payload' => array( 'id' => 'span-1' ),
|
||||
'attempt' => 0,
|
||||
'enqueued' => time(),
|
||||
),
|
||||
);
|
||||
|
||||
$queue->reenqueue( $items );
|
||||
|
||||
$batch = $queue->dequeue( 10 );
|
||||
$this->assertEquals( 1, $batch[0]['attempt'] );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test reenqueue() drops items after max retries.
|
||||
*/
|
||||
public function test_reenqueue_drops_after_max_attempts(): void {
|
||||
$queue = new \PRAutoBlogger_Opik_Span_Queue();
|
||||
|
||||
$items = array(
|
||||
array(
|
||||
'type' => 'span',
|
||||
'payload' => array( 'id' => 'span-1' ),
|
||||
'attempt' => 3, // Already at max.
|
||||
'enqueued' => time(),
|
||||
),
|
||||
);
|
||||
|
||||
$queue->reenqueue( $items );
|
||||
|
||||
// Item should be dropped, queue should be empty.
|
||||
$this->assertEquals( 0, $queue->get_queue_depth() );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test dequeue() removes expired items (TTL > 12 hours).
|
||||
*/
|
||||
public function test_dequeue_removes_expired_items(): void {
|
||||
$queue = new \PRAutoBlogger_Opik_Span_Queue();
|
||||
|
||||
$now = time();
|
||||
|
||||
// Add a fresh item.
|
||||
$fresh = array(
|
||||
'type' => 'span',
|
||||
'payload' => array( 'id' => 'fresh' ),
|
||||
'enqueued' => $now,
|
||||
);
|
||||
|
||||
// Add an expired item (13 hours old).
|
||||
$expired = array(
|
||||
'type' => 'span',
|
||||
'payload' => array( 'id' => 'expired' ),
|
||||
'enqueued' => $now - 46800, // 13 hours in seconds.
|
||||
);
|
||||
|
||||
$this->option_store['prautoblogger_opik_queue'] = array( $fresh, $expired );
|
||||
|
||||
$batch = $queue->dequeue( 10 );
|
||||
|
||||
// Only fresh item should be returned.
|
||||
$this->assertCount( 1, $batch );
|
||||
$this->assertEquals( 'fresh', $batch[0]['payload']['id'] );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test get_queue_depth() returns item count.
|
||||
*/
|
||||
public function test_get_queue_depth(): void {
|
||||
$queue = new \PRAutoBlogger_Opik_Span_Queue();
|
||||
|
||||
$this->assertEquals( 0, $queue->get_queue_depth() );
|
||||
|
||||
$queue->enqueue( array( 'id' => 'span-1' ), 'span' );
|
||||
$this->assertEquals( 1, $queue->get_queue_depth() );
|
||||
|
||||
$queue->enqueue( array( 'id' => 'span-2' ), 'span' );
|
||||
$this->assertEquals( 2, $queue->get_queue_depth() );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test clear() empties the queue.
|
||||
*/
|
||||
public function test_clear_empties_queue(): void {
|
||||
$queue = new \PRAutoBlogger_Opik_Span_Queue();
|
||||
|
||||
$queue->enqueue( array( 'id' => 'span-1' ), 'span' );
|
||||
$queue->enqueue( array( 'id' => 'span-2' ), 'span' );
|
||||
|
||||
$this->assertEquals( 2, $queue->get_queue_depth() );
|
||||
|
||||
$queue->clear();
|
||||
|
||||
$this->assertEquals( 0, $queue->get_queue_depth() );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test enqueue() with both span and trace types.
|
||||
*/
|
||||
public function test_enqueue_handles_trace_type(): void {
|
||||
$queue = new \PRAutoBlogger_Opik_Span_Queue();
|
||||
|
||||
$queue->enqueue( array( 'id' => 'trace-1' ), 'trace' );
|
||||
$queue->enqueue( array( 'id' => 'span-1' ), 'span' );
|
||||
|
||||
$batch = $queue->dequeue( 10 );
|
||||
|
||||
$this->assertCount( 2, $batch );
|
||||
$this->assertEquals( 'trace', $batch[0]['type'] );
|
||||
$this->assertEquals( 'span', $batch[1]['type'] );
|
||||
}
|
||||
}
|
||||
222
tests/unit/Services/test-opik-trace-context.php
Normal file
222
tests/unit/Services/test-opik-trace-context.php
Normal file
@@ -0,0 +1,222 @@
|
||||
<?php
|
||||
declare(strict_types=1);
|
||||
|
||||
/**
|
||||
* Unit tests for PRAutoBlogger_Opik_Trace_Context.
|
||||
*
|
||||
* @see includes/services/opik/class-opik-trace-context.php
|
||||
*/
|
||||
namespace PRAutoBlogger\Tests\Services;
|
||||
|
||||
use PRAutoBlogger\Tests\BaseTestCase;
|
||||
use Brain\Monkey\Functions;
|
||||
|
||||
/**
|
||||
* @group opik
|
||||
* @group services
|
||||
*/
|
||||
class TestOpikTraceContext extends BaseTestCase {
|
||||
|
||||
protected function setUp(): void {
|
||||
parent::setUp();
|
||||
|
||||
// Stub UUID generation.
|
||||
Functions\when( 'wp_generate_uuid4' )->alias( function() {
|
||||
static $counter = 0;
|
||||
return 'uuid-' . ++$counter;
|
||||
} );
|
||||
}
|
||||
|
||||
protected function tearDown(): void {
|
||||
// Clean up singleton.
|
||||
\PRAutoBlogger_Opik_Trace_Context::teardown();
|
||||
parent::tearDown();
|
||||
}
|
||||
|
||||
/**
|
||||
* Test current() returns singleton instance.
|
||||
*/
|
||||
public function test_current_returns_singleton(): void {
|
||||
$ctx1 = \PRAutoBlogger_Opik_Trace_Context::current();
|
||||
$ctx2 = \PRAutoBlogger_Opik_Trace_Context::current();
|
||||
|
||||
$this->assertSame( $ctx1, $ctx2 );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test init_trace() generates a UUID.
|
||||
*/
|
||||
public function test_init_trace_generates_uuid(): void {
|
||||
$ctx = \PRAutoBlogger_Opik_Trace_Context::current();
|
||||
$trace_id = $ctx->init_trace();
|
||||
|
||||
$this->assertNotEmpty( $trace_id );
|
||||
$this->assertStringStartsWith( 'uuid-', $trace_id );
|
||||
$this->assertEquals( $trace_id, $ctx->get_trace_id() );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test start_span() creates span and returns ID.
|
||||
*/
|
||||
public function test_start_span_creates_span(): void {
|
||||
$ctx = \PRAutoBlogger_Opik_Trace_Context::current();
|
||||
$ctx->init_trace();
|
||||
|
||||
$span_id = $ctx->start_span(
|
||||
array(
|
||||
'name' => 'draft_generation',
|
||||
'type' => 'llm',
|
||||
'model' => 'gpt-4',
|
||||
'provider' => 'openrouter',
|
||||
'input' => array( 'prompt_hash' => 'sha256-abc' ),
|
||||
)
|
||||
);
|
||||
|
||||
$this->assertNotEmpty( $span_id );
|
||||
$this->assertStringStartsWith( 'uuid-', $span_id );
|
||||
|
||||
$span = $ctx->get_span( $span_id );
|
||||
$this->assertIsArray( $span );
|
||||
$this->assertEquals( 'draft_generation', $span['name'] );
|
||||
$this->assertEquals( 'llm', $span['type'] );
|
||||
$this->assertEquals( 'gpt-4', $span['model'] );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test end_span() records output and usage.
|
||||
*/
|
||||
public function test_end_span_records_data(): void {
|
||||
$ctx = \PRAutoBlogger_Opik_Trace_Context::current();
|
||||
$ctx->init_trace();
|
||||
|
||||
$span_id = $ctx->start_span(
|
||||
array(
|
||||
'name' => 'test_span',
|
||||
'type' => 'llm',
|
||||
)
|
||||
);
|
||||
|
||||
$ctx->end_span(
|
||||
$span_id,
|
||||
array(
|
||||
'output' => array( 'response' => 'Hello world' ),
|
||||
'usage' => array(
|
||||
'prompt_tokens' => 50,
|
||||
'completion_tokens' => 25,
|
||||
'total_tokens' => 75,
|
||||
),
|
||||
)
|
||||
);
|
||||
|
||||
$span = $ctx->get_span( $span_id );
|
||||
$this->assertIsArray( $span['output'] );
|
||||
$this->assertEquals( 'Hello world', $span['output']['response'] );
|
||||
$this->assertEquals( 75, $span['usage']['total_tokens'] );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test get_spans() returns all active spans.
|
||||
*/
|
||||
public function test_get_spans_returns_all(): void {
|
||||
$ctx = \PRAutoBlogger_Opik_Trace_Context::current();
|
||||
$ctx->init_trace();
|
||||
|
||||
$span1 = $ctx->start_span( array( 'name' => 'span1', 'type' => 'llm' ) );
|
||||
$span2 = $ctx->start_span( array( 'name' => 'span2', 'type' => 'llm' ) );
|
||||
$span3 = $ctx->start_span( array( 'name' => 'span3', 'type' => 'llm' ) );
|
||||
|
||||
$all_spans = $ctx->get_spans();
|
||||
|
||||
$this->assertCount( 3, $all_spans );
|
||||
$this->assertArrayHasKey( $span1, $all_spans );
|
||||
$this->assertArrayHasKey( $span2, $all_spans );
|
||||
$this->assertArrayHasKey( $span3, $all_spans );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test finalize_trace() returns complete trace object.
|
||||
*/
|
||||
public function test_finalize_trace_includes_all_data(): void {
|
||||
Functions\when( 'get_option' )->alias(
|
||||
function( $option, $default = false ) {
|
||||
if ( 'prautoblogger_opik_project_name' === $option ) {
|
||||
return 'prautoblogger';
|
||||
}
|
||||
return $default;
|
||||
}
|
||||
);
|
||||
|
||||
$ctx = \PRAutoBlogger_Opik_Trace_Context::current();
|
||||
$trace_id = $ctx->init_trace();
|
||||
|
||||
$span1 = $ctx->start_span( array( 'name' => 'step1', 'type' => 'llm' ) );
|
||||
$ctx->end_span( $span1, array( 'output' => array( 'ok' => true ) ) );
|
||||
|
||||
$span2 = $ctx->start_span( array( 'name' => 'step2', 'type' => 'llm' ) );
|
||||
$ctx->end_span( $span2, array( 'output' => array( 'ok' => true ) ) );
|
||||
|
||||
$trace = $ctx->finalize_trace();
|
||||
|
||||
$this->assertEquals( $trace_id, $trace['id'] );
|
||||
$this->assertEquals( 'article_generation', $trace['name'] );
|
||||
$this->assertEquals( 'prautoblogger', $trace['project_name'] );
|
||||
$this->assertArrayHasKey( 'start_time', $trace );
|
||||
$this->assertArrayHasKey( 'end_time', $trace );
|
||||
$this->assertCount( 2, $trace['spans'] );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test teardown() clears the singleton.
|
||||
*/
|
||||
public function test_teardown_clears_singleton(): void {
|
||||
$ctx1 = \PRAutoBlogger_Opik_Trace_Context::current();
|
||||
$ctx1->init_trace();
|
||||
|
||||
\PRAutoBlogger_Opik_Trace_Context::teardown();
|
||||
|
||||
$ctx2 = \PRAutoBlogger_Opik_Trace_Context::current();
|
||||
|
||||
// Should be a new instance.
|
||||
$this->assertNotSame( $ctx1, $ctx2 );
|
||||
$this->assertEmpty( $ctx2->get_trace_id() );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test warning logged for ending unknown span.
|
||||
*/
|
||||
public function test_warning_for_unknown_span(): void {
|
||||
$ctx = \PRAutoBlogger_Opik_Trace_Context::current();
|
||||
$ctx->init_trace();
|
||||
|
||||
// Try to end a span that doesn't exist.
|
||||
$ctx->end_span( 'unknown-uuid-12345' );
|
||||
|
||||
// If we got here without an exception, the warning was logged.
|
||||
$this->assertTrue( true );
|
||||
}
|
||||
|
||||
/**
|
||||
* Test span input is stored as-is (caller responsible for redaction).
|
||||
*/
|
||||
public function test_span_input_stored_as_is(): void {
|
||||
$ctx = \PRAutoBlogger_Opik_Trace_Context::current();
|
||||
$ctx->init_trace();
|
||||
|
||||
$input_data = array(
|
||||
'prompt_hash' => 'sha256-abc123',
|
||||
'model' => 'gpt-4',
|
||||
'version' => '1.0',
|
||||
);
|
||||
|
||||
$span_id = $ctx->start_span(
|
||||
array(
|
||||
'name' => 'test',
|
||||
'type' => 'llm',
|
||||
'input' => $input_data,
|
||||
)
|
||||
);
|
||||
|
||||
$span = $ctx->get_span( $span_id );
|
||||
$this->assertEquals( $input_data, $span['input'] );
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user