Opik Wave 1 Commit 3: Version bump, docs, and hook registration

Version & Changelog: - prautoblogger.php: 0.11.0 → 0.12.0 - Add Opik PHP constants (PRAUTOBLOGGER_OPIK_API_KEY, WORKSPACE, URL_OVERRIDE) with fallback empty defaults so wp-config.php can override - CHANGELOG.md: v0.12.0 section describing Wave 1 scope, feature flag, observability benefits, cost model, and Wave 2 preview Documentation: - ARCHITECTURE.md: New 'Opik LLM Observability (Optional)' section with feature-flag design, credential isolation, span lifecycle, call sites, cost/performance implications, and code structure overview - CONVENTIONS.md: Step-by-step guide 'Instrumenting a New LLM Call Site' with code examples, key rules (no full prompts, async dispatch, separate cost tracking), and minimal instrumentation pattern Hook Registration: - class-prautoblogger.php: Register 'prautoblogger_opik_dispatch' cron action * Instantiates Opik_Client, Opik_Span_Queue, Opik_Dispatcher * Checks credentials and calls dispatcher->dispatch() * Updates 'prautoblogger_opik_last_dispatch' option for admin status display - Instantiate Opik_Settings in admin_init for admin UI registration All changes preserve ≤ 300-line files. Part of Wave 1 (observability skeleton). Target: v0.12.0
2026-04-23 14:51:51 +00:00
parent ca5fdfacc4
commit fe8c242a64
5 changed files with 217 additions and 3 deletions
--- a/ARCHITECTURE.md
+++ b/ARCHITECTURE.md
@@ -480,6 +480,50 @@ Stored on every PRAutoBlogger-generated post:

 ---

+## Opik LLM Observability (Optional)
+
+**Status:** Wave 1 (v0.12.0) — tracing skeleton. Wave 2 (future) — eval harness with frozen-dataset regression testing.
+
+Opik is a cloud-hosted LLM observability platform (Comet) providing:
+- **Per-call tracing:** Every OpenRouter call generates a span with model, tokens, latency metadata.
+- **Trace grouping:** One trace per article generation bundles all spans (draft, QA, image prompt).
+- **Prompt versioning:** Foundation for Wave 2 — compare new prompt revisions against a frozen reference dataset using LLM-as-judge scoring on rubric axes (factual accuracy, style match, hallucination risk).
+
+**Architecture:**
+- **Feature flag:** `prautoblogger_opik_enabled` (WP option, default off). When false, zero network traffic to Opik.
+- **Credentials:** API key and workspace from PHP constants `PRAUTOBLOGGER_OPIK_API_KEY`, `PRAUTOBLOGGER_OPIK_WORKSPACE` (defined in wp-config.php). Never stored in DB.
+- **Span lifecycle:** 
+  1. `Opik_Trace_Context::init_trace()` generates UUID at article-worker start.
+  2. Each LLM call wraps with `start_span()` → LLM request → `end_span()` with tokens + response.
+  3. `finalize_trace()` + `Opik_Span_Queue::enqueue()` accumulates trace + spans for async dispatch.
+  4. `prautoblogger_opik_dispatch` cron action batches spans/traces (max 100 per POST) to `/v1/private/spans/batch`, `/v1/private/traces/batch`.
+- **Retry logic:** Exponential backoff (2s, 4s, 8s) on 5xx errors; permanent abort on 4xx; dropped after 3 attempts per item.
+- **Queue:** WP options-backed; max 1000 items; 12-hour TTL; persists across page reloads for truly async dispatch.
+
+**Call sites instrumented (Wave 1):**
+- Draft generation (single-pass or multi-step outline/draft/polish).
+- Editorial review (chief editor LLM call).
+- Image prompt generation (deferred; file at 314 LOC, over limit).
+
+**Cost & Performance:**
+- Opik free tier: 25k spans/month. Estimated current usage ~4.2k/month (20 posts/day × 7 calls × 30 days). Headroom to 5× growth.
+- Impact on article generation: async dispatch, zero blocking latency overhead.
+- Wave 2 (eval harness): adds LLM-as-judge calls (~$14/month at Haiku pricing, assuming 2 eval runs/week).
+
+**Code structure:**
+```
+includes/services/opik/
+  class-opik-client.php            — REST client (auth, retry, batch POSTs)
+  class-opik-trace-context.php     — per-request trace state singleton
+  class-opik-span-queue.php        — WP options-backed async queue
+  class-opik-dispatcher.php        — cron handler, batch dispatch
+
+includes/admin/
+  class-opik-settings.php          — admin UI (toggle, project name, status)
+```
+
+---
+
 ## Key Decisions

 ### #1: OpenRouter over direct provider APIs
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,37 @@ All notable changes to PRAutoBlogger will be documented in this file.
 Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project uses [Semantic Versioning](https://semver.org/).

+## [0.12.0] — 2026-04-23
+
+### Added
+- **Opik LLM observability integration (Wave 1: tracing skeleton).** Cloud-hosted observability via Comet Opik for detailed per-call tracing and prompt-version regression testing. Captured via async fire-and-forget dispatch to avoid blocking article generation.
+
+- **Opik REST client.** Minimal HTTP client (no third-party SDK) with exponential backoff retry, batch endpoints, and credential isolation (API key from wp-config.php constants, never in DB).
+
+- **Per-request trace context singleton.** Holds trace ID + span stack for one article generation. UUID threading across draft generation, editorial review, and image prompt generation.
+
+- **Async span/trace queue.** WP options-backed queue with TTL expiry (12 hours) and max depth (1000 items). Drained by cron dispatcher with per-batch retry logic.
+
+- **Cron dispatcher.** `prautoblogger_opik_dispatch` action scheduled via `wp_schedule_single_event()` drains queue in batches of up to 100 spans.
+
+- **Opik admin settings section.** Toggle to enable/disable tracing (default off), project name field, opt-in full-prompt capture, read-only status display (API key configured, last dispatch, queue depth).
+
+- **Span instrumentation at four LLM call sites:** single-pass article generation, outline/draft/polish stages (multi-step), and editorial review. Each span captures model name, provider, usage tokens.
+
+- **Comprehensive unit test suite.** 4 test files covering client (auth, retry, batch limits), trace context (UUID generation, span stacking), queue (enqueue/dequeue, expiry, reenqueue), and dispatcher (batch POST, separation of traces vs. spans).
+
+### Changed
+- **class-article-worker.php:** Initializes Opik trace context at generation start, enqueues finalized trace + spans at end if feature enabled.
+
+- **class-content-generator.php:** Wraps single-pass and multi-step stages (outline, draft, polish) with Opik spans capturing token usage.
+
+- **class-chief-editor.php:** Wraps editorial review LLM call with Opik span.
+
+### Notes
+- **Feature-flag default OFF.** Zero network traffic to Opik unless `prautoblogger_opik_enabled` option is true AND API credentials are defined in wp-config.php.
+- **Image prompt generation instrumentation deferred.** File already at 314 LOC (over 300-line rule); can be instrumented in Wave 2 without modifying existing code.
+- **Wave 2 holds eval harness.** CLI command for frozen-dataset regression testing with LLM-as-judge scoring (separate thread after v0.12.0 ships and is live-tested).
+
 ## [0.11.0] — 2026-04-23

 ### Added
--- a/CONVENTIONS.md
+++ b/CONVENTIONS.md
@@ -332,4 +332,104 @@ curl -s -X POST -H "Authorization: token $GH_TOKEN" \

 ### When this changes

-If the peptiderepo GitHub account is ever upgraded to Pro (or the repo goes public), replace this soft-enforcement section with a note pointing to the real branch protection rules in repo settings.
+If the peptiderepo GitHub account is ever upgraded to Pro (or the repo goes public), replace this soft-enforcement section with a note pointing to the real branch protection rules in repo settings.
+## Instrumenting a New LLM Call Site for Opik (v0.12.0+)
+
+When you add a new LLM call (via `$llm->send_chat_completion()`), follow this pattern to capture it in Opik:
+
+### 1. Get the trace context
+
+At the start of your method, grab the per-request singleton:
+
+```php
+$ctx = PRAutoBlogger_Opik_Trace_Context::current();
+```
+
+### 2. Start the span
+
+Before the LLM call, declare a span with metadata:
+
+```php
+$span_id = $ctx->start_span(
+	array(
+		'name'     => 'my_llm_operation',  // kebab-case, descriptive
+		'type'     => 'llm',
+		'model'    => $model,               // e.g. 'gpt-4', from get_option()
+		'provider' => 'openrouter',         // or the actual provider name
+		'input'    => array( 'key' => 'value' ), // optional metadata (no full prompts!)
+	)
+);
+```
+
+### 3. Make the LLM call (unchanged)
+
+```php
+$response = $this->llm->send_chat_completion(
+	array(
+		array( 'role' => 'system', 'content' => $system_prompt ),
+		array( 'role' => 'user', 'content' => $user_prompt ),
+	),
+	$model,
+	array( 'temperature' => 0.7, 'max_tokens' => 4000 )
+);
+```
+
+### 4. End the span with output
+
+After the response, record the outcome:
+
+```php
+$ctx->end_span(
+	$span_id,
+	array(
+		'output' => array( 'response_length' => strlen( $response['content'] ?? '' ) ),
+		'usage'  => array(
+			'prompt_tokens'     => $response['prompt_tokens'],
+			'completion_tokens' => $response['completion_tokens'],
+			'total_tokens'      => $response['prompt_tokens'] + $response['completion_tokens'],
+		),
+	)
+);
+```
+
+### Key rules
+
+- **Never log full prompts.** Use prompt hashes or version tags in `input`. Full text blows payload size and leaks content policy.
+- **Check the feature flag at call-site level?** No. The singleton is always available; Opik_Dispatcher checks the flag before dispatch.
+- **Async dispatch?** Yes. Spans are queued; dispatch happens via `prautoblogger_opik_dispatch` cron (no blocking).
+- **Cost logging?** Unchanged. Your `cost_tracker->log_api_call()` remains separate from Opik; Opik is observability-only.
+
+### Example: minimal instrumentation
+
+```php
+public function my_operation( $input ) {
+	$model = get_option( 'my_model', 'default-model' );
+	$ctx = PRAutoBlogger_Opik_Trace_Context::current();
+
+	$span_id = $ctx->start_span(
+		array(
+			'name'     => 'my_operation',
+			'type'     => 'llm',
+			'model'    => $model,
+			'provider' => 'openrouter',
+		)
+	);
+
+	$response = $this->llm->send_chat_completion( /* ... */ );
+
+	$ctx->end_span(
+		$span_id,
+		array(
+			'usage' => array(
+				'prompt_tokens'     => $response['prompt_tokens'],
+				'completion_tokens' => $response['completion_tokens'],
+				'total_tokens'      => $response['prompt_tokens'] + $response['completion_tokens'],
+			),
+		)
+	);
+
+	return $response['content'];
+}
+```
+
+That's it. No config, no feature-flag checks, no try/catch needed.
--- a/includes/class-prautoblogger.php
+++ b/includes/class-prautoblogger.php
@@ -116,6 +116,30 @@ class PRAutoBlogger {
 		$registry = $this->executor->get_model_registry();
 		add_action( 'prautoblogger_refresh_model_registry', array( $registry, 'refresh' ) );

+		// Opik observability: async trace/span dispatch.
+		add_action(
+			'prautoblogger_opik_dispatch',
+			static function (): void {
+				if ( ! defined( 'PRAUTOBLOGGER_OPIK_API_KEY' ) || empty( PRAUTOBLOGGER_OPIK_API_KEY ) ) {
+					return;
+				}
+				$client = new PRAutoBlogger_Opik_Client(
+					PRAUTOBLOGGER_OPIK_API_KEY,
+					PRAUTOBLOGGER_OPIK_WORKSPACE,
+					PRAUTOBLOGGER_OPIK_URL_OVERRIDE
+				);
+				$queue = new PRAutoBlogger_Opik_Span_Queue();
+				$dispatcher = new PRAutoBlogger_Opik_Dispatcher( $client, $queue );
+				$dispatcher->dispatch();
+				update_option( 'prautoblogger_opik_last_dispatch', time() );
+			}
+		);
+
+		// Opik settings registration.
+		if ( is_admin() ) {
+			new PRAutoBlogger_Opik_Settings();
+		}
+
 		// Single-idea generation from the Ideas browser page.
 		add_action( 'prautoblogger_generate_from_idea', array( 'PRAutoBlogger_Ideas_Browser', 'on_cron_generate_from_idea' ) );

--- a/prautoblogger.php
+++ b/prautoblogger.php
@@ -9,7 +9,7 @@
 * Plugin Name:       PRAutoBlogger
 * Plugin URI:        https://peptiderepo.com/prautoblogger
 * Description:       Monitors social media for trending topics, generates SEO-friendly blog posts using AI, and publishes them on a daily schedule with full cost tracking and self-improvement metrics.
- * Version:           0.11.0
+ * Version:           0.12.0
 * Requires at least: 6.0
 * Requires PHP:      7.4
 * Author:            PeptideRepo
@@ -35,7 +35,7 @@ if ( ! defined( 'ABSPATH' ) ) {
 | Defined here so every file in the plugin can reference paths, versions,
 | and limits without magic strings.
 */
-define( 'PRAUTOBLOGGER_VERSION', '0.11.0' );
+define( 'PRAUTOBLOGGER_VERSION', '0.12.0' );
 define( 'PRAUTOBLOGGER_DB_VERSION', '1.1.0' );
 define( 'PRAUTOBLOGGER_PLUGIN_FILE', __FILE__ );
 define( 'PRAUTOBLOGGER_PLUGIN_DIR', plugin_dir_path( __FILE__ ) );
@@ -66,6 +66,21 @@ define(
 	'PRAUTOBLOGGER_DEFAULT_IMAGE_STYLE_SUFFIX',
 	'Style: a single-panel newspaper comic in classic black ink on white paper, simple clean linework like The Far Side or B.C., slightly yellowed newsprint texture, hand-drawn crosshatching for shading, bold panel border, warm and humorous tone, visible in a newspaper comics section. IMPORTANT: Do not render any text, captions, speech bubbles, or words inside the image.'
 );
+// Opik observability (feature-flag gated).
+// API credentials must be defined in wp-config.php constants:
+// - PRAUTOBLOGGER_OPIK_API_KEY: API key for authentication
+// - PRAUTOBLOGGER_OPIK_WORKSPACE: Opik workspace name (e.g. 'peptiderepo')
+// - PRAUTOBLOGGER_OPIK_URL_OVERRIDE: Optional; defaults to https://www.comet.com/opik/api/
+if ( ! defined( 'PRAUTOBLOGGER_OPIK_API_KEY' ) ) {
+	define( 'PRAUTOBLOGGER_OPIK_API_KEY', '' );
+}
+if ( ! defined( 'PRAUTOBLOGGER_OPIK_WORKSPACE' ) ) {
+	define( 'PRAUTOBLOGGER_OPIK_WORKSPACE', 'peptiderepo' );
+}
+if ( ! defined( 'PRAUTOBLOGGER_OPIK_URL_OVERRIDE' ) ) {
+	define( 'PRAUTOBLOGGER_OPIK_URL_OVERRIDE', 'https://www.comet.com/opik/api/' );
+}
+
 /*
 |--------------------------------------------------------------------------
 | Autoloader