feat: Opik LLM observability tracing skeleton (Wave 1) — v0.12.0

Wires PRAutoblogger into Comet Opik cloud for per-call tracing.

- Thin PHP REST client (class-opik-client.php, 276 LOC)
- Async fire-and-forget dispatch via wp_schedule_single_event
- Feature flag PRAUTOBLOGGER_OPIK_ENABLED defaults OFF
- One trace per article, 5 spans across content-generator + chief-editor
- Exponential backoff 2s/4s/8s, max 3 retries, drops on failure
- Admin settings page (api-key configured indicator only, never rendered)
- ARCHITECTURE.md + CONVENTIONS.md updated

PR #113 — QA gate: PASS (qa-reviews/prautoblogger/2026-04-23-0d8dc67.md)
This commit is contained in:
peptiderepo
2026-04-23 22:36:24 +07:00
committed by GitHub
parent 7401addbd9
commit fbbb387311
17 changed files with 2453 additions and 99 deletions

View File

@@ -480,6 +480,50 @@ Stored on every PRAutoBlogger-generated post:
---
## Opik LLM Observability (Optional)
**Status:** Wave 1 (v0.12.0) — tracing skeleton. Wave 2 (future) — eval harness with frozen-dataset regression testing.
Opik is a cloud-hosted LLM observability platform (Comet) providing:
- **Per-call tracing:** Every OpenRouter call generates a span with model, tokens, latency metadata.
- **Trace grouping:** One trace per article generation bundles all spans (draft, QA, image prompt).
- **Prompt versioning:** Foundation for Wave 2 — compare new prompt revisions against a frozen reference dataset using LLM-as-judge scoring on rubric axes (factual accuracy, style match, hallucination risk).
**Architecture:**
- **Feature flag:** `prautoblogger_opik_enabled` (WP option, default off). When false, zero network traffic to Opik.
- **Credentials:** API key and workspace from PHP constants `PRAUTOBLOGGER_OPIK_API_KEY`, `PRAUTOBLOGGER_OPIK_WORKSPACE` (defined in wp-config.php). Never stored in DB.
- **Span lifecycle:**
1. `Opik_Trace_Context::init_trace()` generates UUID at article-worker start.
2. Each LLM call wraps with `start_span()` → LLM request → `end_span()` with tokens + response.
3. `finalize_trace()` + `Opik_Span_Queue::enqueue()` accumulates trace + spans for async dispatch.
4. `prautoblogger_opik_dispatch` cron action batches spans/traces (max 100 per POST) to `/v1/private/spans/batch`, `/v1/private/traces/batch`.
- **Retry logic:** Exponential backoff (2s, 4s, 8s) on 5xx errors; permanent abort on 4xx; dropped after 3 attempts per item.
- **Queue:** WP options-backed; max 1000 items; 12-hour TTL; persists across page reloads for truly async dispatch.
**Call sites instrumented (Wave 1):**
- Draft generation (single-pass or multi-step outline/draft/polish).
- Editorial review (chief editor LLM call).
- Image prompt generation (deferred; file at 314 LOC, over limit).
**Cost & Performance:**
- Opik free tier: 25k spans/month. Estimated current usage ~4.2k/month (20 posts/day × 7 calls × 30 days). Headroom to 5× growth.
- Impact on article generation: async dispatch, zero blocking latency overhead.
- Wave 2 (eval harness): adds LLM-as-judge calls (~$14/month at Haiku pricing, assuming 2 eval runs/week).
**Code structure:**
```
includes/services/opik/
class-opik-client.php — REST client (auth, retry, batch POSTs)
class-opik-trace-context.php — per-request trace state singleton
class-opik-span-queue.php — WP options-backed async queue
class-opik-dispatcher.php — cron handler, batch dispatch
includes/admin/
class-opik-settings.php — admin UI (toggle, project name, status)
```
---
## Key Decisions
### #1: OpenRouter over direct provider APIs