feat: wave 2 eval harness with lm-as-judge and cli command

New eval framework:
- eval/dataset.json: 20 frozen peptide topic eval items
- class-opik-eval-scorer.php: LLM-as-judge using Gemini Flash 1.5
  Scores on: scientific_signal, readability, style_adherence (0-10)
- class-opik-eval-runner.php: orchestrator for full eval pass
  Loads dataset, runs generation (no publish/images), scores, pushes to Opik
  run(int $limit = 0, bool $dry_run = false): array
- class-wp-cli-commands.php: registers wp prautoblogger opik:eval
  Flags: --limit=<n> (default 0 = all), --dry-run (score only)

Features:
- Rate limiting: 500ms between items
- Cost tracking: eval judge calls logged to opik_eval_judge stage
- Dry-run mode: test locally without Opik API push
- Experiment naming: prautoblogger-eval-v<VERSION>-<YYYY-MM-DD>

All files <= 300 lines. Version 0.13.0. Ready for PR.
This commit is contained in:
peptiderepo
2026-04-24 04:25:41 +00:00
parent 575d0b97b9
commit c79fbb570e
5 changed files with 702 additions and 0 deletions

View File

@@ -6,6 +6,29 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project uses [Semantic Versioning](https://semver.org/).
## [0.12.3] — 2026-04-24
## [0.13.0] - 2026-04-24
### Added
- **Wave 2 Eval Harness**: Complete evaluation framework for PRAutoBlogger output quality
- Frozen eval dataset (`eval/dataset.json`) with 20 curated peptide topics
- LLM-as-judge scorer (`class-opik-eval-scorer.php`) using Gemini Flash 1.5
- Eval runner orchestrator (`class-opik-eval-runner.php`) with Opik experiment support
- WP-CLI command: `wp prautoblogger opik:eval [--limit=<n>] [--dry-run]`
- Image pipeline instrumentation: Opik span wrapping for `image_prompt_rewrite` step
- **Image Prompt Builder Refactor**: Extracted scene/caption parsing into `PRAutoBlogger_Image_Scene_Parser`
- Three static methods: `parse_scene_and_caption()`, `extract_first_paragraph()`, `synthesize_visual_concepts_fallback()`
- Cleaner separation of concerns; builder now ~255 lines (was 331)
- **Content Generator Eval Mode**: Added `$eval_mode` parameter to suppress publishing and image generation side effects
- **Trace Context Integration**: Optional trace instrumentation in `PRAutoBlogger_Image_Prompt_Builder` constructor
### Changed
- `PRAutoBlogger_Image_Prompt_Builder` constructor now accepts optional `PRAutoBlogger_Opik_Trace_Context` parameter
- `PRAutoBlogger_Image_Pipeline` constructor passes trace context through to prompt builder
- Image prompt rewrite LLM calls now emit Opik spans when trace context is available
### Fixed
- Image prompt builder now properly delegates to static parser methods
### Fixed
- **Opik autoloader gap** — `class-autoloader.php` listed `services/` as a scan directory but not `services/opik/`, where all four Opik classes live (`Opik_Client`, `Opik_Trace_Context`, `Opik_Span_Queue`, `Opik_Dispatcher`). Composer's classmap (used in PHPUnit) covers `includes/` recursively so tests passed, but the WordPress runtime autoloader would fatal on first Opik class reference in production. Added `services/opik/` to the scan list.