33d3ba4ea1
* feat(cost): persist run_id on generated posts + back-populate for v0.8.1 reaper
The orphan-research-row reaper (upcoming commit) needs a cheap way to
answer "which posts belong to this run_id?" when amortizing a stranded
`llm_research` row back to its siblings. Relying on the generation_log
post_id column is circular: if the pipeline died before amortize_research
ran, we'd trust the same table we're trying to repair.
- `Publisher::build_meta()` now accepts `?string $run_id` and writes
`_prautoblogger_run_id => $run_id` when non-empty. Null/empty preserves
existing null-safe legacy behaviour.
- `Publisher::create_post()` forwards the $run_id it already receives
into build_meta — one-line plumbing, no new parameters at call sites.
- `Activator::backpopulate_run_id_meta_v081()` runs once per site,
gated by `prautoblogger_migrated_run_id_meta_v081` option. Walks
`wp_prautoblogger_generation_log`, pairs (post_id, min(run_id)),
writes `_prautoblogger_run_id` on any post that doesn't already have
it. Uses `metadata_exists()` so it's a no-op on re-run.
- Activator also schedules the new daily `prautoblogger_reap_orphan_research_rows`
cron at 03:15 — 15 min after the primary generation cron — so live
pipelines have time to finish amortizing cleanly before the reaper scans.
- Deactivator unschedules the new hook alongside the existing cron hooks.
No user-visible behaviour change yet. The reaper handler ships in the
next commit; this one just ensures it will have the data + schedule it
needs when it lands.
Part of the v0.8.1 bundle (thread research-cost-reaper).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(cost): daily reaper for orphan llm_research rows
Implements the orphan-research-row reaper per thread research-cost-reaper
seq 1. When a pipeline dies between post creation and
`Post_Assembler::amortize_research_costs()` (Hostinger exec kill, OOM,
LiteSpeed cut, fatal), the shared research row stays in
`wp_prautoblogger_generation_log` with `post_id = NULL` and sibling
articles' cost popovers understate their true spend.
New class `PRAutoBlogger_Research_Reaper` owns:
- `on_cron()` — WP-Cron entrypoint hooked to the daily
`prautoblogger_reap_orphan_research_rows` event (already scheduled
by the Activator in the previous commit at 03:15).
- `reap()` — returns a `{reaped, deleted, skipped}` summary. For each
orphan `llm_research` row older than 1 hour (grace window for live
pipelines):
- `_prautoblogger_run_id` post_meta primary lookup.
- Falls back to distinct post_ids in gen_log with the same run_id.
- If ≥1 post found → delegate to the existing idempotent
`Post_Assembler::amortize_research_costs()`. That method inserts
the per-article rows and deletes the original orphan in one pass,
so the reaper is idempotent on re-run.
- If 0 posts found AND orphan is >7 days old → delete outright as
sunk cost with nothing to attribute to.
- Else → skip; try again the next day.
- Never throws. DB errors caught and logged WARNING.
New WP-CLI command `wp prautoblogger reap-research` registered in the
main loader when `WP_CLI` is defined — manual ops trigger that prints
the same summary.
Departure from the CTO's spec: the reap logic lives on
`Research_Reaper` instead of `Post_Assembler`. Placing it on
Post_Assembler pushed that file to 391 lines, over the 300-line cap.
Moving it to the thin class colocates the attribution logic with the
cron/CLI plumbing that triggers it. `amortize_research_costs()` stays
on Post_Assembler — the reaper delegates to it unchanged. Flagged for
CTO review.
Part of the v0.8.1 bundle (thread research-cost-reaper).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(cost): ResearchReaperTest — five cases per spec
Covers the five scenarios from convo thread research-cost-reaper seq 1:
- test_reaps_orphan_with_matching_run_id_posts — meta lookup returns
two post IDs; reaper delegates to amortize_research_costs which
inserts two rows and deletes the orphan.
- test_skips_orphan_within_grace_period — the SELECT's
`created_at < now - 1h` clause filters fresh orphans at the DB; the
reaper's stats stay zero and no side effects fire.
- test_deletes_orphan_with_no_articles_after_7_days — orphan is 8 days
old, zero siblings via meta or gen_log; reaper deletes outright.
- test_falls_back_to_gen_log_when_post_meta_missing — meta returns
empty (legacy pre-v0.8.1 post); gen_log has two distinct post_ids;
amortize still runs.
- test_idempotent_on_repeat_run — second pass on a cleaned state
returns 0/0/0 with no inserts or deletes.
All DB interaction mocked via $wpdb stub — no real DB round-trips.
Fixtures are driven by property mutations in the test methods so
PHPUnit's "one stub per method" restriction on mocks doesn't bite.
Part of the v0.8.1 bundle.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(v0.8.1): version bump + CHANGELOG + ARCHITECTURE.md for reaper
Bump plugin header and PRAUTOBLOGGER_VERSION 0.8.0 → 0.8.1. Patch
bump — the reaper is internal infrastructure; no user-visible UX
change.
CHANGELOG:
- Fixed: orphan `llm_research` rows now reaped daily.
- Added: WP-CLI `wp prautoblogger reap-research`; `_prautoblogger_run_id`
post_meta is now written by Publisher on every post.
ARCHITECTURE.md:
- Decision #13 (Run-ID based log linking) cross-refs the new post_meta.
- New Decision #19 documents the reaper's scan/amortize/delete policy,
the 1-hour grace window, the 7-day stale cutoff, and the one-shot
activator back-populate migration.
DB schema unchanged — no PRAUTOBLOGGER_DB_VERSION bump.
Final commit of the v0.8.1 bundle.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(cost): widen $wpdb mock so ResearchReaperTest actually runs
BaseTestCase::create_mock_wpdb() only adds methods
[prepare, get_var, get_results, insert, query, get_row, update] — the
reaper and amortize_research_costs also call `delete` and `get_col`,
which the default mock can't stub. CI on PR #100 fired all five new
ResearchReaperTest cases as errors at the first `->method('delete')`
call during setUp.
Fix: build a bespoke $wpdb mock in this file's setUp with the full
superset of methods it needs. Self-contained; no change to
BaseTestCase so existing tests aren't affected.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(cost): filter gen_log inserts from event_log inserts in reaper tests
CI on ec1efee showed the two reap-and-amortize tests failing because
`assertCount(2, $this->inserted_rows)` was seeing 4 rows, not 2. Root
cause: the Logger singleton writes INFO lines via `$wpdb->insert` into
the event_log table — the reaper logs once ("Reaped...") and amortize
logs once ("Amortized LLM research cost...") before the two per-post
gen_log inserts, so the capture ended up with 4 total rows.
Add a `gen_log_inserts()` helper that filters the capture down to rows
whose `__table` contains "generation_log" — the assertions on
attribution count should only measure the actual cost inserts, not
incidental log writes. Two tests updated; logic unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(admin): correct misleading GA4 Property ID description
The GA4 client at includes/core/class-ga4-client.php:85 already wraps the
property_id option with "properties/" in the outbound URL path. The admin
description told users to enter "properties/XXXXXXXXX" — which would have
produced a double-prefixed URL (properties/properties/XXXXXXXXX) and a
guaranteed 404 from the Data API. Description now asks for digits only,
matching actual code behavior.
No code logic change — UX copy + CHANGELOG only. Flagged by CEO while
inspecting the Analytics tab; folded into the in-flight v0.8.1 PR per
CTO call rather than cutting a separate patch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: peptiderepo <peptiderepo@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
556 lines
44 KiB
Markdown
Executable File
556 lines
44 KiB
Markdown
Executable File
# PRAutoBlogger — Architecture
|
||
|
||
> **Cross-app context:** decisions that affect multiple plugins (Cloudflare AI Gateway routing, OpenRouter account sharing, the interface pattern, image-generation stack, social distributor choice) are recorded in `Peptide Repo CTO/docs/engineering/decisions/`. The incident runbook for cross-app failure modes is at `Peptide Repo CTO/docs/engineering/INCIDENT-RUNBOOK.md`. Engineer PRAutoBlogger should read both before making decisions that cross plugin boundaries.
|
||
|
||
PRAutoBlogger is a WordPress plugin that monitors social media (starting with Reddit) for recurring questions, complaints, and comparisons in a configured niche, uses LLM agents (via OpenRouter) to generate high-quality blog articles from those insights, runs them through a chief editor agent for quality review, and auto-publishes them on a configurable daily schedule. All collected data and generation metrics are stored for a self-improvement feedback loop.
|
||
|
||
---
|
||
|
||
## Data Flow
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────────────────────┐
|
||
│ WP-Cron / Action Scheduler │
|
||
│ (Daily trigger, configurable) │
|
||
└────────────┬────────────────────────────────────────────────────────────┘
|
||
│
|
||
▼
|
||
┌─────────────────────────┐
|
||
│ 1. Source Collector │ Pulls data from enabled sources:
|
||
│ (Reddit + LLM Research)│ - Reddit RSS/Atom (primary) / .json (fallback)
|
||
│ │ - LLM Deep Research (reasoning model)
|
||
│ │ Stores raw data in `ab_source_data` table
|
||
└────────────┬────────────┘
|
||
│
|
||
▼
|
||
┌─────────────────────────┐
|
||
│ 2. Content Analyzer │ LLM call (cheap model) to detect:
|
||
│ (Analysis Agent) │ - Recurring questions
|
||
│ │ - Complaints / pain points
|
||
│ │ - Product comparisons
|
||
│ │ Stores analysis in `ab_analysis_results` table
|
||
│ │ Outputs ranked article ideas
|
||
└────────────┬────────────┘
|
||
│
|
||
▼
|
||
┌─────────────────────────┐
|
||
│ 3. Idea Scorer & │ Deduplicates against existing posts
|
||
│ Deduplicator │ Scores ideas by relevance, frequency, freshness
|
||
│ │ Picks top N ideas (N = daily article target)
|
||
└────────────┬────────────┘
|
||
│
|
||
▼
|
||
┌─────────────────────────┐
|
||
│ 4. Writer Agent │ Configurable pipeline:
|
||
│ (Content Generator) │ Single-pass: one LLM call → complete article
|
||
│ │ Multi-step: outline → draft → polish
|
||
│ │ Uses quality model via OpenRouter
|
||
└────────────┬────────────┘
|
||
│
|
||
▼
|
||
┌─────────────────────────┐
|
||
│ 5. Chief Editor Agent │ LLM-powered editorial review:
|
||
│ │ - Checks quality, accuracy, tone
|
||
│ │ - Checks SEO (title, headings, keyword density)
|
||
│ │ - Can request rewrites or approve
|
||
│ │ - Approved → publish; Rejected → flag for human
|
||
└────────────┬────────────┘
|
||
│
|
||
▼
|
||
┌─────────────────────────┐
|
||
│ 6. Publisher │ Creates/updates WordPress post
|
||
│ │ Sets categories, tags, featured image
|
||
│ │ Stores generation metadata in post_meta
|
||
│ │ Logs cost/tokens in `prab_generation_log` table
|
||
└────────────┬────────────┘
|
||
│
|
||
▼
|
||
┌─────────────────────────┐
|
||
│ 6b. Image Pipeline │ Generates two A/B images in parallel:
|
||
│ (optional, if enabled) │ - Image A: article-driven (featured)
|
||
│ │ - Image B: source-driven (post meta)
|
||
│ │ Provider derived from the saved Image Model
|
||
│ │ via Image_Model_Registry::provider_for().
|
||
│ │ NSFW-blocked slots (CF error code 3030) are
|
||
│ │ retried once with a rule-based fallback
|
||
│ │ prompt via Image_NSFW_Retry; a second block
|
||
│ │ logs WARNING and publishes without that image.
|
||
│ │ Sideloads to media library, logs costs.
|
||
└────────────┬────────────┘
|
||
│
|
||
▼
|
||
┌─────────────────────────┐
|
||
│ 7. Metrics Collector │ Tracks post-publish performance:
|
||
│ (Separate cron job) │ - WordPress native (views, comments)
|
||
│ │ - GA4 (pageviews, bounce rate, time on page)
|
||
│ │ - Composite "content score"
|
||
│ │ Feeds back into Analysis step for self-improvement
|
||
└─────────────────────────┘
|
||
```
|
||
|
||
---
|
||
|
||
## File Tree
|
||
|
||
```
|
||
prautoblogger/
|
||
├── prautoblogger.php # Plugin bootstrap — minimal, only hook registration
|
||
├── uninstall.php # Clean removal of ALL plugin data
|
||
├── readme.txt # WordPress.org standard readme
|
||
├── composer.json # Autoloading (PSR-4) and dependencies
|
||
├── ARCHITECTURE.md # This file
|
||
├── CONVENTIONS.md # Naming patterns, extension guides
|
||
├── CHANGELOG.md # Semantic versioning changelog
|
||
│
|
||
├── assets/
|
||
│ ├── css/
|
||
│ │ ├── admin.css # Admin page styles (wp-admin conventions)
|
||
│ │ └── posts-widget.css # Frontend posts widget styles (uses theme CSS vars)
|
||
│ └── js/
|
||
│ ├── admin.js # Admin page interactivity (vanilla JS / Alpine.js)
|
||
│ └── posts-widget.js # React component for frontend post cards (wp.element)
|
||
│
|
||
├── includes/
|
||
│ ├── class-prautoblogger.php # Main orchestrator — registers all hooks, delegates execution
|
||
│ ├── class-executor.php # Cron handlers, generation AJAX (start + status), model registry
|
||
│ ├── class-ajax-handlers.php # Non-generation AJAX: images, models, test connections
|
||
│ ├── class-generation-lock.php # DB-level atomic mutex for single-writer generation
|
||
│ ├── class-activator.php # Activation: create DB tables, set defaults, schedule cron
|
||
│ ├── class-deactivator.php # Deactivation: clear cron, cleanup transients
|
||
│ ├── class-autoloader.php # PSR-4-style autoloader for plugin classes
|
||
│ │
|
||
│ ├── admin/
|
||
│ │ ├── class-admin-page.php # Main settings page (tabbed SaaS-style UI)
|
||
│ │ ├── class-settings-fields.php # Declarative settings: sections + core fields (API/Models/Content/Sources)
|
||
│ │ ├── class-settings-fields-extended.php # Operational fields: schedule, publishing, display, analytics, images
|
||
│ │ ├── class-image-model-registry.php # Static list of image generation models for the model picker
|
||
│ │ ├── class-admin-notices.php # Onboarding notices, error alerts, budget warnings
|
||
│ │ ├── class-dashboard-widget.php # WP Dashboard widget showing generation status
|
||
│ │ ├── class-post-metabox.php # Metabox on posts showing generation metadata
|
||
│ │ ├── class-metrics-page.php # Admin page for cost dashboard and content scores
|
||
│ │ ├── class-ideas-browser.php # Browse all analysis results (article ideas)
|
||
│ │ ├── class-review-queue.php # Approve/edit/reject queue for generated drafts
|
||
│ │ └── class-log-viewer.php # Activity log viewer with level filtering
|
||
│ │
|
||
│ ├── core/
|
||
│ │ ├── class-scheduler.php # WP-Cron / Action Scheduler job management
|
||
│ │ ├── class-source-collector.php # Orchestrates data collection from all sources
|
||
│ │ ├── class-content-analyzer.php # LLM-powered analysis of collected social data
|
||
│ │ ├── class-analysis-prompts.php # System/user prompt builders + performance context for analyzer
|
||
│ │ ├── class-idea-scorer.php # Ranks and deduplicates article ideas (delegates to Semantic_Dedup)
|
||
│ │ ├── class-semantic-dedup.php # Embedding-based cosine similarity dedup with keyword fallback
|
||
│ │ ├── class-content-generator.php# Writer agent — manages the generation pipeline
|
||
│ │ ├── class-content-prompts.php # Builds all LLM prompts for content generation (system, stage, linking rules)
|
||
│ │ ├── class-peptide-linker.php # Deterministic post-processor: hyperlinks peptide mentions to /peptides/ pages
|
||
│ │ ├── class-chief-editor.php # Editor agent — LLM-powered editorial review
|
||
│ │ ├── class-publisher.php # Creates WordPress posts from approved content
|
||
│ │ ├── class-post-assembler.php # Post-creation helpers: taxonomy, log linking, images, sanitization
|
||
│ │ ├── class-image-pipeline.php # Orchestrates A/B image generation (parallel via batch)
|
||
│ │ ├── class-image-prompt-builder.php # Generates visual prompts from article/source data
|
||
│ │ ├── class-image-media-sideloader.php # Imports images into WordPress media library
|
||
│ │ ├── class-cost-tracker.php # Logs all API costs, enforces budget limits
|
||
│ │ ├── class-logger.php # Structured logging singleton (error/warning/info/debug)
|
||
│ │ ├── class-article-worker.php # Single-article generation (content + edit + publish)
|
||
│ │ ├── class-pipeline-runner.php # Orchestrates pipeline; chains per-article cron jobs
|
||
│ │ ├── class-ga4-client.php # Google Analytics 4 API client (OAuth + Data API)
|
||
│ │ └── class-metrics-collector.php# Collects post performance data (WP + GA4)
|
||
│ │
|
||
│ ├── providers/
|
||
│ │ ├── interface-llm-provider.php # Contract for any LLM provider
|
||
│ │ ├── class-open-router-provider.php # OpenRouter API implementation (chat completions)
|
||
│ │ ├── class-open-router-embedding-provider.php # Text embeddings for semantic dedup
|
||
│ │ ├── class-open-router-pricing.php # Model pricing lookup and cost estimation
|
||
│ │ ├── interface-source-provider.php # Contract for any social media source
|
||
│ │ ├── class-reddit-json-client.php # Reddit HTTP client — RSS (primary) + .json (fallback)
|
||
│ │ ├── class-llm-research-provider.php # LLM deep research source (reasoning models)
|
||
│ │ ├── class-reddit-provider.php # Reddit data collection orchestrator (RSS primary)
|
||
│ │ ├── interface-image-provider.php # Contract for any image generation provider (incl. batch)
|
||
│ │ ├── class-open-router-image-provider.php # OpenRouter image gen (single + batch dispatch)
|
||
│ │ ├── class-open-router-image-batch.php # Parallel curl_multi execution for batch image gen
|
||
│ │ ├── class-open-router-image-support.php # API key, response parsing, retry/backoff helpers
|
||
│ │ ├── class-open-router-image-pricing.php # Model resolution + per-image cost estimation
|
||
│ │ ├── class-open-router-config.php # API base URL (direct vs AI Gateway)
|
||
│ │ ├── class-open-router-request-builder.php # Header building + Hostinger cURL auth workaround
|
||
│ │ ├── class-cloudflare-image-provider.php # Cloudflare Workers AI (sequential batch fallback)
|
||
│ │ ├── class-cloudflare-image-pricing.php # Model alias + per-MP cost estimation
|
||
│ │ ├── class-cloudflare-image-validator.php # Non-destructive credential + connectivity check
|
||
│ │ ├── class-cloudflare-image-support.php # Token, account, URL, and log helpers for CF provider
|
||
│ │ └── (new providers go here — see CONVENTIONS.md)
|
||
│ │
|
||
│ ├── services/
|
||
│ │ ├── interface-model-registry.php # Contract for provider-specific model registries
|
||
│ │ ├── class-open-router-model-registry.php # Fetches, caches, queries the OpenRouter model list
|
||
│ │ ├── class-open-router-model-normalizer.php # Maps raw OpenRouter data → standardized record shape
|
||
│ │ └── (Phase 3: class-cloudflare-workers-ai-model-registry.php)
|
||
│ │
|
||
│ ├── frontend/
|
||
│ │ ├── class-article-typography.php # Inline CSS for font, size, table borders on generated posts
|
||
│ │ └── class-posts-widget.php # [prautoblogger_posts] shortcode + REST endpoint
|
||
│ │
|
||
│ └── models/
|
||
│ ├── class-source-data.php # Value object: raw social media post/comment
|
||
│ ├── class-analysis-result.php # Value object: analyzed topic with scores
|
||
│ ├── class-article-idea.php # Value object: scored article idea
|
||
│ ├── class-content-request.php # Value object: generation request with params
|
||
│ ├── class-editorial-review.php # Value object: editor verdict, scores, revised content
|
||
│ ├── class-generation-log.php # Value object: log entry for a generation run
|
||
│ └── class-content-score.php # Value object: composite performance score
|
||
│
|
||
├── templates/
|
||
│ └── admin/
|
||
│ ├── settings-page.php # Settings page template (tabbed sidebar layout)
|
||
│ ├── metrics-page.php # Metrics/cost dashboard template
|
||
│ ├── review-queue.php # Review queue table template
|
||
│ ├── log-viewer.php # Activity log viewer template
|
||
│ └── metabox-generation-info.php# Post metabox template
|
||
│
|
||
├── languages/
|
||
│ └── prautoblogger.pot # i18n translation template
|
||
│
|
||
└── tests/
|
||
├── unit/ # PHPUnit tests (mocked dependencies)
|
||
└── integration/ # WordPress integration tests
|
||
```
|
||
|
||
---
|
||
|
||
## Frontend Widget Data Flow
|
||
|
||
```
|
||
┌──────────────────────────────┐
|
||
│ WordPress Page/Post │ Contains [prautoblogger_posts] shortcode
|
||
│ (e.g. Home page) │
|
||
└────────────┬─────────────────┘
|
||
│ Shortcode renders:
|
||
│ 1. <div id="prab-posts-root"></div>
|
||
│ 2. Enqueues posts-widget.js + posts-widget.css
|
||
│ 3. Passes config via wp_localize_script
|
||
▼
|
||
┌──────────────────────────────┐
|
||
│ React Component (wp.element)│ Mounts into #prab-posts-root
|
||
│ posts-widget.js │ Shows loading skeleton on mount
|
||
└────────────┬─────────────────┘
|
||
│ Fetches asynchronously
|
||
▼
|
||
┌──────────────────────────────┐
|
||
│ REST API Endpoint │ GET /wp-json/prautoblogger/v1/posts
|
||
│ class-posts-widget.php │ Queries published posts with
|
||
│ │ _prautoblogger_generated = '1' meta
|
||
│ │ Returns slim JSON (id, title, excerpt,
|
||
│ │ url, date, category, image, word_count)
|
||
│ │ 5-minute Cache-Control header
|
||
└──────────────────────────────┘
|
||
```
|
||
|
||
**Shortcode Attributes**
|
||
|
||
| Attribute | Default | Description |
|
||
|------------|----------------------------------|-------------------------|
|
||
| `count` | `6` | Posts to display (max 12) |
|
||
| `category` | (all) | Filter by category slug |
|
||
| `title` | `"Latest Research & Insights"` | Widget heading |
|
||
| `subtitle` | `"Evidence-based articles..."` | Widget subheading |
|
||
|
||
**CSS Integration**
|
||
|
||
The widget CSS (`posts-widget.css`) uses the Peptide Starter theme's CSS custom properties (`--color-*`, `--spacing-*`, `--text-*`, `--radius-*`, `--transition-*`) with hardcoded fallbacks. This means:
|
||
|
||
- **On sites using Peptide Starter:** widget automatically adapts to light/dark mode via `data-theme`.
|
||
- **On other themes:** fallback values render a sensible dark-themed design.
|
||
|
||
---
|
||
|
||
## Background Generation Flow (v0.3.0)
|
||
|
||
The "Generate Now" button runs the pipeline in a background WP-Cron process to avoid
|
||
Hostinger's 120-second LiteSpeed connection timeout. The frontend polls for status.
|
||
|
||
```
|
||
┌──────────────────────────────┐
|
||
│ Admin clicks "Generate Now" │ admin.js sends AJAX to on_ajax_generate_now()
|
||
└────────────┬─────────────────┘
|
||
│ Handler returns immediately (< 1 second):
|
||
│ 1. Writes "running" transient with started timestamp
|
||
│ 2. Schedules one-shot cron: prautoblogger_manual_generation
|
||
│ 3. Fires non-blocking loopback to wp-cron.php
|
||
│ 4. Returns JSON success to browser
|
||
▼
|
||
┌──────────────────────────────┐
|
||
│ admin.js starts polling │ setInterval every 5 seconds
|
||
│ on_ajax_generation_status() │ Shows stage text + spinner
|
||
└────────────┬─────────────────┘
|
||
│
|
||
▼ (separate PHP process)
|
||
┌──────────────────────────────┐
|
||
│ on_manual_generation() │ WP-Cron fires this in a new request
|
||
│ (class-executor.php) │ 1. ignore_user_abort(true) — survives HTTP kill
|
||
│ │ 2. set_time_limit(300)
|
||
│ │ 3. Acquires PRAutoBlogger_Generation_Lock
|
||
│ │ 4. Runs PRAutoBlogger_Pipeline_Runner::run()
|
||
│ │ (pipeline broadcasts stage updates to transient)
|
||
│ │ 5. Writes final result to transient
|
||
│ │ 6. Releases lock
|
||
└────────────┬─────────────────┘
|
||
│
|
||
▼
|
||
┌──────────────────────────────┐
|
||
│ Poller detects completion │ Reads transient: status = 'complete' or 'error'
|
||
│ Shows result message │ Resets button to idle state
|
||
└──────────────────────────────┘
|
||
```
|
||
|
||
**Fallback detection:** If Hostinger kills the PHP process before it updates the
|
||
transient (despite `ignore_user_abort`), the status endpoint detects the orphan after
|
||
180 seconds by checking for recently-created posts with `_prautoblogger_run_id` meta.
|
||
It also releases any stale generation lock. After 600 seconds (STATUS_TTL), it gives
|
||
up unconditionally and reports a timeout.
|
||
|
||
---
|
||
|
||
## Database Schema
|
||
|
||
### Custom Tables
|
||
|
||
All tables use `$wpdb->prefix` + `prautoblogger_` prefix.
|
||
|
||
#### `ab_source_data` — Raw social media data
|
||
|
||
| Column | Type | Description |
|
||
|---------------|-------------------|--------------------------------------------------|
|
||
| id | BIGINT UNSIGNED | Auto-increment PK |
|
||
| source_type | VARCHAR(50) | 'reddit' (extensible for future providers) |
|
||
| source_id | VARCHAR(255) | Platform-specific unique ID (post/comment ID) |
|
||
| subreddit | VARCHAR(255) | Subreddit or channel name (nullable) |
|
||
| title | TEXT | Post/video title |
|
||
| content | LONGTEXT | Post body / comment text / transcript |
|
||
| author | VARCHAR(255) | Original author username |
|
||
| score | INT | Upvotes/likes/engagement metric |
|
||
| comment_count | INT | Number of comments/replies |
|
||
| permalink | VARCHAR(500) | URL to original content |
|
||
| collected_at | DATETIME | When we collected this data |
|
||
| metadata_json | LONGTEXT | Platform-specific extra data (JSON) |
|
||
|
||
- INDEX: `source_type`, `collected_at`
|
||
- UNIQUE INDEX: `source_type`, `source_id`
|
||
|
||
#### `ab_analysis_results` — Analyzed topics and patterns
|
||
|
||
| Column | Type | Description |
|
||
|------------------|-------------------|------------------------------------------------|
|
||
| id | BIGINT UNSIGNED | Auto-increment PK |
|
||
| analysis_type | VARCHAR(50) | 'question', 'complaint', 'comparison' |
|
||
| topic | VARCHAR(500) | Detected topic/theme |
|
||
| summary | TEXT | LLM-generated summary of the pattern |
|
||
| frequency | INT | How many source posts mention this |
|
||
| relevance_score | FLOAT | LLM-assigned relevance score (0-1) |
|
||
| source_ids_json | LONGTEXT | JSON array of ab_source_data.id references |
|
||
| analyzed_at | DATETIME | When analysis was performed |
|
||
| metadata_json | LONGTEXT | Extra analysis data (JSON) |
|
||
|
||
- INDEX: `analysis_type`, `relevance_score`
|
||
- INDEX: `analyzed_at`
|
||
|
||
#### `prab_generation_log` — API cost and generation tracking
|
||
|
||
| Column | Type | Description |
|
||
|-------------------|-------------------|-----------------------------------------------|
|
||
| id | BIGINT UNSIGNED | Auto-increment PK |
|
||
| post_id | BIGINT UNSIGNED | WordPress post ID (nullable, set after publish)|
|
||
| run_id | VARCHAR(36) | UUID linking all entries from one pipeline run |
|
||
| stage | VARCHAR(50) | 'analysis', 'outline', 'draft', 'edit', 'review' |
|
||
| provider | VARCHAR(50) | 'openrouter' |
|
||
| model | VARCHAR(100) | Model identifier used |
|
||
| prompt_tokens | INT | Input tokens consumed |
|
||
| completion_tokens | INT | Output tokens generated |
|
||
| estimated_cost | DECIMAL(10,6) | Estimated USD cost |
|
||
| request_json | LONGTEXT | Full request payload (for debugging) |
|
||
| response_status | VARCHAR(20) | 'success', 'error', 'timeout' |
|
||
| error_message | TEXT | Error details if failed (nullable) |
|
||
| created_at | DATETIME | When the API call was made |
|
||
|
||
- INDEX: `post_id`
|
||
- INDEX: `run_id`
|
||
- INDEX: `created_at`
|
||
- INDEX: `stage`
|
||
|
||
#### `prab_event_log` — Structured application logging
|
||
|
||
| Column | Type | Description |
|
||
|------------|-------------------|-------------------------------------------------|
|
||
| id | BIGINT UNSIGNED | Auto-increment PK |
|
||
| level | VARCHAR(10) | 'error', 'warning', 'info', 'debug' |
|
||
| context | VARCHAR(100) | Origin: 'pipeline', 'reddit', 'ga4', etc. |
|
||
| message | TEXT | Human-readable log message |
|
||
| meta_json | LONGTEXT | Optional structured data (nullable) |
|
||
| created_at | DATETIME | When the event occurred |
|
||
|
||
- INDEX: `level`, `created_at`
|
||
- INDEX: `created_at`
|
||
|
||
#### `ab_content_scores` — Post performance metrics
|
||
|
||
| Column | Type | Description |
|
||
|--------------------|-------------------|----------------------------------------------|
|
||
| id | BIGINT UNSIGNED | Auto-increment PK |
|
||
| post_id | BIGINT UNSIGNED | WordPress post ID |
|
||
| pageviews | INT | Total pageviews (WP native or GA4) |
|
||
| avg_time_on_page | FLOAT | Average seconds on page (GA4) |
|
||
| bounce_rate | FLOAT | Bounce rate percentage (GA4) |
|
||
| comment_count | INT | WordPress comment count |
|
||
| composite_score | FLOAT | LLM-computed content quality score (0-100) |
|
||
| score_factors_json | LONGTEXT | JSON breakdown of what contributed to score |
|
||
| measured_at | DATETIME | When metrics were collected |
|
||
|
||
- INDEX: `post_id`
|
||
- INDEX: `measured_at`
|
||
- INDEX: `composite_score`
|
||
|
||
### WordPress Options (`wp_options`)
|
||
|
||
All prefixed with `prautoblogger_`:
|
||
|
||
| Option Key | Description |
|
||
|----------------------------------------|-------------------------------------------------------|
|
||
| `prautoblogger_openrouter_api_key` | Encrypted OpenRouter API key |
|
||
| `prautoblogger_ai_gateway_base_url` | Optional Cloudflare AI Gateway URL (proxies OpenRouter); empty = direct |
|
||
| `prautoblogger_ai_gateway_cache_ttl` | Seconds Cloudflare may cache identical LLM responses (0 = off) |
|
||
| `prautoblogger_ga4_property_id` | Google Analytics 4 property ID |
|
||
| `prautoblogger_ga4_credentials_json` | Encrypted GA4 service account credentials |
|
||
| `prautoblogger_analysis_model` | OpenRouter model for analysis (default: cheap) |
|
||
| `prautoblogger_writing_model` | OpenRouter model for writing (default: quality) |
|
||
| `prautoblogger_editor_model` | OpenRouter model for chief editor (default: quality) |
|
||
| `prautoblogger_daily_article_target` | Number of articles per day (1-10, default: 1) |
|
||
| `prautoblogger_writing_pipeline` | 'single_pass' or 'multi_step' (default: multi_step) |
|
||
| `prautoblogger_niche_description` | Text description of the site's niche |
|
||
| `prautoblogger_target_subreddits` | JSON array of subreddits to monitor |
|
||
| `prautoblogger_monthly_budget_usd` | Monthly API spend limit in USD |
|
||
| `prautoblogger_tone` | Content tone (informational, conversational, etc.) |
|
||
| `prautoblogger_min_word_count` | Minimum article word count (default: 800) |
|
||
| `prautoblogger_max_word_count` | Maximum article word count (default: 2000) |
|
||
| `prautoblogger_topic_exclusions` | JSON array of topics to never write about |
|
||
| `prautoblogger_enabled_sources` | JSON array of active source types |
|
||
| `prautoblogger_auto_publish` | Toggle: auto-publish approved posts (default: '0') |
|
||
| `prautoblogger_default_author` | WP user ID for generated post authorship |
|
||
| `prautoblogger_default_category` | WP category ID fallback for generated posts |
|
||
| `prautoblogger_log_level` | Logging threshold: error/warning/info/debug |
|
||
| `prautoblogger_db_version` | Schema version for migrations |
|
||
| `prautoblogger_schedule_time` | Daily generation time (HH:MM, default: '03:00') |
|
||
| `prautoblogger_cloudflare_ai_token` | Encrypted Cloudflare Workers AI API token |
|
||
| `prautoblogger_cloudflare_account_id` | Cloudflare account UUID (plaintext — identifier, not secret) |
|
||
| `prautoblogger_image_model` | Image model slug from `Image_Model_Registry::get_models()`; provider is derived from this on save |
|
||
| `prautoblogger_image_provider` | Derived from the chosen model on save (v0.8.0+); not editable in admin UI |
|
||
| `prautoblogger_image_prompt_instructions` | System prompt given to the image rewriter LLM (v0.8.0+); falls back to `Image_Prompt_Builder::REWRITER_SYSTEM_PROMPT` when empty |
|
||
| `prautoblogger_image_nsfw_retry` | Toggle: retry NSFW-blocked image slots with a rule-based fallback prompt (default: '1') |
|
||
| `prautoblogger_migrated_image_provider_v080` | One-shot migration flag — auto-heals mismatched provider/model pairs on first admin_init after v0.8.0 upgrade |
|
||
| `prautoblogger_article_font_family` | Font family key: 'default', 'inter', 'georgia', 'merriweather', 'lora', 'open_sans', 'roboto', 'system' |
|
||
| `prautoblogger_article_font_size` | Body font size in px (0 = theme default). Recommended: 16–18. |
|
||
| `prautoblogger_table_borders` | Toggle: add borders/padding/striping to tables (default: '1') |
|
||
| `prautoblogger_image_style_suffix` | Text appended to every image prompt (default: CEO-locked 90s infomercial prompt) |
|
||
| `prautoblogger_openrouter_model_registry` | Normalized OpenRouter model list (JSON array, daily refresh, serves as durable cache) |
|
||
| `prautoblogger_openrouter_model_registry_fetched_at` | Unix timestamp of last successful model registry refresh |
|
||
|
||
### Post Meta
|
||
|
||
Stored on every PRAutoBlogger-generated post:
|
||
|
||
| Meta Key | Description |
|
||
|---------------------------------------|-----------------------------------------------|
|
||
| `_prautoblogger_generated` | Boolean flag — '1' if generated by plugin |
|
||
| `_prautoblogger_analysis_id` | FK to ab_analysis_results.id |
|
||
| `_prautoblogger_source_ids` | JSON array of source data IDs used |
|
||
| `_prautoblogger_model_used` | Model that generated the content |
|
||
| `_prautoblogger_pipeline_mode` | 'single_pass' or 'multi_step' |
|
||
| `_prautoblogger_total_cost` | Total USD cost for this post |
|
||
| `_prautoblogger_total_tokens` | Total tokens consumed for this post |
|
||
| `_prautoblogger_editor_verdict` | 'approved', 'revised', 'rejected' |
|
||
| `_prautoblogger_editor_notes` | Chief editor's review notes |
|
||
| `_prautoblogger_generated_at` | ISO 8601 timestamp of generation |
|
||
| `_prautoblogger_research_sources` | JSON array of source URLs used |
|
||
|
||
---
|
||
|
||
## External API Integrations
|
||
|
||
| Service | Purpose | Auth | Rate Limit | Code |
|
||
|---------|---------|------|------------|------|
|
||
| OpenRouter | All LLM calls (analysis, writing, editing) | API key (encrypted in wp_options) | Per-model | `providers/class-open-router-provider.php`, `providers/class-open-router-pricing.php` |
|
||
| Cloudflare AI Gateway (optional) | Transparent proxy in front of OpenRouter — adds response caching, cost logging, rate limiting, provider fallback | Same OpenRouter key; gateway URL in `prautoblogger_ai_gateway_base_url` | Gateway-side quotas | Same provider file; activated when gateway URL option is non-empty |
|
||
| Reddit RSS | Primary Reddit data source — Atom feeds for subreddit hot posts | None (unauthenticated) | No known rate limit; reliable from datacenter IPs | `providers/class-reddit-json-client.php` |
|
||
| Reddit .json | Fallback for posts + only source for comments | None (unauthenticated) | ~10 req/min (datacenter IPs often blocked) | `providers/class-reddit-json-client.php` |
|
||
| Google Analytics 4 | Post performance metrics | OAuth2 service account | Standard GA4 limits | `core/class-ga4-client.php`, `core/class-metrics-collector.php` |
|
||
| Cloudflare Workers AI | Image generation (FLUX.1 schnell / dev) for article hero, thumbnail, and IG placements | API token (encrypted in wp_options) + account ID | Workers AI per-account quotas | `providers/class-cloudflare-image-provider.php`, `providers/class-cloudflare-image-pricing.php`, `providers/class-cloudflare-image-validator.php` |
|
||
|
||
---
|
||
|
||
## Key Decisions
|
||
|
||
### #1: OpenRouter over direct provider APIs
|
||
Gives access to many models through one API, simplifies provider management, and lets the user switch models without code changes. Trade-off: slight latency overhead and dependency on OpenRouter's availability.
|
||
|
||
### #2: Reddit as first social source (RSS primary, .json fallback)
|
||
Reddit has rich discussion data ideal for identifying recurring questions and pain points. Reddit RSS/Atom feeds are the primary data source — they work reliably from datacenter IPs (Hostinger) where .json endpoints return 403. The .json endpoints serve as a fallback for posts and are the only source for comment fetching (RSS doesn't include comments). Trade-off: RSS lacks engagement metrics (score, comment count), but post titles and content are sufficient for topic analysis.
|
||
|
||
### #3: Chief editor agent instead of human review queue
|
||
The user wants full automation — a second LLM pass reviews quality, SEO, and accuracy before publishing. Posts that fail editorial review are flagged for human intervention rather than published. Trade-off: doubles the LLM cost of the "review" step, but catches quality issues.
|
||
|
||
### #4: Configurable pipeline (single-pass vs. multi-step)
|
||
Some users want cheap/fast, others want high quality. Making this configurable avoids forcing one approach. Trade-off: more code paths to maintain.
|
||
|
||
### #5: Composite content score using LLM
|
||
Rather than just tracking raw metrics, we periodically have the LLM evaluate what made high-performing posts succeed and low-performing posts fail. This feeds back into the analysis and generation prompts. Trade-off: additional API cost for scoring, but enables the self-improvement loop.
|
||
|
||
### #6: Source provider interface for future platforms
|
||
Reddit is the only implemented source, but the interface pattern means adding a new source (YouTube, TikTok, etc.) is one class implementation plus a settings checkbox. We removed the old stub provider files (TikTok, Instagram, YouTube) because dead code confuses AI agents and humans alike — the interface is the contract, not empty classes. Trade-off: slight over-engineering for day one, but pays off immediately at source #2.
|
||
|
||
### #7: Custom tables for high-volume data
|
||
Source data, analysis results, and generation logs are high-write, time-series data. WordPress post_meta and options are wrong for this. Custom tables with proper indexes. Trade-off: more complex activation/uninstall, but correct.
|
||
|
||
### #8: All API keys encrypted at rest
|
||
Using `wp_salt()` as encryption key with OpenSSL. Not bulletproof (salt is in wp-config.php) but significantly better than plaintext in wp_options. Trade-off: adds complexity to option get/set, but necessary for security.
|
||
|
||
### #9: Structured logger instead of raw error_log()
|
||
All application logging flows through `PRAutoBlogger_Logger` singleton, which writes to the `prab_event_log` table and forwards errors/warnings to PHP's `error_log()`. This gives users an in-admin Activity Log page with filtering by level, search, and pagination — much more accessible than server logs. Trade-off: one extra DB write per log entry, but the table is prunable and indexed.
|
||
|
||
### #10: Database-level atomic mutex for generation lock
|
||
Uses `INSERT IGNORE` on wp_options (which has a UNIQUE index on `option_name`) instead of transient-based locking. This eliminates the TOCTOU race condition where two concurrent cron runs could both read the transient as "not locked" before either sets it. Expired locks older than 1 hour are cleaned up first to prevent permanent deadlock.
|
||
|
||
### #11: wp.element for frontend React (no build step)
|
||
The posts widget uses WordPress-bundled React (`wp.element`) with raw `createElement` calls instead of JSX. This avoids requiring a Node.js build pipeline for what is a simple card grid. Trade-off: more verbose component code, but zero tooling dependencies for the plugin consumer.
|
||
|
||
### #12: REST API for frontend data fetching
|
||
The widget fetches posts asynchronously from a dedicated REST endpoint rather than rendering server-side. This keeps initial page load fast and allows the endpoint to be cached independently (5-minute Cache-Control). The permission callback uses a filter (`prautoblogger_rest_posts_public`) so sites can restrict access if needed.
|
||
|
||
### #13: Run-ID based log linking
|
||
Each pipeline execution generates a UUID (`run_id`) that tags every `prab_generation_log` entry. When a post is published, `link_generation_logs()` uses `UPDATE WHERE run_id = X` to associate entries with the `post_id`. This is more reliable than the previous timestamp-window approach, especially in batch runs where multiple posts are generated in quick succession. **v0.8.1+**: `run_id` is also persisted on each post as `_prautoblogger_run_id` post_meta, giving the orphan-research reaper a direct path from run to siblings without walking the gen_log table (see #19).
|
||
|
||
### #19: Daily orphan-research-row reaper (v0.8.1)
|
||
When a pipeline dies between post creation and the final `Post_Assembler::amortize_research_costs()` call (Hostinger exec kill, OOM, LiteSpeed cut, fatal), the shared `llm_research` cost row stays in `prab_generation_log` with `post_id = NULL` and sibling articles understate their true cost in the breakdown popover. A new daily WP-Cron event `prautoblogger_reap_orphan_research_rows` (hooked to `PRAutoBlogger_Research_Reaper::on_cron`, 03:15 server time — 15 min after the primary generation cron) scans for orphan rows older than a 1-hour grace window, looks up sibling posts by `_prautoblogger_run_id` post_meta (fallback: distinct `post_id`s in gen_log for the same run_id), and delegates to the existing idempotent `amortize_research_costs()` on match. Orphans that still have no siblings after 7 days are deleted outright as sunk cost. A one-time activator migration (`prautoblogger_migrated_run_id_meta_v081`) back-populates `_prautoblogger_run_id` on existing posts from gen_log so the meta-primary path works on upgrade. Manual ops trigger: `wp prautoblogger reap-research`.
|
||
|
||
### #14: Reddit RSS replaces PullPush.io (and earlier Reddit OAuth)
|
||
Reddit rejected our OAuth API application (April 2026). We initially switched to PullPush.io, but its index was frequently stale or unavailable. Reddit's RSS/Atom feeds (`/r/{sub}/hot.rss`) proved the most reliable option — they work from datacenter IPs where .json gets 403, require no auth, and have no apparent rate limit. The .json endpoints are kept as a fallback for posts and as the sole source for comment data. Each collected item's metadata includes a `data_source` field (`reddit_rss` or `reddit_json`) for auditability.
|
||
|
||
### #18: OpenRouter model registry — daily refresh, WP option + transient cache, zero-coupling
|
||
|
||
The admin model picker (v1) needs to list OpenRouter models with pricing and capabilities. We fetch `https://openrouter.ai/api/v1/models` (free, unauthenticated) once daily and store the normalized payload in a WP option fronted by a 24h transient. On stale-and-fetch-fails, we serve last-good + surface a warning. The registry class (`class-open-router-model-registry.php`) takes all config via constructor (option name, transient name, endpoint URL) — no PRAUTOBLOGGER_* constants inside the class body. Phase 2 lifts it into a shared Composer package with zero internal edits. Phase 3 adds a parallel `Cloudflare_WorkersAI_Model_Registry` behind the same interface. Capability vocabulary: `text→text`, `text+image→text`, `text+audio→text`, `text→image`, `text→audio`, `text→video`, `text→embedding`. Cost: $0/month.
|
||
|
||
### #16: Image generation via Cloudflare Workers AI (FLUX.1), direct (not via AI Gateway)
|
||
The image-generation layer (started 2026-04-15, tracked in `convo/prautoblogger/threads/2026-04-image-pipeline/`) uses FLUX.1 [schnell] on Cloudflare Workers AI as its default model, chosen for its ~$0.0011/MP cost and 2–3 sec latency. FLUX.1 [dev] is a ~4× cost upgrade exposed as a dropdown for specific posts that warrant it. Calls go directly to `https://api.cloudflare.com/client/v4/accounts/{id}/ai/run/@cf/black-forest-labs/...`, *not* through the Cloudflare AI Gateway that we use for OpenRouter — the gateway route to Workers AI currently 403s (pre-existing open issue). The provider lives behind `PRAutoBlogger_Image_Provider_Interface`, so the decision is trivially reversible: switching to a different image API (DALL-E, Replicate, a future AI Gateway route) is a new class implementation plus a one-line swap in the pipeline wiring. Trade-off: we run two different Cloudflare integration paths (gateway for OpenRouter LLMs, direct for Workers AI images) until the gateway route stabilizes — minor cognitive overhead, zero functional downside.
|
||
|
||
### #15: Optional Cloudflare AI Gateway in front of OpenRouter
|
||
We already use Cloudflare for DNS/CDN on peptiderepo.com, so layering AI Gateway in front of OpenRouter is zero marginal infrastructure. It gives us response caching (meaningful for repeated classification/scoring calls), a unified cost/latency dashboard, rate limiting, and provider fallback — all of which we would otherwise have to build ourselves to satisfy the CTO cost-tracking rules. Kept as an opt-in URL setting (`prautoblogger_ai_gateway_base_url`) so the plugin still works unchanged out of the box and can be bypassed instantly if the gateway misbehaves. The gateway is a transparent OpenRouter-compatible proxy; no new provider class is needed, and the response parsing path (`usage`, `choices[0].message.content`) is unchanged.
|
||
|
||
---
|
||
|
||
## Cross-System LLM Budget Coordination
|
||
|
||
PRAutoBlogger and Peptide News both call OpenRouter and may share a single API key / billing account. Their combined spend should be considered when setting per-plugin budgets.
|
||
|
||
| Plugin | Default Models | Typical Daily Spend | Budget Control |
|
||
|--------|---------------|--------------------:|----------------|
|
||
| PRAutoBlogger | Gemini 2.5 Flash Lite (analysis + editing), Claude Sonnet 4 (writing) | $0.05–$0.30 depending on article count | Hard-stop monthly budget in plugin settings |
|
||
| Peptide News | Google Gemini 2.0 Flash (keywords + summaries) | $0.01–$0.05 | No hard budget yet (planned) |
|
||
|
||
**Important:** If your OpenRouter account has a global spending limit, set each plugin's budget to less than half the total. PRAutoBlogger will hard-stop when its budget is exhausted, but Peptide News currently has no budget enforcement — a spike in news fetches could consume shared quota.
|
||
|
||
**Future improvement:** A shared `wp_options` key (e.g., `ecosystem_monthly_llm_budget`) that both plugins read, with each plugin reserving its allocation on startup. This requires coordination at the ecosystem level and is tracked as a medium-term goal.
|