diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 9cc8683..a34e926 100755 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -15,8 +15,8 @@ PRAutoBlogger is a WordPress plugin that monitors social media (starting with Re ▼ ┌─────────────────────────┐ │ 1. Source Collector │ Pulls raw posts/comments from social platforms -│ (PullPush.io primary │ PullPush.io API (primary) / Reddit .json (fallback) -│ / Reddit .json fallback) Stores raw data in `ab_source_data` table +│ (Reddit RSS primary │ Reddit RSS/Atom feeds (primary) / .json (fallback) +│ / .json fallback) Stores raw data in `ab_source_data` table └────────────┬────────────┘ │ ▼ @@ -128,9 +128,8 @@ prautoblogger/ │ │ ├── class-open-router-provider.php # OpenRouter API implementation │ │ ├── class-open-router-pricing.php # Model pricing lookup and cost estimation │ │ ├── interface-source-provider.php # Contract for any social media source -│ │ ├── class-pull-push-client.php # PullPush.io HTTP client (primary, no auth) -│ │ ├── class-reddit-json-client.php # Reddit .json endpoint client (fallback, no auth) -│ │ ├── class-reddit-provider.php # Reddit data collection (PullPush primary + .json fallback) +│ │ ├── class-reddit-json-client.php # Reddit HTTP client — RSS (primary) + .json (fallback) +│ │ ├── class-reddit-provider.php # Reddit data collection orchestrator (RSS primary) │ │ ├── class-tiktok-provider.php # TikTok — stub/future implementation │ │ ├── class-instagram-provider.php # Instagram — stub/future implementation │ │ └── class-youtube-provider.php # YouTube — stub/future implementation @@ -363,8 +362,8 @@ Stored on every PRAutoBlogger-generated post: | Service | Purpose | Auth | Rate Limit | Code | |---------|---------|------|------------|------| | OpenRouter | All LLM calls (analysis, writing, editing) | API key (encrypted in wp_options) | Per-model | `providers/class-open-router-provider.php`, `providers/class-open-router-pricing.php` | -| PullPush.io | Primary Reddit data source — submissions and comments by subreddit | None (free public API) | 30 req/min, 1000/hour | `providers/class-pull-push-client.php` | -| Reddit .json | Fallback Reddit data — appends .json to Reddit URLs | None (unauthenticated) | ~10 req/min (datacenter IPs may be blocked) | `providers/class-reddit-json-client.php` | +| Reddit RSS | Primary Reddit data source — Atom feeds for subreddit hot posts | None (unauthenticated) | No known rate limit; reliable from datacenter IPs | `providers/class-reddit-json-client.php` | +| Reddit .json | Fallback for posts + only source for comments | None (unauthenticated) | ~10 req/min (datacenter IPs often blocked) | `providers/class-reddit-json-client.php` | | Google Analytics 4 | Post performance metrics | OAuth2 service account | Standard GA4 limits | `core/class-ga4-client.php`, `core/class-metrics-collector.php` | --- @@ -374,8 +373,8 @@ Stored on every PRAutoBlogger-generated post: ### #1: OpenRouter over direct provider APIs Gives access to many models through one API, simplifies provider management, and lets the user switch models without code changes. Trade-off: slight latency overhead and dependency on OpenRouter's availability. -### #2: Reddit as first social source (via PullPush.io + .json fallback) -Reddit has rich discussion data ideal for identifying recurring questions and pain points. PullPush.io is the primary data source (free, no auth required). Reddit .json endpoints serve as a fallback. See Key Decision #14 for the migration from Reddit OAuth. Trade-off: Reddit data is text-heavy (no video transcription needed), making it the simplest starting point. +### #2: Reddit as first social source (RSS primary, .json fallback) +Reddit has rich discussion data ideal for identifying recurring questions and pain points. Reddit RSS/Atom feeds are the primary data source — they work reliably from datacenter IPs (Hostinger) where .json endpoints return 403. The .json endpoints serve as a fallback for posts and are the only source for comment fetching (RSS doesn't include comments). Trade-off: RSS lacks engagement metrics (score, comment count), but post titles and content are sufficient for topic analysis. ### #3: Chief editor agent instead of human review queue The user wants full automation — a second LLM pass reviews quality, SEO, and accuracy before publishing. Posts that fail editorial review are flagged for human intervention rather than published. Trade-off: doubles the LLM cost of the "review" step, but catches quality issues. @@ -410,5 +409,5 @@ The widget fetches posts asynchronously from a dedicated REST endpoint rather th ### #13: Run-ID based log linking Each pipeline execution generates a UUID (`run_id`) that tags every `prab_generation_log` entry. When a post is published, `link_generation_logs()` uses `UPDATE WHERE run_id = X` to associate entries with the `post_id`. This is more reliable than the previous timestamp-window approach, especially in batch runs where multiple posts are generated in quick succession. -### #14: PullPush.io replaces Reddit OAuth -Reddit rejected our OAuth API application (April 2026). Rather than reapply or use a third-party proxy, we switched to PullPush.io as the primary data source. PullPush mirrors Reddit's submission and comment data via a free, no-auth API with 30 req/min limits. Reddit's .json endpoints (appending .json to any Reddit URL) serve as a fallback when PullPush is unavailable. The provider auto-detects PullPush availability on each collection run (cached for 5 minutes) and transparently falls back. Each collected item's metadata includes a `data_source` field (`pullpush` or `reddit_json`) for auditability. This eliminates the need for Reddit client ID/secret storage and the OAuth token lifecycle. +### #14: Reddit RSS replaces PullPush.io (and earlier Reddit OAuth) +Reddit rejected our OAuth API application (April 2026). We initially switched to PullPush.io, but its index was frequently stale or unavailable. Reddit's RSS/Atom feeds (`/r/{sub}/hot.rss`) proved the most reliable option — they work from datacenter IPs where .json gets 403, require no auth, and have no apparent rate limit. The .json endpoints are kept as a fallback for posts and as the sole source for comment data. Each collected item's metadata includes a `data_source` field (`reddit_rss` or `reddit_json`) for auditability. diff --git a/CHANGELOG.md b/CHANGELOG.md index 7b6fcd6..9e26dce 100755 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,20 +5,36 @@ All notable changes to PRAutoBlogger will be documented in this file. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project uses [Semantic Versioning](https://semver.org/). +## [0.2.1] — 2026-04-12 + +### Changed +- **Reddit data source switched to RSS primary, .json fallback.** + PullPush.io was frequently stale/unavailable. Reddit RSS/Atom feeds work + reliably from datacenter IPs (Hostinger) where .json gets 403. Comments + are still fetched via .json (unavailable in RSS). +- Updated "Enabled Sources" checkbox label to "Reddit (RSS + .json)". +- Updated "Test Connections" to show RSS + .json status. +- Source status indicator now shows Reddit RSS (Primary) and .json (Fallback). + +### Removed +- `class-pull-push-client.php` — replaced by RSS feeds in `class-reddit-json-client.php`. +- All PullPush.io references from admin UI, docs, and provider code. + +### Fixed +- Generate Now button now always sends `force: '1'` to clear stale generation locks. +- Fixed encryption double-encryption bug with `enc:` prefix detection. + ## [0.2.0] — 2026-04-12 ### Changed -- **Reddit data source migrated from OAuth to PullPush.io + .json fallback.** - Reddit rejected our API application; PullPush.io is free, requires no auth, - and provides better subreddit-wide comment search. Reddit .json endpoints - serve as an automatic fallback when PullPush is down. +- **Reddit data source migrated from OAuth to RSS + .json (no auth required).** + Reddit rejected our API application; RSS feeds and .json endpoints are free + and require no authentication. - Removed Reddit Client ID and Client Secret fields from admin API Keys tab. -- Updated "Test Connections" results to show which Reddit source is active - (PullPush.io or .json fallback) instead of generic "Reddit API connected." -- Updated "Enabled Sources" checkbox label from "Reddit" to "Reddit (via PullPush.io)". +- Updated "Test Connections" results to show Reddit source status. ### Added -- Source status indicator in API Keys tab showing PullPush.io and Reddit .json +- Source status indicator in API Keys tab showing RSS and .json availability with live status dots and last-collection timestamp. - Configurable research cache TTL (1–72 hours) in Sources tab. - Reddit time window selector (24h / week / month) in Sources tab. @@ -26,8 +42,7 @@ and this project uses [Semantic Versioning](https://semver.org/). - LiteSpeed cache purge step in CI/CD deploy pipeline. ### Removed -- `class-reddit-api-client.php` — replaced by `class-pull-push-client.php` - and `class-reddit-json-client.php`. +- `class-reddit-api-client.php` — replaced by `class-reddit-json-client.php`. - Reddit OAuth credential fields and encrypted storage entry. ### Fixed diff --git a/includes/admin/class-admin-page.php b/includes/admin/class-admin-page.php index ceaa8aa..f62c4bb 100755 --- a/includes/admin/class-admin-page.php +++ b/includes/admin/class-admin-page.php @@ -175,7 +175,7 @@ class PRAutoBlogger_Admin_Page { /** * Render the Reddit source status indicator. * - * Shows PullPush.io (primary) and Reddit .json (fallback) availability + * Shows Reddit RSS (primary) and .json (fallback) availability * with live status checks via transient cache and rate limit info. * * Side effects: Reads transients for cached availability status. @@ -183,37 +183,24 @@ class PRAutoBlogger_Admin_Page { * @return void */ private function render_source_status_field(): void { - $pullpush_available = get_transient( 'prautoblogger_pullpush_available' ); - $last_collection = get_option( 'prautoblogger_last_collection_time', '' ); - $last_source_used = get_option( 'prautoblogger_last_source_used', '' ); - - // Determine status display. - $pp_status_class = 'ab-status-unknown'; - $pp_status_label = __( 'Unknown', 'prautoblogger' ); - if ( true === $pullpush_available ) { - $pp_status_class = 'ab-status-ok'; - $pp_status_label = __( 'Available', 'prautoblogger' ); - } elseif ( false === $pullpush_available ) { - $pp_status_class = 'ab-status-warn'; - $pp_status_label = __( 'Down — using fallback', 'prautoblogger' ); - } + $last_collection = get_option( 'prautoblogger_last_collection_time', '' ); + $last_source_used = get_option( 'prautoblogger_last_source_used', '' ); echo '
'; - // PullPush status. + // Reddit RSS — primary source. printf( '
' - . '' + . '' . '%s %s' . '%s' . '
', - esc_attr( $pp_status_class ), - esc_html__( 'PullPush.io', 'prautoblogger' ), + esc_html__( 'Reddit RSS', 'prautoblogger' ), esc_html__( 'Primary', 'prautoblogger' ), - esc_html( $pp_status_label ) + esc_html__( 'Reliable from all IPs', 'prautoblogger' ) ); - // Reddit .json fallback status. + // Reddit .json — fallback source. printf( '
' . '' @@ -221,8 +208,8 @@ class PRAutoBlogger_Admin_Page { . '%s' . '
', esc_html__( 'Reddit .json', 'prautoblogger' ), - esc_html__( 'Fallback', 'prautoblogger' ), - esc_html__( 'Always available (rate-limited)', 'prautoblogger' ) + esc_html__( 'Fallback + Comments', 'prautoblogger' ), + esc_html__( 'May be IP-blocked on some hosts', 'prautoblogger' ) ); // Last collection info. diff --git a/includes/admin/class-settings-fields.php b/includes/admin/class-settings-fields.php index 207cae3..427e392 100755 --- a/includes/admin/class-settings-fields.php +++ b/includes/admin/class-settings-fields.php @@ -81,7 +81,7 @@ class PRAutoBlogger_Settings_Fields { 'label' => __( 'Reddit Data Source', 'prautoblogger' ), 'type' => 'source_status', 'section' => 'prautoblogger_api', - 'description' => __( 'Uses PullPush.io (primary) and Reddit .json (fallback). No API key required.', 'prautoblogger' ), + 'description' => __( 'Uses Reddit RSS (primary) and .json (fallback). No API key required.', 'prautoblogger' ), 'icon' => '📡', ], @@ -185,7 +185,7 @@ class PRAutoBlogger_Settings_Fields { 'type' => 'checkboxes', 'section' => 'prautoblogger_sources', 'options' => [ - 'reddit' => __( 'Reddit (via PullPush.io)', 'prautoblogger' ), + 'reddit' => __( 'Reddit (RSS + .json)', 'prautoblogger' ), 'tiktok' => __( 'TikTok (coming soon)', 'prautoblogger' ), 'instagram' => __( 'Instagram (coming soon)', 'prautoblogger' ), 'youtube' => __( 'YouTube (coming soon)', 'prautoblogger' ), @@ -201,7 +201,7 @@ class PRAutoBlogger_Settings_Fields { 'default' => 6, 'min' => 1, 'max' => 72, - 'description' => __( 'How long to cache PullPush research results before re-fetching. Saves API calls on low-traffic sites.', 'prautoblogger' ), + 'description' => __( 'How long to cache Reddit research results before re-fetching. Saves bandwidth on low-traffic sites.', 'prautoblogger' ), ], [ 'id' => 'prautoblogger_reddit_time_filter', diff --git a/includes/class-prautoblogger.php b/includes/class-prautoblogger.php index b40c3f4..c7d5acc 100644 --- a/includes/class-prautoblogger.php +++ b/includes/class-prautoblogger.php @@ -316,11 +316,9 @@ class PRAutoBlogger { if ( 'reddit' === $service || 'all' === $service ) { $reddit = new PRAutoBlogger_Reddit_Provider(); $is_ok = $reddit->validate_credentials(); - $pp_up = get_transient( 'prautoblogger_pullpush_available' ); - $source = true === $pp_up ? 'PullPush.io' : 'Reddit .json fallback'; $results['reddit'] = $is_ok - ? [ 'status' => 'ok', 'message' => sprintf( __( 'Reddit source available via %s.', 'prautoblogger' ), $source ) ] - : [ 'status' => 'error', 'message' => __( 'Both PullPush.io and Reddit .json are unreachable.', 'prautoblogger' ) ]; + ? [ 'status' => 'ok', 'message' => __( 'Reddit source available (RSS primary, .json fallback).', 'prautoblogger' ) ] + : [ 'status' => 'error', 'message' => __( 'Reddit sources unreachable (both RSS and .json failed).', 'prautoblogger' ) ]; } wp_send_json_success( $results ); } diff --git a/includes/providers/class-pull-push-client.php b/includes/providers/class-pull-push-client.php deleted file mode 100644 index 418f471..0000000 --- a/includes/providers/class-pull-push-client.php +++ /dev/null @@ -1,382 +0,0 @@ -> Array of submission data arrays. - */ - public function fetch_posts( string $subreddit, int $limit, string $time_filter = 'day' ): array { - $size = min( $limit, self::MAX_SIZE ); - $after = $this->time_filter_to_epoch( $time_filter ); - - $url = sprintf( - '%s/search/submission?subreddit=%s&sort=score&sort_type=desc&size=%d&after=%d', - self::API_BASE, - rawurlencode( $subreddit ), - $size, - $after - ); - - $response = $this->api_get( $url ); - - if ( null === $response ) { - return []; - } - - // PullPush returns { "data": [...] } or just [...] depending on endpoint version. - $items = $response['data'] ?? $response; - - if ( ! is_array( $items ) ) { - return []; - } - - return $items; - } - - /** - * Fetch recent comments from a subreddit, sorted by score. - * - * Unlike the Reddit API which fetches comments per-post, PullPush lets us - * search comments across an entire subreddit. This is more efficient for - * finding high-quality discussion content. - * - * Side effects: HTTP request to PullPush API, updates rate limit transient. - * - * @param string $subreddit Subreddit name (without r/ prefix). - * @param int $limit Max comments to fetch (capped at 100 by API). - * @param string $time_filter Time window for filtering. - * - * @return array> Array of comment data arrays. - */ - public function fetch_comments( string $subreddit, int $limit, string $time_filter = 'day' ): array { - $size = min( $limit, self::MAX_SIZE ); - $after = $this->time_filter_to_epoch( $time_filter ); - - $url = sprintf( - '%s/search/comment?subreddit=%s&sort=score&sort_type=desc&size=%d&after=%d', - self::API_BASE, - rawurlencode( $subreddit ), - $size, - $after - ); - - $response = $this->api_get( $url ); - - if ( null === $response ) { - return []; - } - - $items = $response['data'] ?? $response; - - if ( ! is_array( $items ) ) { - return []; - } - - return $items; - } - - /** - * Fetch comments for a specific post by its Reddit ID. - * - * Uses the link_id filter (t3_ prefixed) to get comments for one post. - * - * Side effects: HTTP request to PullPush API. - * - * @param string $post_id Reddit post ID (without t3_ prefix). - * @param int $limit Max comments to fetch. - * - * @return array> Array of comment data arrays. - */ - public function fetch_comments_for_post( string $post_id, int $limit = 5 ): array { - $size = min( $limit, self::MAX_SIZE ); - - $url = sprintf( - '%s/search/comment?link_id=t3_%s&sort=score&sort_type=desc&size=%d', - self::API_BASE, - rawurlencode( $post_id ), - $size - ); - - $response = $this->api_get( $url ); - - if ( null === $response ) { - return []; - } - - $items = $response['data'] ?? $response; - - return is_array( $items ) ? $items : []; - } - - /** - * Check if PullPush API is reachable AND has reasonably fresh data. - * - * PullPush can return HTTP 200 but have a stale index (months behind Reddit). - * We check that the most recent post is no older than 7 days. If the index - * is stale, we return false so the caller falls back to Reddit .json. - * - * Side effects: HTTP request. - * - * @return bool True if API responded with HTTP 200 and has data < 7 days old. - */ - public function is_available(): bool { - // Fetch 1 recent submission sorted by date to check freshness. - $url = self::API_BASE . '/search/submission?subreddit=all&size=1&sort_type=created_utc&sort=desc'; - $response = wp_remote_get( - $url, - [ - 'timeout' => 10, - 'headers' => [ - 'User-Agent' => self::USER_AGENT, - ], - ] - ); - - if ( is_wp_error( $response ) ) { - return false; - } - - if ( 200 !== wp_remote_retrieve_response_code( $response ) ) { - return false; - } - - $body = json_decode( wp_remote_retrieve_body( $response ), true ); - $items = $body['data'] ?? []; - - if ( empty( $items ) ) { - return false; - } - - // Check that the most recent post is within the last 7 days. - // If PullPush's index is stale (e.g. months behind), we should - // fall back to Reddit .json which always has live data. - $latest_utc = (int) ( $items[0]['created_utc'] ?? 0 ); - $seven_days = 7 * DAY_IN_SECONDS; - $is_fresh = ( time() - $latest_utc ) < $seven_days; - - if ( ! $is_fresh ) { - PRAutoBlogger_Logger::instance()->warning( - sprintf( - 'PullPush index is stale — latest post is from %s (%d days old). Falling back to Reddit .json.', - gmdate( 'Y-m-d', $latest_utc ), - (int) ( ( time() - $latest_utc ) / DAY_IN_SECONDS ) - ), - 'pullpush' - ); - } - - return $is_fresh; - } - - /** - * Get current rate limit status. - * - * PullPush doesn't return rate limit headers, so we track our own - * request count using a transient with a 60-second TTL. - * - * @return array{remaining: int, limit: int, resets_at: string} - */ - public function get_rate_limit_status(): array { - $status = get_transient( self::RATE_LIMIT_TRANSIENT ); - - if ( is_array( $status ) ) { - return $status; - } - - // No tracking data yet — assume fresh window. - return [ - 'remaining' => 30, - 'limit' => 30, - 'resets_at' => '', - ]; - } - - /** - * Make an HTTP GET request to the PullPush API. - * - * Handles error logging, response parsing, and self-imposed rate limit tracking. - * No authentication required — PullPush is a free public API. - * - * Side effects: HTTP request, sets rate limit transient, logs errors. - * - * @param string $url Full API URL with query parameters. - * - * @return array|null Decoded JSON response, or null on failure. - */ - private function api_get( string $url ): ?array { - // Self-imposed rate limiting: track requests per minute. - $this->track_rate_limit(); - - $response = wp_remote_get( - $url, - [ - 'timeout' => self::TIMEOUT_SECONDS, - 'headers' => [ - 'User-Agent' => self::USER_AGENT, - 'Accept' => 'application/json', - ], - ] - ); - - if ( is_wp_error( $response ) ) { - PRAutoBlogger_Logger::instance()->error( - 'PullPush API GET failed: ' . $response->get_error_message(), - 'pullpush' - ); - return null; - } - - $status_code = wp_remote_retrieve_response_code( $response ); - - if ( 200 !== $status_code ) { - PRAutoBlogger_Logger::instance()->error( - sprintf( - 'PullPush API HTTP %d: %s', - $status_code, - substr( wp_remote_retrieve_body( $response ), 0, 300 ) - ), - 'pullpush' - ); - return null; - } - - $body = wp_remote_retrieve_body( $response ); - $data = json_decode( $body, true ); - - if ( ! is_array( $data ) ) { - PRAutoBlogger_Logger::instance()->error( - 'PullPush API returned non-JSON response.', - 'pullpush' - ); - return null; - } - - return $data; - } - - /** - * Track our own request count to avoid exceeding PullPush's 30 req/min limit. - * - * Uses a transient that stores the request count and window start time. - * If we're close to the limit, we don't block — the caller should check - * get_rate_limit_status() before making bulk requests. - * - * Side effects: Sets transient. - */ - private function track_rate_limit(): void { - $status = get_transient( self::RATE_LIMIT_TRANSIENT ); - - if ( ! is_array( $status ) ) { - $status = [ - 'remaining' => 29, // This request uses one. - 'limit' => 30, - 'resets_at' => gmdate( 'Y-m-d H:i:s', time() + 60 ), - ]; - } else { - $status['remaining'] = max( 0, (int) $status['remaining'] - 1 ); - } - - // TTL of 60 seconds — the window auto-resets. - set_transient( self::RATE_LIMIT_TRANSIENT, $status, 60 ); - } - - /** - * Convert a human-readable time filter to a Unix epoch timestamp. - * - * PullPush uses `after` parameter as a Unix timestamp, not Reddit's - * `t=day` style. This maps between the two conventions. - * - * @param string $time_filter One of: 'hour', 'day', 'week', 'month', 'year', 'all'. - * - * @return int Unix timestamp to use as the `after` parameter. - */ - private function time_filter_to_epoch( string $time_filter ): int { - $now = time(); - - switch ( $time_filter ) { - case 'hour': - return $now - 3600; - case 'day': - return $now - 86400; - case 'week': - return $now - 604800; - case 'month': - return $now - 2592000; - case 'year': - return $now - 31536000; - case 'all': - return 0; - default: - return $now - 86400; // Default to 1 day. - } - } -} diff --git a/includes/providers/class-reddit-json-client.php b/includes/providers/class-reddit-json-client.php index 1771f67..4079f8f 100644 --- a/includes/providers/class-reddit-json-client.php +++ b/includes/providers/class-reddit-json-client.php @@ -2,24 +2,23 @@ declare(strict_types=1); /** - * Reddit .json endpoint client — fallback when PullPush.io is unavailable. + * Reddit HTTP client — fetches posts via RSS (primary) and .json (fallback). * - * Appends `.json` to standard Reddit URLs to get JSON responses without OAuth. - * No authentication required, but heavily rate-limited (~10 req/min for - * unauthenticated requests) and datacenter IPs may be blocked aggressively. - * This is a FALLBACK only — PullPush is the primary data source. + * RSS feeds are the primary data source because they work reliably from datacenter + * IPs (Hostinger), while .json endpoints return 403 for server IPs. The .json + * endpoints are kept as a fallback in case RSS ever breaks, and are always used + * for comment fetching (comments are not available in RSS feeds). * - * Triggered by: PRAutoBlogger_Reddit_Provider when PullPush is_available() returns false. + * Triggered by: PRAutoBlogger_Reddit_Provider::collect_data(). * Dependencies: wp_remote_get(), PRAutoBlogger_Logger. * - * @see providers/class-pullpush-client.php — Primary client (preferred). - * @see providers/class-reddit-provider.php — Instantiates this class as fallback. - * @see ARCHITECTURE.md — External API integrations table. + * @see providers/class-reddit-provider.php — Instantiates and calls this class. + * @see ARCHITECTURE.md — External API integrations table. */ class PRAutoBlogger_Reddit_JSON_Client { /** - * Reddit base URL for .json endpoints. + * Reddit base URL for all endpoints. * * @var string */ @@ -31,7 +30,7 @@ class PRAutoBlogger_Reddit_JSON_Client { * * @var string */ - private const USER_AGENT = 'Mozilla/5.0 (compatible; PRAutoBlogger/0.1.0; +https://peptiderepo.com)'; + private const USER_AGENT = 'Mozilla/5.0 (compatible; PRAutoBlogger/0.2.1; +https://peptiderepo.com)'; /** * HTTP timeout in seconds. @@ -41,63 +40,50 @@ class PRAutoBlogger_Reddit_JSON_Client { private const TIMEOUT_SECONDS = 30; /** - * Transient key for rate limit tracking. + * Transient key for rate limit tracking (.json endpoints only). * * @var string */ private const RATE_LIMIT_TRANSIENT = 'prautoblogger_reddit_json_rate_limit'; /** - * Fetch hot posts from a subreddit via Reddit's .json endpoint. + * Fetch hot posts from a subreddit. Tries RSS first, .json fallback. * - * Returns data in the same format as the old OAuth client's fetch_posts(): - * an array of post data arrays with standard Reddit field names. + * RSS is the primary source because it works from datacenter IPs where + * .json returns 403. If RSS fails, we fall back to .json. Both return + * data in the same normalized format. * - * Side effects: HTTP request, updates rate limit transient. + * Side effects: HTTP request to Reddit. * * @param string $subreddit Subreddit name (without r/ prefix). * @param int $limit Max posts to fetch (Reddit caps at 100). - * @param string $time_filter Time filter for /top (hour, day, week, month, year, all). + * @param string $time_filter Time filter (unused by RSS, used by .json). * * @return array> Array of post data. */ public function fetch_posts( string $subreddit, int $limit, string $time_filter = 'day' ): array { - $url = sprintf( - '%s/r/%s/hot.json?limit=%d&t=%s&raw_json=1', - self::BASE_URL, - rawurlencode( $subreddit ), - min( $limit, 100 ), - rawurlencode( $time_filter ) + // Try RSS first — most reliable from datacenter IPs. + $posts = $this->fetch_posts_via_rss( $subreddit, $limit ); + + if ( ! empty( $posts ) ) { + return $posts; + } + + // RSS failed — fall back to .json endpoints. + PRAutoBlogger_Logger::instance()->info( + sprintf( 'Reddit RSS returned 0 posts for r/%s — trying .json fallback.', $subreddit ), + 'reddit' ); - $response = $this->api_get( $url ); - - if ( null === $response || ! isset( $response['data']['children'] ) ) { - // .json endpoint failed (likely 403 IP block) — try RSS fallback. - PRAutoBlogger_Logger::instance()->info( - sprintf( 'Reddit .json failed for r/%s — trying RSS fallback.', $subreddit ), - 'reddit_json' - ); - return $this->fetch_posts_via_rss( $subreddit, $limit ); - } - - $posts = []; - foreach ( $response['data']['children'] as $child ) { - if ( isset( $child['data'] ) ) { - $posts[] = $child['data']; - } - } - - return $posts; + return $this->fetch_posts_via_json( $subreddit, $limit, $time_filter ); } /** * Fetch posts from a subreddit via Reddit's RSS/Atom feed. * - * RSS endpoints are less commonly blocked by Reddit than .json. - * Returns data in the same format as fetch_posts() for compatibility, - * but some fields (score, num_comments) are unavailable in RSS and - * set to defaults. + * RSS endpoints work reliably from datacenter IPs where .json gets blocked. + * Returns data normalized to the same format as .json for pipeline compatibility. + * Some fields (score, num_comments) are unavailable in RSS and set to defaults. * * Side effects: HTTP request to Reddit. * @@ -106,7 +92,7 @@ class PRAutoBlogger_Reddit_JSON_Client { * * @return array> Array of post data. */ - private function fetch_posts_via_rss( string $subreddit, int $limit ): array { + public function fetch_posts_via_rss( string $subreddit, int $limit ): array { $url = sprintf( '%s/r/%s/hot.rss?limit=%d', self::BASE_URL, @@ -128,7 +114,7 @@ class PRAutoBlogger_Reddit_JSON_Client { if ( is_wp_error( $response ) ) { PRAutoBlogger_Logger::instance()->error( 'Reddit RSS GET failed: ' . $response->get_error_message(), - 'reddit_json' + 'reddit' ); return []; } @@ -137,7 +123,7 @@ class PRAutoBlogger_Reddit_JSON_Client { if ( 200 !== $status_code ) { PRAutoBlogger_Logger::instance()->error( sprintf( 'Reddit RSS HTTP %d for r/%s', $status_code, $subreddit ), - 'reddit_json' + 'reddit' ); return []; } @@ -146,6 +132,47 @@ class PRAutoBlogger_Reddit_JSON_Client { return $this->parse_atom_feed( $body, $subreddit ); } + /** + * Fetch hot posts from a subreddit via Reddit's .json endpoint (fallback). + * + * Datacenter IPs are often blocked by Reddit for .json endpoints (HTTP 403). + * This method is only used as a fallback when RSS fails. + * + * Side effects: HTTP request, updates rate limit transient. + * + * @param string $subreddit Subreddit name (without r/ prefix). + * @param int $limit Max posts to fetch (Reddit caps at 100). + * @param string $time_filter Time filter for /hot (hour, day, week, month, year, all). + * + * @return array> Array of post data. + */ + public function fetch_posts_via_json( string $subreddit, int $limit, string $time_filter = 'day' ): array { + $url = sprintf( + '%s/r/%s/hot.json?limit=%d&t=%s&raw_json=1', + self::BASE_URL, + rawurlencode( $subreddit ), + min( $limit, 100 ), + rawurlencode( $time_filter ) + ); + + $response = $this->api_get( $url ); + + if ( null === $response || ! isset( $response['data']['children'] ) ) { + return []; + } + + $posts = []; + foreach ( $response['data']['children'] as $child ) { + if ( isset( $child['data'] ) ) { + $post = $child['data']; + $post['data_source'] = 'reddit_json'; + $posts[] = $post; + } + } + + return $posts; + } + /** * Parse a Reddit Atom feed into post-data arrays matching .json format. * @@ -165,7 +192,7 @@ class PRAutoBlogger_Reddit_JSON_Client { libxml_use_internal_errors( $prev_errors ); if ( false === $feed ) { - PRAutoBlogger_Logger::instance()->error( 'Failed to parse Reddit RSS XML.', 'reddit_json' ); + PRAutoBlogger_Logger::instance()->error( 'Failed to parse Reddit RSS XML.', 'reddit' ); return []; } @@ -193,11 +220,10 @@ class PRAutoBlogger_Reddit_JSON_Client { // Content is in tag as HTML. if ( isset( $entry->content ) ) { - // Strip HTML tags to get plain text for content field. $content = wp_strip_all_tags( (string) $entry->content ); } - // Extract post ID from the entry id (format: /r/subreddit/comments/ID/...) + // Extract post ID from the entry id (format: /r/subreddit/comments/ID/...). $entry_id = (string) $entry->id; $post_id = ''; if ( preg_match( '#/comments/([a-z0-9]+)#', $entry_id, $m ) ) { @@ -217,30 +243,30 @@ class PRAutoBlogger_Reddit_JSON_Client { // Extract permalink (relative path) from full URL. $permalink = ''; if ( '' !== $link ) { - $parsed = wp_parse_url( $link ); + $parsed = wp_parse_url( $link ); $permalink = $parsed['path'] ?? ''; } $posts[] = [ - 'id' => $post_id, - 'title' => $title, - 'selftext' => $content, - 'author' => $author, - 'score' => 1, // Not available in RSS. - 'num_comments' => 0, // Not available in RSS. - 'permalink' => $permalink, - 'created_utc' => $created_utc, - 'is_self' => true, - 'link_flair_text' => null, - 'upvote_ratio' => null, + 'id' => $post_id, + 'title' => $title, + 'selftext' => $content, + 'author' => $author, + 'score' => 1, // Not available in RSS. + 'num_comments' => 0, // Not available in RSS. + 'permalink' => $permalink, + 'created_utc' => $created_utc, + 'is_self' => true, + 'link_flair_text' => null, + 'upvote_ratio' => null, 'is_original_content' => false, - 'data_source' => 'reddit_rss', // Tag for debugging. + 'data_source' => 'reddit_rss', ]; } PRAutoBlogger_Logger::instance()->info( sprintf( 'Parsed %d posts from Reddit RSS for r/%s.', count( $posts ), $subreddit ), - 'reddit_json' + 'reddit' ); return $posts; @@ -249,8 +275,9 @@ class PRAutoBlogger_Reddit_JSON_Client { /** * Fetch top comments for a specific post via Reddit's .json endpoint. * - * Returns data in the same format as the old OAuth client's fetch_comments(): - * an array of comment data arrays with standard Reddit field names. + * Comments are always fetched via .json because RSS feeds don't include them. + * Datacenter IP blocks may prevent this from working, in which case we + * return an empty array (articles can still be generated from post titles alone). * * Side effects: HTTP request, updates rate limit transient. * @@ -288,22 +315,49 @@ class PRAutoBlogger_Reddit_JSON_Client { } /** - * Check if Reddit .json or RSS endpoints are reachable from this server. + * Check if at least one Reddit endpoint is reachable from this server. * - * Datacenter IPs are often blocked by Reddit for .json endpoints. - * We check .json first, then fall back to RSS availability. + * Tries RSS first (most reliable), then .json as fallback. * * Side effects: HTTP request. * * @return bool True if we got a valid response from either endpoint. */ public function is_available(): bool { - return $this->is_json_available() || $this->is_rss_available(); + return $this->is_rss_available() || $this->is_json_available(); + } + + /** + * Check if Reddit RSS endpoints are reachable. + * + * RSS endpoints are rarely blocked by Reddit, even for datacenter IPs. + * + * @return bool True if RSS endpoint returns HTTP 200. + */ + public function is_rss_available(): bool { + $url = self::BASE_URL . '/r/all/hot.rss?limit=1'; + $response = wp_remote_get( + $url, + [ + 'timeout' => 10, + 'headers' => [ + 'User-Agent' => self::USER_AGENT, + ], + ] + ); + + if ( is_wp_error( $response ) ) { + return false; + } + + return 200 === wp_remote_retrieve_response_code( $response ); } /** * Check if Reddit .json endpoints are reachable. * + * Datacenter IPs are often blocked by Reddit for .json endpoints. + * * @return bool True if .json endpoint returns HTTP 200. */ public function is_json_available(): bool { @@ -325,37 +379,10 @@ class PRAutoBlogger_Reddit_JSON_Client { return 200 === wp_remote_retrieve_response_code( $response ); } - /** - * Check if Reddit RSS endpoints are reachable. - * - * RSS endpoints are less commonly blocked by Reddit than .json. - * - * @return bool True if RSS endpoint returns HTTP 200 with XML content. - */ - private function is_rss_available(): bool { - $url = self::BASE_URL . '/r/all/hot.rss?limit=1'; - $response = wp_remote_get( - $url, - [ - 'timeout' => 10, - 'headers' => [ - 'User-Agent' => self::USER_AGENT, - ], - ] - ); - - if ( is_wp_error( $response ) ) { - return false; - } - - return 200 === wp_remote_retrieve_response_code( $response ); - } - /** * Get current rate limit status. * - * Reddit .json endpoints return rate limit headers similar to the OAuth API. - * We track them in a transient. + * Reddit .json endpoints return rate limit headers. RSS has no rate limits. * * @return array{remaining: int, limit: int, resets_at: string} */ @@ -400,12 +427,12 @@ class PRAutoBlogger_Reddit_JSON_Client { if ( is_wp_error( $response ) ) { PRAutoBlogger_Logger::instance()->error( 'Reddit .json GET failed: ' . $response->get_error_message(), - 'reddit_json' + 'reddit' ); return null; } - // Track rate limits from response headers (same as OAuth API). + // Track rate limits from response headers. $rate_remaining = wp_remote_retrieve_header( $response, 'x-ratelimit-remaining' ); $rate_used = wp_remote_retrieve_header( $response, 'x-ratelimit-used' ); $rate_reset = wp_remote_retrieve_header( $response, 'x-ratelimit-reset' ); @@ -431,7 +458,7 @@ class PRAutoBlogger_Reddit_JSON_Client { $status_code, substr( wp_remote_retrieve_body( $response ), 0, 300 ) ), - 'reddit_json' + 'reddit' ); return null; } diff --git a/includes/providers/class-reddit-provider.php b/includes/providers/class-reddit-provider.php index 8d54430..fb38c9c 100755 --- a/includes/providers/class-reddit-provider.php +++ b/includes/providers/class-reddit-provider.php @@ -4,47 +4,30 @@ declare(strict_types=1); /** * Reddit data provider — collects posts and comments from target subreddits. * - * Uses PullPush.io as the PRIMARY data source (free, no auth required, 30 req/min). - * Falls back to Reddit .json endpoints if PullPush is unavailable. - * Reddit OAuth was removed because Reddit rejected our API application. + * Uses Reddit RSS/Atom feeds as the PRIMARY data source (reliable from datacenter IPs, + * no auth required, no rate-limit headers). Falls back to Reddit .json endpoints if + * RSS fails. PullPush.io was removed because its index was frequently stale/unavailable. * * Triggered by: Source_Collector calls collect_data() on each enabled provider. - * Dependencies: PRAutoBlogger_PullPush_Client (primary), PRAutoBlogger_Reddit_JSON_Client (fallback). + * Dependencies: PRAutoBlogger_Reddit_JSON_Client (handles both RSS and .json). * - * @see interface-source-provider.php — Interface this class implements. - * @see providers/class-pullpush-client.php — Primary HTTP client (PullPush.io). - * @see providers/class-reddit-json-client.php — Fallback HTTP client (Reddit .json). + * @see providers/class-reddit-json-client.php — HTTP client for RSS and .json endpoints. * @see core/class-source-collector.php — Orchestrates collection from all providers. * @see ARCHITECTURE.md — External API integrations table. */ class PRAutoBlogger_Reddit_Provider implements PRAutoBlogger_Source_Provider_Interface { - /** - * Transient key for caching PullPush availability check. - * Avoids re-checking on every request within a collection run. - * - * @var string - */ - private const PULLPUSH_AVAILABLE_TRANSIENT = 'prautoblogger_pullpush_available'; - - /** - * How long to cache PullPush availability status (5 minutes). - * - * @var int - */ - private const AVAILABILITY_CACHE_TTL = 300; - /** * Collect posts and comments from configured subreddits. * - * Strategy: Try PullPush first (better API, subreddit-wide comment search). - * If PullPush is down, fall back to Reddit .json endpoints (same response - * format as the old OAuth API, but unauthenticated and rate-limited). + * Strategy: RSS first (works reliably from Hostinger datacenter IPs). + * If RSS fails, falls back to .json endpoints. Comment fetching always + * uses .json since comments are not available in RSS feeds. * - * For each subreddit: fetches top posts, then fetches top comments for - * the most-engaged posts. Returns an array of Source_Data objects. + * For each subreddit: fetches posts via RSS, then fetches top comments + * for high-engagement posts via .json. Returns Source_Data objects. * - * Side effects: HTTP requests to PullPush or Reddit, uses/sets transients. + * Side effects: HTTP requests to Reddit, uses/sets transients. * * @param array{ * subreddits?: string[], @@ -63,16 +46,14 @@ class PRAutoBlogger_Reddit_Provider implements PRAutoBlogger_Source_Provider_Int $include_comments = $config['include_comments'] ?? true; $comments_limit = $config['comments_per_post'] ?? 5; - // Determine which client to use. - $use_pullpush = $this->is_pullpush_available(); - $collected = []; + $client = new PRAutoBlogger_Reddit_JSON_Client(); + $collected = []; PRAutoBlogger_Logger::instance()->info( sprintf( - 'Reddit collect_data: %d subreddits=%s, use_pullpush=%s, limit=%d, time=%s', + 'Reddit collect_data: %d subreddits=%s, limit=%d, time=%s', count( $subreddits ), wp_json_encode( $subreddits ), - $use_pullpush ? 'yes' : 'no', $limit, $time_filter ), @@ -84,24 +65,70 @@ class PRAutoBlogger_Reddit_Provider implements PRAutoBlogger_Source_Provider_Int try { $before_count = count( $collected ); - if ( $use_pullpush ) { - $collected = array_merge( - $collected, - $this->collect_from_pullpush( $subreddit, $limit, $time_filter, $include_comments, $comments_limit ) - ); - } else { - $collected = array_merge( - $collected, - $this->collect_from_reddit_json( $subreddit, $limit, $time_filter, $include_comments, $comments_limit ) - ); + + // Fetch posts — client tries RSS first, .json fallback internally. + $posts = $client->fetch_posts( $subreddit, $limit, $time_filter ); + + foreach ( $posts as $post ) { + $data_source = $post['data_source'] ?? 'reddit'; + + $collected[] = new PRAutoBlogger_Source_Data( [ + 'source_type' => 'reddit', + 'source_id' => 't3_' . ( $post['id'] ?? '' ), + 'subreddit' => $subreddit, + 'title' => $post['title'] ?? '', + 'content' => $post['selftext'] ?? '', + 'author' => $post['author'] ?? '[deleted]', + 'score' => (int) ( $post['score'] ?? 0 ), + 'comment_count' => (int) ( $post['num_comments'] ?? 0 ), + 'permalink' => 'https://reddit.com' . ( $post['permalink'] ?? '' ), + 'collected_at' => current_time( 'mysql' ), + 'metadata' => [ + 'post_type' => ( $post['is_self'] ?? false ) ? 'self' : 'link', + 'flair' => $post['link_flair_text'] ?? null, + 'created_utc' => $post['created_utc'] ?? null, + 'upvote_ratio' => $post['upvote_ratio'] ?? null, + 'is_original' => $post['is_original_content'] ?? false, + 'data_source' => $data_source, + ], + ] ); + + // Fetch comments for high-engagement posts via .json + // (comments are not available in RSS feeds). + if ( $include_comments && ( $post['num_comments'] ?? 0 ) > 0 ) { + $post_id = $post['id'] ?? ''; + $comments = $client->fetch_comments( $subreddit, $post_id, $comments_limit ); + + foreach ( $comments as $comment ) { + $collected[] = new PRAutoBlogger_Source_Data( [ + 'source_type' => 'reddit', + 'source_id' => 't1_' . ( $comment['id'] ?? '' ), + 'subreddit' => $subreddit, + 'title' => null, + 'content' => $comment['body'] ?? '', + 'author' => $comment['author'] ?? '[deleted]', + 'score' => (int) ( $comment['score'] ?? 0 ), + 'comment_count' => 0, + 'permalink' => 'https://reddit.com' . ( $comment['permalink'] ?? '' ), + 'collected_at' => current_time( 'mysql' ), + 'metadata' => [ + 'parent_post_id' => 't3_' . $post_id, + 'parent_title' => $post['title'] ?? '', + 'is_comment' => true, + 'created_utc' => $comment['created_utc'] ?? null, + 'data_source' => 'reddit_json', + ], + ] ); + } + } } + $added = count( $collected ) - $before_count; PRAutoBlogger_Logger::instance()->info( - sprintf( 'Reddit r/%s: fetched %d items via %s', $subreddit, $added, $use_pullpush ? 'pullpush' : 'reddit_json' ), + sprintf( 'Reddit r/%s: fetched %d items', $subreddit, $added ), 'reddit' ); } catch ( \Exception $e ) { - // Log but continue with other subreddits. PRAutoBlogger_Logger::instance()->error( sprintf( 'Reddit: Failed to collect from r/%s: %s', @@ -126,236 +153,28 @@ class PRAutoBlogger_Reddit_Provider implements PRAutoBlogger_Source_Provider_Int /** * Validate that at least one data source is reachable. * - * No credentials needed anymore — PullPush and Reddit .json are both - * unauthenticated. We just check if either endpoint is responding. + * No credentials needed — RSS and .json are both unauthenticated. + * We just check if either endpoint is responding. * - * Side effects: HTTP requests to PullPush and/or Reddit. + * Side effects: HTTP requests to Reddit. * * @return bool True if at least one data source is available. */ public function validate_credentials(): bool { - $pullpush = new PRAutoBlogger_PullPush_Client(); - if ( $pullpush->is_available() ) { - return true; - } - - $reddit_json = new PRAutoBlogger_Reddit_JSON_Client(); - return $reddit_json->is_available(); + $client = new PRAutoBlogger_Reddit_JSON_Client(); + return $client->is_available(); } /** - * Get current rate limit status from the active data source. + * Get current rate limit status from the Reddit client. * * @return array{remaining: int, limit: int, resets_at: string} */ public function get_rate_limit_status(): array { - if ( $this->is_pullpush_available() ) { - $client = new PRAutoBlogger_PullPush_Client(); - return $client->get_rate_limit_status(); - } - $client = new PRAutoBlogger_Reddit_JSON_Client(); return $client->get_rate_limit_status(); } - /** - * Collect data from PullPush.io API. - * - * PullPush returns flat arrays of Reddit objects with standard field names. - * We can search comments subreddit-wide (more efficient than per-post). - * - * @param string $subreddit Subreddit name. - * @param int $limit Max posts. - * @param string $time_filter Time window. - * @param bool $include_comments Whether to fetch comments. - * @param int $comments_limit Max comments per post. - * - * @return PRAutoBlogger_Source_Data[] - */ - private function collect_from_pullpush( - string $subreddit, - int $limit, - string $time_filter, - bool $include_comments, - int $comments_limit - ): array { - $client = new PRAutoBlogger_PullPush_Client(); - $collected = []; - - // Fetch posts. - $posts = $client->fetch_posts( $subreddit, $limit, $time_filter ); - - foreach ( $posts as $post ) { - $collected[] = new PRAutoBlogger_Source_Data( [ - 'source_type' => 'reddit', - 'source_id' => 't3_' . ( $post['id'] ?? '' ), - 'subreddit' => $subreddit, - 'title' => $post['title'] ?? '', - 'content' => $post['selftext'] ?? '', - 'author' => $post['author'] ?? '[deleted]', - 'score' => (int) ( $post['score'] ?? 0 ), - 'comment_count' => (int) ( $post['num_comments'] ?? 0 ), - 'permalink' => 'https://reddit.com' . ( $post['permalink'] ?? '' ), - 'collected_at' => current_time( 'mysql' ), - 'metadata' => [ - 'post_type' => ( $post['is_self'] ?? false ) ? 'self' : 'link', - 'flair' => $post['link_flair_text'] ?? null, - 'created_utc' => $post['created_utc'] ?? null, - 'upvote_ratio' => $post['upvote_ratio'] ?? null, - 'is_original' => $post['is_original_content'] ?? false, - 'data_source' => 'pullpush', - ], - ] ); - - // Fetch comments for high-engagement posts. - if ( $include_comments && ( $post['num_comments'] ?? 0 ) > 0 ) { - $post_id = $post['id'] ?? ''; - $comments = $client->fetch_comments_for_post( $post_id, $comments_limit ); - - foreach ( $comments as $comment ) { - $collected[] = new PRAutoBlogger_Source_Data( [ - 'source_type' => 'reddit', - 'source_id' => 't1_' . ( $comment['id'] ?? '' ), - 'subreddit' => $subreddit, - 'title' => null, - 'content' => $comment['body'] ?? '', - 'author' => $comment['author'] ?? '[deleted]', - 'score' => (int) ( $comment['score'] ?? 0 ), - 'comment_count' => 0, - 'permalink' => 'https://reddit.com' . ( $comment['permalink'] ?? '' ), - 'collected_at' => current_time( 'mysql' ), - 'metadata' => [ - 'parent_post_id' => 't3_' . $post_id, - 'parent_title' => $post['title'] ?? '', - 'is_comment' => true, - 'created_utc' => $comment['created_utc'] ?? null, - 'data_source' => 'pullpush', - ], - ] ); - } - } - } - - return $collected; - } - - /** - * Collect data from Reddit .json endpoints (fallback). - * - * Response format matches the old OAuth API: data.children[].data. - * Per-post comment fetching (not subreddit-wide like PullPush). - * - * @param string $subreddit Subreddit name. - * @param int $limit Max posts. - * @param string $time_filter Time window. - * @param bool $include_comments Whether to fetch comments. - * @param int $comments_limit Max comments per post. - * - * @return PRAutoBlogger_Source_Data[] - */ - private function collect_from_reddit_json( - string $subreddit, - int $limit, - string $time_filter, - bool $include_comments, - int $comments_limit - ): array { - $client = new PRAutoBlogger_Reddit_JSON_Client(); - $collected = []; - - // Fetch posts — same response format as old OAuth API. - $posts = $client->fetch_posts( $subreddit, $limit, $time_filter ); - - foreach ( $posts as $post ) { - $collected[] = new PRAutoBlogger_Source_Data( [ - 'source_type' => 'reddit', - 'source_id' => 't3_' . ( $post['id'] ?? '' ), - 'subreddit' => $subreddit, - 'title' => $post['title'] ?? '', - 'content' => $post['selftext'] ?? '', - 'author' => $post['author'] ?? '[deleted]', - 'score' => (int) ( $post['score'] ?? 0 ), - 'comment_count' => (int) ( $post['num_comments'] ?? 0 ), - 'permalink' => 'https://reddit.com' . ( $post['permalink'] ?? '' ), - 'collected_at' => current_time( 'mysql' ), - 'metadata' => [ - 'post_type' => ( $post['is_self'] ?? false ) ? 'self' : 'link', - 'flair' => $post['link_flair_text'] ?? null, - 'created_utc' => $post['created_utc'] ?? null, - 'upvote_ratio' => $post['upvote_ratio'] ?? null, - 'is_original' => $post['is_original_content'] ?? false, - 'data_source' => 'reddit_json', - ], - ] ); - - // Collect top comments for high-engagement posts. - if ( $include_comments && ( $post['num_comments'] ?? 0 ) > 0 ) { - $post_id = $post['id'] ?? ''; - $comments = $client->fetch_comments( $subreddit, $post_id, $comments_limit ); - - foreach ( $comments as $comment ) { - $collected[] = new PRAutoBlogger_Source_Data( [ - 'source_type' => 'reddit', - 'source_id' => 't1_' . ( $comment['id'] ?? '' ), - 'subreddit' => $subreddit, - 'title' => null, - 'content' => $comment['body'] ?? '', - 'author' => $comment['author'] ?? '[deleted]', - 'score' => (int) ( $comment['score'] ?? 0 ), - 'comment_count' => 0, - 'permalink' => 'https://reddit.com' . ( $comment['permalink'] ?? '' ), - 'collected_at' => current_time( 'mysql' ), - 'metadata' => [ - 'parent_post_id' => 't3_' . $post_id, - 'parent_title' => $post['title'] ?? '', - 'is_comment' => true, - 'created_utc' => $comment['created_utc'] ?? null, - 'data_source' => 'reddit_json', - ], - ] ); - } - } - } - - return $collected; - } - - /** - * Check if PullPush.io is currently available, with caching. - * - * Caches the result for 5 minutes to avoid re-checking availability - * on every subreddit within a single collection run. - * - * Side effects: May make HTTP request, sets transient. - * - * @return bool True if PullPush is reachable. - */ - private function is_pullpush_available(): bool { - $cached = get_transient( self::PULLPUSH_AVAILABLE_TRANSIENT ); - - if ( false !== $cached ) { - return '1' === $cached; - } - - $client = new PRAutoBlogger_PullPush_Client(); - $available = $client->is_available(); - - set_transient( - self::PULLPUSH_AVAILABLE_TRANSIENT, - $available ? '1' : '0', - self::AVAILABILITY_CACHE_TTL - ); - - if ( ! $available ) { - PRAutoBlogger_Logger::instance()->warning( - 'PullPush.io unavailable — falling back to Reddit .json endpoints.', - 'reddit' - ); - } - - return $available; - } - /** * Get the list of subreddits configured in plugin settings. * diff --git a/prautoblogger.php b/prautoblogger.php index c182ea6..c85e55b 100644 --- a/prautoblogger.php +++ b/prautoblogger.php @@ -8,7 +8,7 @@ declare(strict_types=1); * Plugin Name: PRAutoBlogger * Plugin URI: https://peptiderepo.com/prautoblogger * Description: Monitors social media for trending topics, generates SEO-friendly blog posts using AI, and publishes them on a daily schedule with full cost tracking and self-improvement metrics. - * Version: 0.2.0 + * Version: 0.2.1 * Requires at least: 6.0 * Requires PHP: 7.4 * Author: PeptideRepo diff --git a/readme.txt b/readme.txt index 78ac1e3..f90928a 100755 --- a/readme.txt +++ b/readme.txt @@ -35,7 +35,7 @@ Uses OpenRouter for access to models from Anthropic, OpenAI, Google, Meta, and m 3. Go to PRAutoBlogger > Settings to configure your API key and preferences. 4. Add your OpenRouter API key. 5. Configure your target subreddits and content preferences. -6. Click "Test Connections" to verify setup (Reddit sources use PullPush.io — no API key needed). +6. Click "Test Connections" to verify setup (Reddit sources use RSS — no API key needed). 7. Click "Generate Now" for your first article, or wait for the daily schedule. == Frequently Asked Questions == @@ -50,14 +50,14 @@ By default, articles go through the chief editor agent. Articles that pass revie = Do I need a Reddit API key? = -No. PRAutoBlogger uses PullPush.io as its primary data source for Reddit content, with Reddit .json endpoints as a fallback. Both are free and require no authentication. You only need an OpenRouter API key for the AI models. +No. PRAutoBlogger uses Reddit RSS feeds as its primary data source, with Reddit .json endpoints as a fallback. Both are free and require no authentication. You only need an OpenRouter API key for the AI models. == Changelog == = 0.2.0 = -* Replaced Reddit OAuth with PullPush.io (primary) and Reddit .json (fallback) — no API key needed. +* Reddit data source uses RSS feeds (primary) with .json endpoints as fallback — no API key needed. * Removed Reddit Client ID/Secret fields from admin settings. -* Added source status indicator showing PullPush and fallback availability. +* Added source status indicator showing RSS and .json fallback availability. * Added configurable research cache TTL, time window, and posts-per-subreddit settings. * Updated Test Connections to show which Reddit source is active. * Fixed deployment pipeline to purge LiteSpeed cache after deploy.