Replace batchexecute API with direct fetch + HTML parsing strategies:
data-n-au attributes, anchor href extraction, meta refresh, JS redirect.
Add verbose debug output to diagnose server-side response from Google News.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace broken base64 decoding with Google's internal batchexecute API
endpoint to resolve encrypted article IDs to actual publisher URLs.
This is the approach used by known Google News decoder libraries.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Temporary debug version that returns query counts, URL resolution results,
and OG scraping outcomes to identify why thumbnail scraping returns 0.
Also adds Google News URL decoder (base64/protobuf approach).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The OG image scraper was using Google News redirect URLs (source_url) directly,
which don't serve og:image tags. Added resolve_scrape_url() to follow redirects
(including meta-refresh and JS redirects) to reach the actual publisher page
before scraping. Also added a backfill AJAX endpoint and admin button to
re-scrape thumbnails for existing articles.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Google News RSS feeds report every article as coming from
"news.google.com" because the source was extracted from the feed URL
rather than the individual article. This adds a multi-strategy
resolver that extracts the real publisher name via:
1. SimplePie <source> element (Google News provides this)
2. Title suffix parsing ("Headline - Source" format)
3. HTTP HEAD redirect resolution as a fallback
Also adds a one-click "Fix Article Sources" backfill button on the
settings page to correct existing articles in the database.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two-tier filtering during article fetch: keyword/pattern rules as a fast
first pass, then optional LLM classification for borderline cases via
OpenRouter. Configurable sensitivity (strict/moderate/lenient), editable
keyword lists, blocked domain list, and admin stats display. Includes
30 unit tests covering domain blocking, title/body keywords, URL paths,
edge cases, and prompt structure.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use $wpdb->prepare() with esc_like() in clear_article_cache()
- Add wp_http_validate_url() SSRF prevention in download_thumbnail()
- Add limit_response_size (5MB) on image downloads
- Set chmod 0644 after file_put_contents()
- Add transient-based concurrent fetch lock
