--- name: ghost-blog-scraper description: Pull posts from any Ghost-powered publication via the Apify Actor arman-bd/ghost-blog-scraper. Returns one record per post with title, excerpt, full body as clean plain text, tags, authors, feature image, publish date, reading time and canonical URL. Use when a task needs to track independent publishers and engineering blogs, archive a publication, or build a text corpus across several Ghost sites, optionally with a Content API key for the full archive. Not for members-only or paid posts, comments, newsletter subscribers, or sites running other blog software. --- # Ghost Blog Scraper: Posts, Tags & Authors Apify Actor `arman-bd/ghost-blog-scraper`. Give it Ghost site addresses, get one dataset record per post, body included as readable text rather than markup. It runs on the public feed with nothing supplied. Add a Content API key and it reads the site's full archive instead of the recent window. ## When to use it - Watching a shortlist of independent blogs and newsletters for new posts. - Archiving one publication end to end, with a Content API key. - Building a text corpus across several Ghost sites for search or embedding. - Topic research: one tag filter applied across many publications in a single run. - Cheap change detection, with bodies off and `postId` plus `publishedAt` diffed between runs. ## When not to use it - Members-only or paid posts. They are not in the public feed and not in the public Content API scope, and nothing here attempts to reach them. - Comments, reactions, subscriber lists or any member data. - Sites that are not Ghost. Other platforms are identified and reported as failures, never parsed on a best-effort basis. - Fetching a single known post URL. The Actor reads publications, not individual pages. ## Call it ```js import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('arman-bd/ghost-blog-scraper').call({ sites: ['blog.codinghorror.com', 'blog.duolingo.com'], tags: ['engineering'], maxPostsPerSite: 10, includeContent: true, }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); const { value: summary } = await client .keyValueStore(run.defaultKeyValueStoreId) .getRecord('RUN_SUMMARY'); ``` One-shot over HTTP, when you want the rows back in the same request: ```bash curl -X POST "https://api.apify.com/v2/acts/arman-bd~ghost-blog-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{"sites":["blog.codinghorror.com"],"maxPostsPerSite":10,"includeContent":false}' ``` The Actor is also exposed through Apify's MCP server as `arman-bd/ghost-blog-scraper`, so an MCP-capable agent can call it with no extra wiring. ## Input | Field | Type | Required | Default | Notes | |---|---|---|---|---| | `sites` | string[] | yes | | Domains or site URLs. A feed URL is reduced to the site root, trailing slashes are trimmed, and a publication on a sub-path keeps that path. Duplicates collapse on the resolved origin. Entries that are not usable addresses are dropped; if none survive, the run throws. | | `contentApiKey` | string | no | `""` | A Ghost Content API key, 26 hex characters, read-only. Switches the whole run to the Content API: full archive, real `updatedAt`, Ghost's own `readingTime`. Keys belong to one site. Secret field. Do not pass an Admin key; it has a different shape and is not accepted. | | `tags` | string[] | no | `[]` | Case-insensitive substring match against tag names, and against tag slugs on the Content API path. `engineer` matches `Engineering`. Empty keeps every post. | | `maxPostsPerSite` | integer | no | `0` | Cap per site, applied **after** tag filtering. `0` means no cap. Minimum `0`. | | `includeContent` | boolean | no | `true` | Include `contentPlain`. Off gives a much smaller dataset with every other field intact. | **The key decides both reach and batch size, and it is one decision, not two.** Without one you see only what the public feed carries, which on a default install is the fifteen or so most recent posts, so `maxPostsPerSite: 500` quietly returns fifteen. With one you reach the entire archive, but a key authenticates a single site, so every other site in the same run fails. Batch freely when you have no key, run one site per job when you do. Note also that a `tags` filter forces the source to be read out in full before the cap is applied, because the cap counts matches: on a large archive that is the expensive combination. ## Output One record per post that survived the tag filter and the cap. | Field | Type | Notes | |---|---|---| | `site` | string | The **hostname** the post came from, so a sub-path publication reports the bare host. Join multi-site runs on this. | | `postId` | string \| null | Ghost's post id, a 24-character hex ObjectID, stable across edits. Taken from the feed entry identifier on the public path. | | `title` | string \| null | Post title, emoji included. | | `slug` | string \| null | URL slug. Derived from the post URL on the public path. | | `excerpt` | string \| null | The site's own summary, plain text. Prefers a custom excerpt when the Content API supplies one. | | `contentPlain` | string | **Key is absent entirely when `includeContent` is false.** Full body as plain text: markup stripped first, then entities decoded. | | `featureImage` | string \| null | Feature image URL, falling back to the first image in the body. | | `tags` | string[] | Tag display names. Slugs are matched against but are not returned. | | `authors` | string[] | Credited author names. | | `publishedAt` | string \| null | ISO 8601 UTC, normalised from whatever the source formatted. | | `updatedAt` | string \| null | Last edit time. **Always `null` without a Content API key.** | | `readingTime` | number | Minutes. Ghost's own figure on the Content API path, otherwise derived from the word count at 275 words per minute. `0` for an empty body. | | `url` | string \| null | Canonical post URL. | | `scrapedAt` | string | Run timestamp, ISO 8601. | A real record from a public-feed run, long strings trimmed: ```json { "site": "blog.duolingo.com", "postId": "6a7091b92083b20001810ddd", "title": "Making production-ready agents the default: building Duolingo's agent platform", "slug": "production-ready-ai-agent-platform", "excerpt": "See how we made AI agents easy to build, run, and improve in production.", "contentPlain": "TL;DR\nAt Duolingo, teams were repeatedly rebuilding the same infrastructure around AI agents. We solved this by …", "featureImage": "https://storage.ghost.io/c/7a/33/7a33d0f4-927d-4fe8-a6bf-96131b5e76d4/content/images/2026/08/cover.png", "tags": ["Life at Duolingo", "Engineering"], "authors": ["Guadalupe Aliseda-Canton"], "publishedAt": "2026-08-04T12:00:00.000Z", "updatedAt": null, "readingTime": 7, "url": "https://blog.duolingo.com/production-ready-ai-agent-platform/", "scrapedAt": "2026-08-06T11:52:06.331Z" } ``` ## RUN_SUMMARY Written to the run's key-value store under the key `RUN_SUMMARY`. **Read it.** It is where a partial run admits that it was partial. ```json { "sitesRequested": 3, "sitesFailed": 1, "failures": [ { "site": "www.taniarascia.com", "error": "not a Ghost site, GatsbyJS" } ], "postsSaved": 30, "source": "rss", "perSite": [ { "site": "blog.codinghorror.com", "source": "rss", "listed": 15, "matched": 15, "saved": 15, "feedExhaustedAt": 15 }, { "site": "blog.duolingo.com", "source": "rss", "listed": 15, "matched": 15, "saved": 15, "feedExhaustedAt": 15 } ], "filters": { "tags": [], "maxPostsPerSite": 0, "includeContent": true }, "finishedAt": "2026-08-06T11:52:06.331Z" } ``` `perSite` is the interesting part. `listed` is what the source offered, `matched` is what survived `tags`, `saved` is what the cap allowed through, so a low row count is always attributable to one of the three. `feedExhaustedAt` is a number when the public feed ran out before your cap was satisfied: that is the signal that a Content API key, not a bigger cap, is what you needed. `source` at the top says which path the run took overall, and each `perSite` entry repeats it. `failures` names the site and the reason, most often that it is running something other than Ghost. ## Behaviour to plan around - **`maxPostsPerSite` cannot exceed the public feed window.** Without a key you get roughly the fifteen most recent posts per site however large the number. Check `perSite[].feedExhaustedAt` before concluding a blog has stopped publishing. - **One key, one site.** A run with a key applies it to every site listed, so the others come back in `failures` with a rejection. Split the job. - **`contentPlain` is absent, not empty, when `includeContent` is false.** Test for the key rather than for a falsy value. - **`updatedAt` is null on the public path.** It is not "never edited", it is "not reported". Only a Content API run distinguishes the two. - **`tags` returns names, filtering also sees slugs.** With a key, `tags: ["ai"]` can match a post whose visible tag is `Artificial Intelligence` through its slug. That is a substring match, so short terms over-match: `art` catches `Artificial`. - **The cap runs after the filter.** `tags` plus `maxPostsPerSite: 10` gives ten matching posts, which means the whole archive is read first when a key is present. - **`site` is the hostname, not what you typed.** A publication under a sub-path reports the bare host, so several publications on one domain are not distinguishable by this field alone. Use `url` for that. - **A site whose platform cannot be confirmed is rejected, not guessed at.** The failure message names what was found instead. One bad site never aborts the run; the Actor throws only when every site failed. - **Transient failures are retried** three times with linear backoff. Not-found and other client errors are final. ## Recipes **Hourly watch on a shortlist.** Bodies off, so the dataset stays small. ```json { "sites": ["blog.codinghorror.com", "blog.duolingo.com", "blog.ghost.org"], "includeContent": false } ``` Diff on `postId`, then re-run with `includeContent: true` for the ids that are new. **Archive one publication.** A key, one site, everything. ```json { "sites": ["demo.ghost.io"], "contentApiKey": "22444f78447824223cefc48062", "includeContent": true, "maxPostsPerSite": 0 } ``` `updatedAt` is populated here, so re-runs can be reconciled against your copy. **Topic research across publishers.** One tag, several sites, capped per site. ```json { "sites": ["blog.duolingo.com", "blog.codinghorror.com"], "tags": ["engineering"], "maxPostsPerSite": 20, "includeContent": true } ``` Group by `site` and sort by `publishedAt` to see who is writing about what, and when. **Feed a retrieval index.** Titles, bodies and canonical links, no cap. ```json { "sites": ["blog.ghost.org"], "includeContent": true } ``` Chunk `contentPlain`, cite `url`, and keep `readingTime` as a cheap length proxy for ranking.