--- name: woocommerce-store-scraper description: Read the public catalogue of any WooCommerce store as structured records via the Apify Actor arman-bd/woocommerce-store-scraper. Returns one row per product with name, slug, permalink, current, regular and sale prices as real decimals in the store's currency, stock status, category and tag slugs, attributes with their terms, variation ids and attribute values, rating aggregates, image URLs and a plain-text description. Use when a task needs price monitoring across independent retailers, a catalogue index, stock availability checks or category and price-band research. Not for orders, customers, inventory counts, checkout, or stores on Shopify, BigCommerce or Squarespace. --- # WooCommerce Store Scraper Apify Actor `arman-bd/woocommerce-store-scraper`. Give it a list of store domains, get one dataset record per public product. It takes no credentials of your own. This reads the public storefront catalogue that the shop already serves to visitors, not the authenticated administration interface. ## When to use it - Monitoring competitor pricing across many small independent retailers on a schedule. - Building a product comparison index from several stores at once. - Sourcing or dropshipping research, watching `stockStatus` for what is actually available. - Measuring catalogue size, price bands and category mix across a niche. - Any of the above repeatedly, diffing on `productId` plus `priceCurrent` to build a price history. ## When not to use it - Orders, customers, carts, coupons or anything behind a login. None of it is reachable. - Exact inventory counts. See the note on `stockQuantity` below. - Stores on Shopify, BigCommerce, Squarespace or a bespoke platform. Those are separate Actors; this one only reads WooCommerce. - Placing an order or interacting with checkout in any way. ## Call it ```js import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('arman-bd/woocommerce-store-scraper').call({ stores: ['barefootbuttons.com', 'https://woocommerce.com/products/'], categories: ['accessories'], inStockOnly: true, maxProductsPerStore: 200, }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); const { value: summary } = await client .keyValueStore(run.defaultKeyValueStoreId) .getRecord('RUN_SUMMARY'); ``` One-shot over HTTP, when you want the rows back in the same request: ```bash curl -X POST "https://api.apify.com/v2/acts/arman-bd~woocommerce-store-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{"stores":["barefootbuttons.com"],"inStockOnly":true,"maxProductsPerStore":100}' ``` The Actor is also exposed through Apify's MCP server as `arman-bd/woocommerce-store-scraper`, so an MCP-capable agent can call it with no extra wiring. ## Input | Field | Type | Required | Default | Notes | |---|---|---|---|---| | `stores` | string[] | yes | | Domains or URLs. A bare domain gets `https://`, a `www.` host is kept as given, and any path is discarded, so pasting a deep product URL works. A value with no dot in the hostname is dropped. Deduplicated by origin. | | `categories` | string[] | no | `[]` | Category slugs, as in the store's own category URL, or numeric term IDs. Multiple values are OR-ed together. | | `searchQuery` | string | no | `""` | Full-text search run by the store itself, over names and descriptions. | | `maxProductsPerStore` | integer | no | `0` | Cap per store, minimum `0`. `0` means the whole catalogue. Applied after the store has filtered. | | `inStockOnly` | boolean | no | `false` | Ask the store to return only products currently in stock. | **All three filters run on the store's side, so narrowing makes a run faster as well as smaller.** `categories`, `searchQuery` and `inStockOnly` combine with AND and are sent with the request, which means the store never sends you the rows you excluded. Products arrive 100 per page and paging stops as soon as `maxProductsPerStore` is reached, so a cap genuinely saves requests here rather than merely truncating a payload. For a broad sweep across many stores, cap first and widen later. ## Output One record per product returned. | Field | Type | Notes | |---|---|---| | `store` | string | The store origin this row came from, for example `https://barefootbuttons.com`. Join back to your input list on this. | | `productId` | number | The store's product ID. Unique within a store, not across stores. Key on `store` plus `productId`. | | `name` | string | Product name, HTML entities decoded. | | `slug` | string \| null | URL slug. | | `permalink` | string \| null | Public product page. | | `priceCurrent` | number \| null | The price being charged now, as a real decimal in `currency`. | | `priceRegular` | number \| null | The undiscounted price. | | `priceSale` | number \| null | The sale price, only when `onSale` is true. `null` otherwise, even if the store carries a stale sale figure. | | `currency` | string \| null | ISO currency code of the store. | | `onSale` | boolean | Whether the store is currently discounting. | | `stockStatus` | string | `instock`, `outofstock` or `onbackorder`. Always present. | | `stockQuantity` | number \| null | Units remaining, and almost always `null`. See below. | | `categories` | string[] | Category slugs. Empty array when uncategorised. | | `tags` | string[] | Tag slugs. | | `attributes` | object[] | `{ name, taxonomy, terms }`, where `terms` is the list of available values. `taxonomy` is `null` for custom, non-global attributes. | | `variations` | object[] | `{ id, attributes: [{ name, value }] }`. `value` can be `null` for an unconfigured variation attribute. Empty array for simple products. | | `averageRating` | number \| null | `null` when the store publishes no rating at all, `0` when it publishes one and there are no reviews. | | `reviewCount` | number | Defaults to `0`. | | `images` | string[] | Full-size image URLs. | | `descriptionPlain` | string | Description as plain text: entities decoded, tags stripped, list items turned into bullets. Falls back to the short description when the long one is empty. `""` when the product has neither. | | `scrapedAt` | string | Run timestamp, ISO 8601. | A real record, description trimmed: ```json { "store": "https://barefootbuttons.com", "productId": 1475, "name": "Replacement Set Screw and Allen Wrench Kit", "slug": "replacement-set-screw-and-allen-wrench-kit", "permalink": "https://barefootbuttons.com/product/replacement-set-screw-and-allen-wrench-kit/", "priceCurrent": 1.95, "priceRegular": 1.95, "priceSale": null, "currency": "USD", "onSale": false, "stockStatus": "instock", "stockQuantity": null, "categories": ["accessories"], "tags": [], "attributes": [ { "name": "Set Screw Size", "taxonomy": null, "terms": ["For V1/V2 Mini", "For Big Bore Standard"] } ], "variations": [ { "id": 1485, "attributes": [{ "name": "Set Screw Size", "value": "For V1/V2 Mini" }] } ], "averageRating": 0, "reviewCount": 0, "images": ["https://barefootbuttons.com/wp-content/uploads/2026/06/set-screws-allen-wrench-prodcut.png"], "descriptionPlain": "Every Barefoot Button includes the hardware needed for installation…", "scrapedAt": "2026-08-06T12:00:00.000Z" } ``` ## RUN_SUMMARY Written to the run's key-value store under the key `RUN_SUMMARY`. **Read it.** It is where a partial run admits that it was partial. ```json { "storesRequested": 2, "storesFailed": 1, "failures": [ { "store": "https://example.com", "error": "Store API not found (404) …" } ], "productsSaved": 412, "filters": { "categories": ["accessories"], "searchQuery": "", "maxProductsPerStore": 200, "inStockOnly": true }, "finishedAt": "2026-08-06T12:00:07.884Z" } ``` `storesFailed` above zero means whole stores are missing from the dataset, and the `error` string says which kind of miss it was: a site that does not expose the public catalogue at all, one that refused the request, or a transient fault that outlived its retries. A store that succeeded but contributed no rows is a different situation, and it means your filters matched nothing there; the per-store matched count is printed in the run log rather than in the summary, so re-run with the filters cleared if you need to tell an empty catalogue from an over-narrow query. ## Behaviour to plan around - **Prices are real decimals, converted from the store's minor unit.** The underlying values are integer strings such as `"5900"` with a separate decimal-places field, and the Actor divides that out, so `priceCurrent` is `59` rather than `5900`. Do not apply a second conversion. - **`stockQuantity` is `null` almost everywhere, and that is honest.** The public catalogue never publishes an exact inventory figure. The only quantity available is a low-stock remainder, which a store fills in only once its own threshold is crossed. A number here is real; `null` means the store published none. Nothing is estimated, so use `stockStatus` for availability. - **`productId` is unique per store only.** Two stores can both have product 1475. Key on `store` plus `productId` in any cross-store table. - **A 200 response is not proof of a catalogue.** Many hosts answer the catalogue path with a themed HTML error page at status 200, so the content type is checked and a non-JSON response is treated as "no public catalogue here" rather than as data. That store lands in `failures`. - **Refusals and missing catalogues are final.** A not-found, a refusal and a non-JSON body all fail immediately without retries, because retrying cannot change them. Transient faults get three attempts with exponential backoff and a 30-second timeout. - **Paging is read, not guessed.** The number of pages comes from the response itself, 100 products at a time, so a run does not have to page until it hits an empty page. - **One bad store never aborts the run.** The Actor only throws when every store failed. - **`priceSale` is deliberately blanked when `onSale` is false,** so a store's leftover sale price cannot be mistaken for an active discount. Compare `priceCurrent` against `priceRegular` if you want the effective discount. - **Variation-level prices are not returned per variation.** `variations` carries each variation's id and attribute values; a variable product's price fields reflect the range the store publishes on the parent. - **`averageRating` distinguishes absent from zero.** `null` means the store published no rating field; `0` means it did and nobody has reviewed the product. - **Any path you paste is discarded.** The catalogue is always read from the site root, so a category page URL will not narrow the run. Use `categories` for that. ## Recipes **Price monitoring across a competitor set.** Whole catalogues, run daily. ```json { "stores": ["barefootbuttons.com", "woocommerce.com"] } ``` Diff `priceCurrent`, `priceRegular` and `onSale` against the previous run, keyed on `store` plus `productId`. A `productId` that disappears has been unpublished. **One category, in stock, capped.** Filters run on the store's side, so this is the fast shape. ```json { "stores": ["barefootbuttons.com"], "categories": ["accessories"], "inStockOnly": true, "maxProductsPerStore": 200 } ``` Category slugs come from the store's own category URLs; numeric term IDs work too. **Search a niche across many stores.** Small cap per store so one large catalogue does not dominate. ```json { "stores": ["store-a.com", "store-b.com", "store-c.com"], "searchQuery": "hoodie", "maxProductsPerStore": 50 } ``` Group by `store` and compare `priceCurrent` for the same product name to find the cheapest supplier. **Availability watch.** Cheapest useful repeat run. ```json { "stores": ["barefootbuttons.com"], "inStockOnly": false, "maxProductsPerStore": 0 } ``` Track `stockStatus` transitions per `productId`. Leaving `inStockOnly` off is deliberate: you need the out-of-stock rows to see something come back.