--- name: wikidata-entities-scraper description: Read Wikidata entities as flattened structured records via the Apify Actor arman-bd/wikidata-entities-scraper. Returns the Q-identifier, label, description and aliases in a language chain, the entity type, every property claim with its human-readable label, datatype and unwrapped values, statement qualifiers with rank, the sitelink map, the Wikipedia URL and the last edit timestamp. Use for entity resolution from names to stable identifiers, knowledge-graph extraction, attaching external identifiers such as VIAF or IMDb to your own records, and building a licence-clean fact layer for agents. Not for Wikipedia article text, SPARQL graph queries, edit history or anything requiring a signed-in account. --- # Wikidata Scraper: Structured Entity Facts & Claims Apify Actor `arman-bd/wikidata-entities-scraper`. Give it Q-identifiers, or names to resolve first, and get one dataset record per entity with the five-level-deep claim structure flattened into something usable. It runs without credentials. ## When to use it - Turning free-text names into stable Q-identifiers, then into canonical facts. - Extracting typed facts at scale: birth dates, coordinates, parent companies, populations. - Attaching external identifiers (VIAF, ISNI, GRID, IMDb, MusicBrainz and thousands more) to records you already hold. - Building a fact layer for an agent or a RAG index where provenance and licence matter. - Change detection over a watchlist of entities, diffing on `modified`. ## When not to use it - Wikipedia prose. You get the article URL, never the article body. - Graph traversal or aggregate queries across millions of entities. This fetches the entities you name; a query service is the right tool for "all cities over 1M people". - Edit history, contributor data or anything account-scoped. - Free-text search over article content. Search here matches labels and aliases only. ## Call it ```js import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('arman-bd/wikidata-entities-scraper').call({ searchTerms: ['Douglas Adams', 'Ada Lovelace'], entityIds: ['Q42'], properties: ['P31', 'P569', 'P570', 'P106'], languages: ['en'], resolveLabels: true, includeSitelinks: false, resultsPerSearchTerm: 1, }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); const { value: summary } = await client .keyValueStore(run.defaultKeyValueStoreId) .getRecord('RUN_SUMMARY'); ``` One-shot over HTTP, when you want the rows back in the same request: ```bash curl -X POST "https://api.apify.com/v2/acts/arman-bd~wikidata-entities-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{"entityIds":["Q42"],"properties":["P31","P569","P106"],"includeSitelinks":false}' ``` The Actor is also exposed through Apify's MCP server as `arman-bd/wikidata-entities-scraper`, so an MCP-capable agent can call it with no extra wiring. ## Input | Field | Type | Required | Default | Notes | |---|---|---|---|---| | `entityIds` | string[] | no | `[]` | Identifiers such as `Q42`, case-insensitive. A `wikidata.org/wiki/…` URL, an entity URI or a `Property:P31` form are all accepted. Anything that is not an identifier is quietly treated as a search term instead of failing. `L` and `P` identifiers work alongside `Q` items. | | `searchTerms` | string[] | no | `[]` | Names resolved to entities before fetching. Matched against labels and aliases in the first language you request. | | `properties` | string[] | no | `[]` | Keep only these P-properties in `claims` and `qualifiers`. Written as `P31` or `p31`; anything else is ignored. Empty returns every claim on the entity. | | `languages` | string[] | no | `["en"]` | Fallback chain for label, description and aliases, most preferred first. The first language with a value wins, and the shared multilingual bucket is always appended for you. | | `resolveLabels` | boolean | no | `true` | Fill in human-readable labels for every referenced property and item, so `P31` also carries "instance of" and `Q5` also carries "human". Costs extra batched requests, capped at 2000 distinct lookups per run and cached across the whole run. | | `includeSitelinks` | boolean | no | `true` | Emit the full site-to-URL map. A popular entity carries more than 300 sitelinks, so this dominates record size. | | `resultsPerSearchTerm` | integer | no | `1` | Entities kept per search term, 1 to 20. `1` takes the best match; raise it when names are ambiguous and de-duplicate yourself. | At least one of `entityIds` and `searchTerms` must resolve to something, otherwise the run throws immediately. **The two flags decide whether the output is readable or compact.** With `resolveLabels` on, every claim and every referenced item carries a label, which is what makes a record legible to a model without a second lookup pass; off, you get bare identifiers and a much faster run. With `includeSitelinks` on, the sitelink map is often larger than everything else in the record combined. Note that `wikipediaUrl` is derived from sitelinks and is present either way, so turning sitelinks off never costs you the Wikipedia link. The other lever is `properties`: filtering to the four or five P-ids you actually consume turns a 200-claim entity into a small row and cuts label resolution proportionally. ## Output One record per entity that resolved. | Field | Type | Notes | |---|---|---| | `entityId` | string | The canonical identifier, for example `Q42`. The join key. | | `label` | string \| null | Preferred label, taken from the first language in the chain that has one. | | `description` | string \| null | Short description, same chain. | | `aliases` | string[] | Alternative names from the first language bucket that has any. `[]` when none. | | `instanceOf` | object[] | The entity's type as `[{ id, label }]`, lifted out of the `P31` claim for convenience. `[]` when the entity has no `P31`, and also `[]` when your `properties` filter excluded `P31`. | | `claims` | object | Keyed by P-identifier. Each value is `{ label, datatype, values }`, where `values` is an array of unwrapped values. Always present, `{}` when the filter matched nothing. | | `qualifiers` | object | Keyed by P-identifier, but only for statements that carry qualifiers. Each entry is a list of `{ value, rank, qualifiers }`, with the inner qualifiers flattened the same way. `{}` when none. | | `sitelinks` | object | Site-to-URL map, for example `enwiki`. The key is absent from the record entirely when `includeSitelinks` is off. | | `wikipediaUrl` | string \| null | Wikipedia link in the first requested language, falling back to English. Present regardless of `includeSitelinks`. | | `modified` | string \| null | Last edit timestamp of the entity, ISO 8601. Diff on this for cheap change detection. | | `scrapedAt` | string | Run timestamp, ISO 8601. | Claim values are unwrapped per datatype, so downstream code never has to walk the raw nesting: - An item or property reference becomes `{ id, label }`, with `label` filled in only when `resolveLabels` is on. - A time becomes `{ time, precision, calendar, raw }`, trimmed to the stated precision, so a year-precision date stays `"1952"` instead of pretending to be 1 January. - A quantity becomes `{ amount, unit, lowerBound, upperBound }`, where `amount` is a real number and `unit` is an `{ id, label }` reference or `null`. - A coordinate becomes `{ latitude, longitude, altitude, precision, globe }`, and `globe` other than `Q2` means the point is not on Earth. - Monolingual text becomes `{ text, language }`. - A media file becomes `{ file, url }`, with a resolvable URL rather than a bare filename. - Plain strings, URLs and external identifiers stay plain strings. - A statement asserting no value or an unknown value becomes `{ "special": "novalue" }` or `{ "special": "somevalue" }`, preserved rather than flattened to `null`. A real record, claims and sitelinks trimmed: ```json { "entityId": "Q42", "label": "Douglas Adams", "description": "British science fiction writer and humorist (1952-2001)", "aliases": ["Douglas Noël Adams", "Douglas N. Adams"], "instanceOf": [{ "id": "Q5", "label": "human" }], "claims": { "P31": { "label": "instance of", "datatype": "wikibase-item", "values": [{ "id": "Q5", "label": "human" }] }, "P569": { "label": "date of birth", "datatype": "time", "values": [{ "time": "1952-03-11", "precision": "day", "calendar": "Gregorian", "raw": "+1952-03-11T00:00:00Z" }] }, "P2048": { "label": "height", "datatype": "quantity", "values": [{ "amount": 1.96, "unit": { "id": "Q11573", "label": "metre" }, "lowerBound": null, "upperBound": null }] }, "P856": { "label": "official website", "datatype": "url", "values": ["https://douglasadams.com"] }, "P18": { "label": "image", "datatype": "commonsMedia", "values": [{ "file": "Douglas adams portrait.jpg", "url": "https://commons.wikimedia.org/wiki/File:Douglas_adams_portrait.jpg" }] } }, "qualifiers": { "P26": [{ "value": { "id": "Q14623681", "label": "Jane Belson" }, "rank": "normal", "qualifiers": { "P580": { "label": "start time", "values": [{ "time": "1991-11-25", "precision": "day", "calendar": "Gregorian" }] } } }] }, "sitelinks": { "enwiki": "https://en.wikipedia.org/wiki/Douglas_Adams" }, "wikipediaUrl": "https://en.wikipedia.org/wiki/Douglas_Adams", "modified": "2026-08-03T13:13:51Z", "scrapedAt": "2026-08-06T11:40:00.000Z" } ``` ## RUN_SUMMARY Written to the run's key-value store under the key `RUN_SUMMARY`. **Read it.** It is where a partial run admits that it was partial. ```json { "entitiesRequested": 3, "entitiesSaved": 2, "entitiesFailed": 1, "searchTermsResolved": 2, "labelsResolved": 184, "failures": [ { "source": "Q999999999", "error": "Could not find an entity with the ID \"Q999999999\"." } ], "filters": { "properties": ["P31", "P569"], "languages": ["en"], "resolveLabels": true, "includeSitelinks": false }, "finishedAt": "2026-08-06T11:40:02.771Z" } ``` `entitiesRequested` counts unique identifiers after search resolution, so with `resultsPerSearchTerm` above 1 it will exceed the number of terms you sent, and a term that resolved to an entity you also listed explicitly is counted once. `searchTermsResolved` is the number of hits search produced, not the number of terms. `failures` entries carry `source`, which is either a term or an identifier, and `error`. A search term that matched nothing appears there too. `labelsResolved` is the size of the label cache and is a useful proxy for how much the label pass cost. A run that saved nothing throws instead of finishing. Note that `resultsPerSearchTerm` is not echoed in `filters`. ## Behaviour to plan around - **One bad identifier does not lose the batch.** Entities are fetched fifty at a time, and a single unknown identifier rejects the whole batch upstream. The Actor reads the offending identifier out of the error, records it in `failures` and retries the remaining forty-nine, so a typo never costs you good entities. - **A name in `entityIds` is not an error.** Anything that does not look like an identifier is moved to the search list. If you expected an exact fetch and got a best-match resolution, check `searchTermsResolved`. - **`instanceOf` depends on your `properties` filter.** It is derived from `P31`, so a filter that omits `P31` returns `instanceOf: []` on an entity that certainly has a type. Include `P31` in the filter whenever you rely on the field. - **`sitelinks` is absent, not empty, when the flag is off.** Test for `undefined`, not just for an empty object. - **A label of `null` inside a claim means resolution was skipped or capped**, not that the referenced item has no label. Label lookups stop at 2000 distinct identifiers per run, and a failed label pass is logged and swallowed so the entity is still saved. - **Requesting one language is not enough on its own**, and the Actor already compensates: a large share of entities keep their Latin-script terms in a shared multilingual bucket, which is appended to your chain automatically for labels, descriptions and aliases. - **Times are not ISO 8601.** A value like `"1952"` or `"1952-03"` is correct and reflects the stated precision; the untrimmed original is in `raw`, which can carry a leading `+`, a `00` month or a BCE sign. Never pass `time` straight to a strict date parser. - **`claims` keys are dynamic.** Any P-identifier can appear or disappear as editors work. Read defensively and never assume a property exists on every record. - **Transient failures get four attempts with exponential backoff and jitter.** A not-found and a malformed request are final and are not retried. ## Recipes **Resolve names to identifiers and a few facts.** Compact and cheap. ```json { "searchTerms": ["Douglas Adams", "Ada Lovelace", "Grace Hopper"], "properties": ["P31", "P569", "P570", "P106"], "languages": ["en"], "resolveLabels": true, "includeSitelinks": false, "resultsPerSearchTerm": 1 } ``` Take `entityId` as your canonical key and store `label` for display. **Disambiguate an ambiguous name.** Several candidates per term, then pick with `instanceOf`. ```json { "searchTerms": ["Mercury"], "properties": ["P31"], "languages": ["en"], "resolveLabels": true, "includeSitelinks": false, "resultsPerSearchTerm": 10 } ``` **Harvest external identifiers for record linkage.** No property filter, so every external identifier claim on the entity comes back. ```json { "entityIds": ["Q42", "Q64", "Q95"], "languages": ["en"], "resolveLabels": true, "includeSitelinks": false, "resultsPerSearchTerm": 1 } ``` Walk `claims`, keep entries whose `datatype` is `external-id`, and index them by their P-identifier. **Multilingual labels, raw identifiers, smallest possible payload.** ```json { "entityIds": ["Q64"], "languages": ["de", "en"], "properties": ["P31", "P1082"], "resolveLabels": false, "includeSitelinks": false, "resultsPerSearchTerm": 1 } ``` Every `label` inside `claims` will be `null` here by design; join the P-identifiers against your own dictionary.