---
name: openalex-scraper
description: Query the OpenAlex scholarly graph as structured records via the Apify Actor arman-bd/openalex-scraper. Returns works (title, DOI, year, type, open-access status and free full-text URL, journal, authorships with ORCIDs and affiliations, topics, concepts, citation counts and funding awards) or the authors, institutions, sources, concepts, topics, funders and publishers those works link to, selected by free-text search, by structured filter expressions, or both. Use for institutional output analysis, open-access reporting, systematic-review candidate sets, collaborator discovery or DOI-keyed metadata for a knowledge base. Not for abstracts, full text, reference lists, per-work citation edges or anything requiring a licensed database.
---

# OpenAlex Scraper

Apify Actor `arman-bd/openalex-scraper`. Pick an entity type, narrow it with searches
and filters, get one dataset record per entity. Every record carries the same key set
whatever the entity type, so a mixed export still loads into one table and fields that
do not apply are `null`. It runs without credentials, and with no input at all it
returns the first 200 works.

## When to use it

- Institutional or funder output analysis: everything with a given ROR or award, by
  year, type and topic.
- Open-access reporting: the share of a corpus by `openAccessStatus`, per publisher,
  funder or year.
- Systematic-review candidate sets: several search strings in one run, tagged by query,
  deduplicated on `doi` afterwards.
- Collaborator and expertise discovery: works on a topic grouped by author, ORCID and
  affiliation.
- DOI-keyed metadata for a knowledge base, joining cleanly to other bibliographic
  sources.
- Resolving names to identifiers: run it once over `institutions` or `funders` to get
  the `ror` and `shortId` values the filters expect.

## When not to use it

- Abstracts or full text. Neither is emitted; `oaUrl` and `landingPageUrl` are there so
  you can fetch the text yourself where it is openly available.
- Reference lists or citation edges. You get `citedByCount` and
  `referencedWorksCount`, both scalars, not the works behind them.
- Author disambiguation you can rely on blindly. Author identifiers are algorithmic and
  merge and split over time.
- Paywalled or licensed indexes. Only what the open graph carries is available.

## Call it

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('arman-bd/openalex-scraper').call({
  entityType: 'works',
  searchQueries: ['retrieval augmented generation'],
  filters: { publication_year: '>2023', is_oa: 'true', type: 'article' },
  mailto: 'you@example.com',
  maxResults: 500,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
const { value: summary } = await client
  .keyValueStore(run.defaultKeyValueStoreId)
  .getRecord('RUN_SUMMARY');
```

One-shot over HTTP, when you want the rows back in the same request:

```bash
curl -X POST "https://api.apify.com/v2/acts/arman-bd~openalex-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"entityType":"works","searchQueries":["large language models"],"filters":{"is_oa":"true"},"maxResults":200}'
```

The Actor is also exposed through Apify's MCP server as `arman-bd/openalex-scraper`, so
an MCP-capable agent can call it with no extra wiring.

## Input

| Field | Type | Required | Default | Notes |
|---|---|---|---|---|
| `entityType` | string | no | `"works"` | One of `works`, `authors`, `institutions`, `sources`, `concepts`, `topics`, `funders`, `publishers`. Case-insensitive. Anything else aborts the run before a request is made. One run reads one collection. |
| `searchQueries` | string[] | no | `[]` | Free-text searches, each run separately and tagged on its rows as `searchQuery`. Empty means one unfiltered sweep of whatever `filters` matches. |
| `filters` | object | no | `{}` | Filter expressions as key/value pairs, joined into a single conjunction. Values pass through untouched, so operators work: `">2020"`, `"!null"`, `"2020-2024"`. An array value is joined with a pipe and read as OR, so `{"type": ["article", "preprint"]}` means either. Keys with an empty, null or undefined value are dropped. |
| `mailto` | string | no | `""` | A contact address. Supplying one routes the run into the source's less contended request pool and identifies the client, which is what its documentation asks of automated readers. Validated as an email before the run starts. |
| `maxResults` | integer | no | `200` | Cap per search query, or for the whole run when no query is given. `0` is read as no cap at all. Paging is by cursor at 200 records a request, so large caps are a question of run time, not of a ceiling. |

**`searchQueries` and `filters` do different jobs and the cap sits between them.**
Filters are exact and are applied by the source; free-text search is ranked and fuzzy.
`maxResults` applies per query, so three queries at 1000 is up to 3000 rows, not 1000.
For a defined population (an institution's output, a year, a funder) use `filters`
alone and leave `searchQueries` empty: one target, one cap, no ranking involved. Use
`searchQueries` when you want relevance ordering, and expect to deduplicate afterwards.

## Output

One record per entity. The key set is identical across entity types; the table below
notes which type populates each field.

| Field | Type | Notes |
|---|---|---|
| `entityType` | string | The collection this record came from. Echoes the input. |
| `openalexId` | string \| null | Full URI form of the identifier. |
| `shortId` | string \| null | The bare identifier, for example `W4404534210`. This is what filters elsewhere expect. |
| `displayName` | string \| null | Entity name. For works, the title. |
| `title` | string \| null | Works only. Falls back to the display name. |
| `doi` | string \| null | Works only, as a full URI. |
| `publicationYear` | number \| null | Works only. |
| `publicationDate` | string \| null | Works only, `YYYY-MM-DD`. |
| `type` | string \| null | Works: `article`, `preprint`, `book-chapter` and so on. Institutions and sources: their own type vocabulary. |
| `language` | string \| null | Works only, two-letter code. |
| `isRetracted` | boolean \| null | Works only. |
| `openAccessStatus` | string \| null | Works only: `gold`, `green`, `bronze`, `hybrid`, `closed`. |
| `isOpenAccess` | boolean \| null | Works, and sources where the whole venue is open access. |
| `oaUrl` | string \| null | Works only. Free full-text location where one is known. |
| `landingPageUrl` | string \| null | Works only. Publisher landing page. |
| `sourceName` | string \| null | The journal, repository or conference. Works and sources. |
| `sourceType` | string \| null | Venue type. Works and sources. |
| `sourceIssn` | string \| null | Linking ISSN of the venue. Works and sources. |
| `hostOrganization` | string \| null | Publisher behind the venue. Works and sources. |
| `authorships` | object[] | Works only. Keys: `name`, `authorId`, `orcid`, `position`, `isCorresponding`, `institutions`, `countries`. Capped at the first 100 authors. Empty array on other types. |
| `authorCount` | number \| null | Works only. The true total, uncapped, so it can exceed the length of `authorships`. |
| `institutions` | string[] | Works only. Distinct affiliation names across all authorships. |
| `concepts` | object[] | Works only. Legacy vocabulary. Keys: `name`, `level`, `score`. |
| `topics` | object[] | Works: `name`, `field`, `domain`, `score`. Authors and institutions: `name`, `field`, `domain`, `works`, a count rather than a relevance score, top 10 only. |
| `citedByCount` | number \| null | Citations to this entity. Present on most types. |
| `worksCount` | number \| null | Output count. Authors, institutions, sources and the generic types. Always `null` on works. |
| `referencedWorksCount` | number \| null | Works only. Size of the reference list, not the references. |
| `fundingGrants` | object[] | Works only. Keys: `funderName`, `funderId`, `awardId`. |
| `orcid` | string \| null | Authors only at the top level. A work's ORCIDs live inside `authorships`. |
| `ror` | string \| null | Institutions only. |
| `countryCode` | string \| null | Authors (from the first known affiliation), institutions, sources and the generic types. |
| `homepageUrl` | string \| null | Institutions, sources and the generic types. |
| `hIndex` | number \| null | Authors, institutions, sources and the generic types. |
| `i10Index` | number \| null | Same types as `hIndex`. |
| `twoYearMeanCitedness` | number \| null | Same types as `hIndex`. |
| `lastKnownInstitutions` | object[] | Authors only. Keys: `name`, `ror`, `countryCode`, `type`. |
| `updatedDate` | string \| null | When the source last revised the record. The field to diff on for incremental runs. |
| `url` | string \| null | Same value as `openalexId`. |
| `searchQuery` | string \| null | Which `searchQueries` entry produced this row. `null` when no query was given. |
| `scrapedAt` | string | Run timestamp, ISO 8601. |

A real works record, `null` template fields omitted for length:

```json
{
  "entityType": "works",
  "openalexId": "https://openalex.org/W4404534210",
  "shortId": "W4404534210",
  "displayName": "A Survey on Hallucination in Large Language Models…",
  "title": "A Survey on Hallucination in Large Language Models…",
  "doi": "https://doi.org/10.1145/3703155",
  "publicationYear": 2024,
  "publicationDate": "2024-11-20",
  "type": "article",
  "language": "en",
  "isRetracted": false,
  "openAccessStatus": "bronze",
  "isOpenAccess": true,
  "oaUrl": "https://dl.acm.org/doi/pdf/10.1145/3703155",
  "landingPageUrl": "https://doi.org/10.1145/3703155",
  "sourceName": "ACM Transactions on Information Systems",
  "sourceType": "journal",
  "sourceIssn": "1046-8188",
  "hostOrganization": null,
  "authorships": [
    {
      "name": "Weijiang Yu",
      "authorId": "A5055989750",
      "orcid": "https://orcid.org/0000-0002-7449-3093",
      "position": "middle",
      "isCorresponding": false,
      "institutions": ["Huawei Technologies (China)"],
      "countries": ["CN"]
    }
  ],
  "authorCount": 11,
  "institutions": ["Harbin Institute of Technology", "Huawei Technologies (China)"],
  "concepts": [{ "name": "Visual Hallucination", "level": 2, "score": 0.564 }],
  "topics": [
    { "name": "Topic Modeling", "field": "Computer Science", "domain": "Physical Sciences", "score": 0.9926 }
  ],
  "citedByCount": 1757,
  "referencedWorksCount": 214,
  "fundingGrants": [],
  "updatedDate": "2026-08-05T07:39:15.569665",
  "url": "https://openalex.org/W4404534210",
  "searchQuery": "large language models",
  "scrapedAt": "2026-08-06T11:43:13.229Z"
}
```

## RUN_SUMMARY

Written to the run's key-value store under the key `RUN_SUMMARY`. **Read it.** A query
with a bad filter key fails on its own and the run still exits successfully.

```json
{
  "entityType": "works",
  "targetsRequested": 2,
  "targetsFailed": 1,
  "failures": [
    { "query": "base editing", "error": "rejected: publication_yr is not a valid field. Valid fields are …" }
  ],
  "entitiesSaved": 1000,
  "filters": {
    "searchQueries": ["crispr gene editing", "base editing"],
    "filters": { "publication_year": ">2022", "is_oa": "true" },
    "maxResults": 1000,
    "politePool": true
  },
  "finishedAt": "2026-08-06T11:44:02.910Z"
}
```

`targetsRequested` is the number of search queries, or `1` when none was given, in
which case `failures[].query` reads `(no search query)`. `targetsFailed` above zero
means part of your intended corpus is simply absent, and the message usually names the
offending filter key along with the valid ones. `filters.maxResults` is `null` when the
run was uncapped. `filters.politePool` records whether a `mailto` was supplied.

## Behaviour to plan around

- **Every record has every key.** Fields that do not apply to the entity type are
  `null` or an empty array, not missing. So `null` means "not applicable or not known",
  never "absent from the schema", and a works record always has `worksCount: null`.
- **`maxResults` is per query, not per run.** Three queries with `maxResults: 1000` can
  save 3000 rows. Budget accordingly.
- **There is no deduplication between queries.** A work matching two of your searches is
  saved twice with different `searchQuery` values. Deduplicate on `shortId` or `doi`
  yourself, and decide which `searchQuery` to keep.
- **`authorships` is capped at 100 entries, `authorCount` is not.** Consortium papers
  list thousands of authors. Never derive the author count from the array length.
- **`topics` means two different things.** On works each entry has a relevance `score`;
  on authors and institutions each entry has a `works` count instead and the list is
  the top 10. Branch on `entityType` before aggregating them.
- **`concepts` is the legacy vocabulary and `topics` is the current one.** Both come
  back on works. Prefer `topics` for anything new; use `concepts` only to line up with
  historical data.
- **`doi` and `orcid` are full URIs, not bare identifiers.** Strip the prefix before
  joining against a source that stores them bare.
- **A rejected filter kills only its own query.** The remaining queries continue and the
  run exits successfully. The Actor throws only when every query failed.
- **Transient errors are retried** three times with linear backoff and a generous
  request timeout, because a 200-record page can take ten seconds. A rejected request
  and an unexpected payload shape are final.
- **`maxResults: 0` means unlimited**, despite the schema minimum of 1. That is a real
  way to start a run with no end in sight; set a number unless you mean it.
- **`updatedDate` is the incremental key.** For a scheduled run, filter on it and diff
  on `shortId` plus `updatedDate` rather than re-fetching a whole corpus.

## Recipes

**One institution's output for a year.** No free-text search, so exactly one target and
one cap.

```json
{
  "entityType": "works",
  "filters": {
    "authorships.institutions.ror": "https://ror.org/042nb2s44",
    "publication_year": "2025"
  },
  "mailto": "you@example.com",
  "maxResults": 5000
}
```

Count by `openAccessStatus` for an OA report, and by `topics[0].field` for a subject
profile.

**Systematic-review candidate set.** Several strings, tagged per query.

```json
{
  "entityType": "works",
  "searchQueries": ["crispr gene editing", "base editing", "prime editing"],
  "filters": { "publication_year": ">2022", "type": "article" },
  "mailto": "you@example.com",
  "maxResults": 1000
}
```

Deduplicate on `doi`, keeping the earliest `searchQuery`, then export for screening.
Expect up to 3000 rows before deduplication.

**Resolve identifiers before filtering.** Filters want identifiers, and this is how you
find them.

```json
{
  "entityType": "institutions",
  "searchQueries": ["Karolinska Institute"],
  "maxResults": 5
}
```

Take `ror` for an `authorships.institutions.ror` filter and `shortId` for anything that
wants the bare identifier. The same trick over `funders` gives you an
`awards.funder_id` value.

**Highly cited open-access work in a field.** Filters do the selection, not ranking.

```json
{
  "entityType": "works",
  "filters": { "cited_by_count": ">100", "is_oa": "true", "type": "article", "publication_year": ">2021" },
  "mailto": "you@example.com",
  "maxResults": 2000
}
```

Sort on `citedByCount` locally and use `oaUrl` to fetch text where you need it.