---
name: doaj-journals-scraper
description: Search the Directory of Open Access Journals as structured records via the Apify Actor arman-bd/doaj-journals-scraper. In journal mode it returns title, print and electronic ISSN, publisher, country, LCC subject labels, licence types, peer-review process, open-access start year and article processing charges as a number plus a currency, with every published per-currency price. In article mode it returns DOI, authors, abstract, keywords, journal, volume, issue and a full-text link. Use for APC and publishing-cost comparison, funder open-access compliance checks, OA landscape analysis or building an article metadata corpus. Not for full text, citation counts, impact factors, closed-access literature or preprints.
---

# DOAJ Scraper

Apify Actor `arman-bd/doaj-journals-scraper`. Give it search queries, get one dataset
record per journal or per article. One run searches one index; `resourceType` picks
which. It runs without credentials; you supply nothing but the queries.

## When to use it

- Comparing article processing charges across journals, publishers or subjects, with
  the figure as a number and a currency rather than prose.
- Checking a funder or institutional open-access mandate against `license`, `hasApc`
  and `peerReviewProcess`.
- Mapping the OA landscape: diamond OA (`hasApc: false`), publisher share, country
  distribution, subject coverage.
- Resolving a publisher's or a subject's full journal list into rows, including ISSNs
  for matching against your own holdings.
- Building an article metadata corpus with abstracts, DOIs and full-text links for
  indexing or retrieval.

## When not to use it

- Full text of articles. `url` points at the full text; fetching it is your job.
- Citation counts, h-index, impact factor or any bibliometric ranking. DOAJ carries
  none of it.
- Closed-access or hybrid journals, preprints and repositories. Only vetted
  open-access titles and their articles are indexed.
- A complete mirror of the directory. The search window stops at 1000 records per
  query; a full copy is what DOAJ's public data dump is for.

## Call it

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('arman-bd/doaj-journals-scraper').call({
  searchQueries: ['bibjson.apc.has_apc:true AND bibjson.publisher.country:GB'],
  resourceType: 'journals',
  subjects: ['Medicine'],
  maxResults: 500,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
const { value: summary } = await client
  .keyValueStore(run.defaultKeyValueStoreId)
  .getRecord('RUN_SUMMARY');
```

One-shot over HTTP, when you want the rows back in the same request:

```bash
curl -X POST "https://api.apify.com/v2/acts/arman-bd~doaj-journals-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"searchQueries":["bibjson.subject.term:\"Physics\""],"resourceType":"journals","maxResults":100}'
```

The Actor is also exposed through Apify's MCP server as
`arman-bd/doaj-journals-scraper`, so an MCP-capable agent can call it with no extra
wiring.

## Input

| Field | Type | Required | Default | Notes |
|---|---|---|---|---|
| `searchQueries` | string[] | yes | | One search per entry, paginated independently, so three queries can return three times the per-query ceiling. Plain words work (`machine learning`); DOAJ field syntax is more precise (`bibjson.publisher.name:"Elsevier"`, `bibjson.apc.has_apc:false`, `issn:2731-3395`, `bibjson.apc.has_apc:true AND bibjson.publisher.country:GB`, or `*` for everything). Trimmed and deduplicated. |
| `resourceType` | string | no | `"journals"` | `journals` or `articles`. One run searches one index and the record shape differs between them. Anything other than `articles` is read as `journals`. |
| `subjects` | string[] | no | `[]` | Post-filter on the record's own `subjects`, case-insensitive substring against the joined LCC labels (`Physics`, `Electronic computers. Computer science`). Applied after paging, so it narrows what came back, it does not narrow what was fetched. |
| `maxResults` | integer | no | `200` | Total records saved across all queries, not per query. `0` means no cap. Queries are consumed in order and the run stops as soon as the cap is met. |

**`subjects` and the query do the same job at different prices.** Anything you can
express as `bibjson.subject.term:"Physics"` inside `searchQueries` is filtered by the
source before paging, so it costs nothing and gets you a full 1000 records of matches.
The `subjects` input filters after the records arrive, so a broad query with a narrow
`subjects` list can page through 1000 records and save a handful. Use the query for
selectivity; use `subjects` only when you want one broad query sliced several ways in a
single run.

## Output

One record per journal or per article. `resourceType` on a record is singular
(`journal`, `article`) even though the input takes the plural.

Journal records:

| Field | Type | Notes |
|---|---|---|
| `resourceType` | string | Always `journal` in this mode. |
| `query` | string | The `searchQueries` entry that produced this row. Join back on it. |
| `id` | string \| null | DOAJ's internal journal id. |
| `title` | string \| null | Journal title, whitespace collapsed. |
| `issn` | string \| null | Print ISSN. `null` for electronic-only titles. |
| `eissn` | string \| null | Electronic ISSN. At least one of the two is present. |
| `publisher` | string \| null | Publisher name. |
| `country` | string \| null | Publisher country, ISO-2. |
| `subjects` | string[] | Library of Congress subject labels. |
| `language` | string[] | Publication languages, ISO codes. Empty array when unlisted. |
| `license` | string[] | Licence types, for example `["CC BY"]`. |
| `apcAmount` | number \| null | Headline article processing charge. See below on currency. |
| `apcCurrency` | string \| null | Currency of `apcAmount`. |
| `apcPrices` | object[] | Every published price, as `{ price, currency }`. Empty array when none is published. |
| `hasApc` | boolean \| null | Whether the journal charges authors at all. `null` when DOAJ has not recorded it. |
| `peerReviewProcess` | string[] | For example `["Anonymous peer review"]`. |
| `oaStartYear` | number \| null | Year the journal became fully open access. |
| `url` | string \| null | The journal's own homepage. |
| `lastUpdated` | string \| null | When DOAJ last revised the record, ISO 8601. |
| `scrapedAt` | string | Run timestamp, ISO 8601. |

Article records carry `resourceType` (`article`), `query`, `id`, `title`, `scrapedAt`
and `lastUpdated` with the same meanings, plus:

| Field | Type | Notes |
|---|---|---|
| `doi` | string \| null | DOI, when the article declares one. |
| `issn` | string \| null | Print ISSN of the containing journal. |
| `eissn` | string \| null | Electronic ISSN of the containing journal. |
| `journalTitle` | string \| null | Containing journal title. |
| `publisher` | string \| null | Publisher of the containing journal. |
| `country` | string \| null | Publisher country, ISO-2. |
| `subjects` | string[] | LCC subject labels. |
| `language` | string[] | Languages of the containing journal. |
| `keywords` | string[] | Author keywords. Often empty. |
| `authors` | string[] | Author names as strings, in the order published. No affiliations, no ORCIDs. |
| `abstract` | string \| null | Abstract, whitespace collapsed. |
| `year` | string \| null | Publication year. |
| `month` | string \| null | Publication month, when given. |
| `volume` | string \| null | Volume. |
| `issue` | string \| null | Issue number. |
| `url` | string \| null | Full-text link where one is typed as full text, otherwise the first link on the record. |

A real journal record:

```json
{
  "resourceType": "journal",
  "query": "bibjson.apc.has_apc:true",
  "id": "e5b4b3d3f1a04a…",
  "title": "Communications Engineering",
  "issn": null,
  "eissn": "2731-3395",
  "publisher": "Nature Portfolio",
  "country": "GB",
  "subjects": ["Engineering (General). Civil engineering (General)"],
  "language": ["EN"],
  "license": ["CC BY", "CC BY-NC-ND"],
  "apcAmount": 2290,
  "apcCurrency": "USD",
  "apcPrices": [
    { "price": 1990, "currency": "EUR" },
    { "price": 2290, "currency": "USD" },
    { "price": 1650, "currency": "GBP" }
  ],
  "hasApc": true,
  "peerReviewProcess": ["Anonymous peer review"],
  "oaStartYear": 2022,
  "url": "https://www.nature.com/commseng/",
  "lastUpdated": "2026-01-15T10:33:48Z",
  "scrapedAt": "2026-08-06T12:00:00.000Z"
}
```

## RUN_SUMMARY

Written to the run's key-value store under the key `RUN_SUMMARY`. **Read it.** Without
it you cannot tell a query that returned everything from one that was cut off at the
ceiling.

```json
{
  "resourceType": "journals",
  "queriesRequested": 2,
  "queriesFailed": 0,
  "failures": [],
  "recordsSaved": 300,
  "queriesTruncatedByApiCeiling": 1,
  "apiResultCeiling": 1000,
  "filters": {
    "searchQueries": ["deep learning", "bibjson.subject.term:\"Physics\""],
    "resourceType": "journals",
    "subjects": ["Physics"],
    "maxResults": 300
  },
  "finishedAt": "2026-08-06T12:00:04.512Z"
}
```

`queriesTruncatedByApiCeiling` above zero means at least one query had more matches
than could be paged: the rows you have are a prefix, not the set. `failures` names any
query rejected outright, usually a syntax error. `recordsSaved` equal to
`filters.maxResults` means the cap bound first and later queries may not have run at
all. All three can be true at once, so check each before reporting a count as complete.

## Behaviour to plan around

- **1000 records per query is a hard ceiling on the search window.** The Actor stops one
  page short of it rather than letting the run fail, counts the query in
  `queriesTruncatedByApiCeiling`, and logs the query's true total. Split a broad query
  into several narrower ones to go deeper: three subject-scoped queries fetch 3000
  records where one broad query fetches 1000.
- **`maxResults` is a total across queries, and queries run in order.** With
  `maxResults: 200` and three queries, the third may never execute. Put the query you
  care most about first, or give each its own run.
- **`resourceType` on a record is singular.** The input takes `journals` and `articles`;
  records carry `journal` and `article`. `RUN_SUMMARY.resourceType` uses the plural.
  Do not match records against the input value directly.
- **Never compare `apcAmount` across journals without `apcCurrency`.** DOAJ publishes a
  list of per-currency maxima, not one price. `apcAmount` is the first of USD, EUR, GBP
  that exists, falling back to whatever came first, so two rows can be in different
  currencies. `apcPrices` has the whole list; convert from that.
- **`hasApc: true` with `apcAmount: null` is a real combination.** A minority of records
  flag a charge without publishing the figure, and `apcPrices` is then empty. Treat it
  as unknown, not as free.
- **`issn` is null far more often than `eissn`.** Electronic-only titles have no print
  ISSN. Match on either, never on `issn` alone.
- **Partial query results are kept.** A query that fails on page four leaves the first
  three pages in the dataset and still reports the failure. Row counts per `query` are
  therefore not evidence the query completed.
- **One bad query never aborts the run.** A rejected query is recorded in `failures` and
  the rest continue. The Actor only throws when every query failed.
- **Transient errors are retried** three times with linear backoff. A rejected query, a
  404 and an unexpected payload shape are final and are not retried.
- **`subjects` matches as a substring, not a term.** `Computer science` matches the
  label `Electronic computers. Computer science`, and a short term like `Art` will also
  match `Cardiology. Arteries`. Prefer the longest distinctive fragment.

## Recipes

**Compare publishing costs in one field.** Charging journals only, then sort.

```json
{
  "searchQueries": ["bibjson.apc.has_apc:true AND bibjson.subject.term:\"Medicine\""],
  "resourceType": "journals",
  "maxResults": 1000
}
```

Group by `apcCurrency` before sorting on `apcAmount`, and drop rows where `apcAmount`
is `null`.

**Funder compliance check on a publisher's list.** Licence and author-side charge in
one pass.

```json
{
  "searchQueries": ["bibjson.publisher.name:\"Elsevier\""],
  "resourceType": "journals",
  "subjects": ["Medicine"],
  "maxResults": 500
}
```

A title is compliant with a typical CC BY mandate when `license` contains `CC BY`;
check `queriesTruncatedByApiCeiling` before declaring the list complete.

**Find diamond open access.** Free to read and free to publish.

```json
{
  "searchQueries": ["bibjson.apc.has_apc:false"],
  "resourceType": "journals",
  "maxResults": 1000
}
```

Aggregate by `country` and `publisher`. Rows with `hasApc: null` are unrecorded, not
diamond, so exclude them.

**Article metadata for a retrieval corpus.** Abstracts, DOIs and full-text links.

```json
{
  "searchQueries": ["climate adaptation", "bibjson.subject.term:\"Physics\""],
  "resourceType": "articles",
  "maxResults": 1000
}
```

Deduplicate on `doi` first and on `id` where `doi` is `null`, since two queries can
return the same article.