--- name: clinicaltrials-scraper description: Search the ClinicalTrials.gov registry and return structured study records via the Apify Actor arman-bd/clinicaltrials-scraper. Each record carries the NCT id, brief and official titles, recruitment status, phases, study type, conditions, interventions, lead sponsor and collaborators, enrolment, start and completion dates, every trial site and every primary outcome measure. Use when a task needs an indication landscape, a sponsor pipeline profile, trial-site discovery for recruitment, or status change tracking across runs. Not for results data, adverse events, publications, patient records or non-US registries. --- # ClinicalTrials.gov Scraper Apify Actor `arman-bd/clinicaltrials-scraper`. Give it any combination of condition, intervention, sponsor, phase and status filters, get one dataset record per matching study. Every filter is optional and no credentials are involved. ## When to use it - An indication landscape: every study for a disease, with sponsor, phase and enrolment. - A competitor pipeline: everything one sponsor has in mid or late-stage development. - Trial-site discovery: recruiting studies for an indication, exploded by `locations`. - Molecule tracking: every trial touching a drug, across indications and sponsors. - Change detection on a schedule: diff `overallStatus`, `completionDate` and the set of `nctId` values between runs to catch new starts, phase moves and terminations. ## When not to use it - Results, adverse events, participant flow or outcome *measurements*. Only the protocol-side primary outcome definitions are returned, never the numbers. - Publications, citations or linked literature. Use a bibliographic Actor for those. - Non-US registries. This is the ClinicalTrials.gov registry alone; a trial registered only in EudraCT, ISRCTN, jRCT or CTRI will not appear. - Free-text search across a whole protocol. The filters are condition, intervention, sponsor, phase and status, not arbitrary keyword search. ## Call it ```js import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('arman-bd/clinicaltrials-scraper').call({ conditions: ['melanoma', 'non-small cell lung cancer'], interventions: ['pembrolizumab'], phases: ['PHASE2', 'PHASE3'], statuses: ['RECRUITING', 'ACTIVE_NOT_RECRUITING'], maxResults: 250, }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); const { value: summary } = await client .keyValueStore(run.defaultKeyValueStoreId) .getRecord('RUN_SUMMARY'); ``` One-shot over HTTP, when you want the rows back in the same request: ```bash curl -X POST "https://api.apify.com/v2/acts/arman-bd~clinicaltrials-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{"conditions":["melanoma"],"phases":["PHASE3"],"statuses":["RECRUITING"],"maxResults":100}' ``` The Actor is also exposed through Apify's MCP server as `arman-bd/clinicaltrials-scraper`, so an MCP-capable agent can call it with no extra wiring. ## Input | Field | Type | Required | Default | Notes | |---|---|---|---|---| | `conditions` | string[] | no | `[]` | Disease or condition terms. Multi-word terms are quoted for you so they match as a phrase, not as loose words. Blanks and duplicates are dropped. | | `interventions` | string[] | no | `[]` | Drug, device or procedure names. | | `sponsors` | string[] | no | `[]` | Lead sponsor or collaborator names, matched as filed, so `Merck Sharp & Dohme LLC` and `Merck` are different queries. | | `phases` | string[] | no | `[]` | Enum, uppercased for you: `EARLY_PHASE1`, `PHASE1`, `PHASE2`, `PHASE3`, `PHASE4`, `NA`. An unrecognised value throws before any request. Empty means every phase. | | `statuses` | string[] | no | `[]` | Enum, uppercased for you: `RECRUITING`, `NOT_YET_RECRUITING`, `ENROLLING_BY_INVITATION`, `ACTIVE_NOT_RECRUITING`, `COMPLETED`, `SUSPENDED`, `TERMINATED`, `WITHDRAWN`, `AVAILABLE`, `NO_LONGER_AVAILABLE`, `TEMPORARILY_NOT_AVAILABLE`, `APPROVED_FOR_MARKETING`, `WITHHELD`, `UNKNOWN`. An unrecognised value throws before any request. Empty means every status. | | `maxResults` | integer | no | `1000` | Total studies saved across all pages. `0` means no cap, which with no filters is the whole registry, 500,000 studies and up. | **Terms inside one field are OR-ed; separate fields are AND-ed.** So `conditions: ["melanoma", "lung cancer"]` with `phases: ["PHASE3"]` reads as (melanoma OR lung cancer) AND phase 3. That is the one thing that decides how you configure this Actor: adding a second condition widens the result set, adding a second *field* narrows it. Start with `conditions` alone to size the landscape, read `studiesMatched` in `RUN_SUMMARY`, then add fields until the count is workable rather than guessing at `maxResults`. ## Output One record per study. | Field | Type | Notes | |---|---|---| | `nctId` | string \| null | Registry identifier, the stable join key across runs. | | `briefTitle` | string \| null | Short public title. | | `officialTitle` | string \| null | Full protocol title. Often absent on older or observational filings. | | `overallStatus` | string \| null | Recruitment status at scrape time, same vocabulary as the `statuses` input. | | `phases` | string[] | Array, not a scalar. Combined-phase trials really do report `["PHASE1", "PHASE2"]`. Empty for observational studies. | | `studyType` | string \| null | `INTERVENTIONAL`, `OBSERVATIONAL` or `EXPANDED_ACCESS`. | | `conditions` | string[] | Indications as the sponsor filed them, so wording varies between records. | | `interventions` | object[] | One entry per intervention, keys `type` and `name`. `type` is `DRUG`, `DEVICE`, `BIOLOGICAL`, `PROCEDURE`, `BEHAVIORAL` and similar. | | `leadSponsor` | string \| null | Lead sponsor organisation name. | | `collaborators` | string[] | Collaborating organisation names, empty array when there are none. | | `enrollmentCount` | number \| null | Target or actual enrolment. The record does not say which. | | `startDate` | string \| null | As filed. Month precision is legal, so `2016-07` occurs alongside `2024-11-18`. | | `completionDate` | string \| null | As filed, same precision caveat. Frequently a future date. | | `locations` | object[] | Every trial site, keys `facility`, `city`, `state`, `country`, `status`. Empty for many industry filings. | | `primaryOutcomes` | object[] | One entry per primary endpoint, keys `measure`, `description`, `timeFrame`. Definitions only, no measured values. | | `scrapedAt` | string | Run timestamp, ISO 8601 UTC. | A real record, long strings trimmed: ```json { "nctId": "NCT06692738", "briefTitle": "A Global Phase III Study of Rilvegostomig or Pembrolizumab Plus Chemotherapy for First-Line Treatment of Locally Advanced or Metastatic Squamous Non-small Cell Lung Cancer (NSCLC)", "officialTitle": "A Phase III, Randomized, Double-blind, Multicenter, Global Study of Rilvegostomig or Pembrolizumab in Combination With Platinum-based Chemotherapy …", "overallStatus": "RECRUITING", "phases": ["PHASE3"], "studyType": "INTERVENTIONAL", "conditions": ["Non-small Cell Lung Cancer"], "interventions": [ { "type": "DRUG", "name": "Rilvegostomig" }, { "type": "DRUG", "name": "Pembrolizumab" }, { "type": "DRUG", "name": "Carboplatin" } ], "leadSponsor": "AstraZeneca", "collaborators": [], "enrollmentCount": 1160, "startDate": "2024-11-18", "completionDate": "2030-07-01", "locations": [ { "facility": "Research Site", "city": "Tucson", "state": "Arizona", "country": "United States", "status": "RECRUITING" } ], "primaryOutcomes": [ { "measure": "Overall survival (OS)", "description": "OS is defined as the time from randomization until the date of death due to any cause.", "timeFrame": "Up to approximately 6 years" } ], "scrapedAt": "2026-08-06T11:43:12.806Z" } ``` ## RUN_SUMMARY Written to the run's key-value store under the key `RUN_SUMMARY`. **Read it.** It is where a partial run admits that it was partial, and it is the only place the registry's own match count appears. ```json { "studiesMatched": 276, "studiesSaved": 250, "pagesFetched": 1, "pagesFailed": 0, "failures": [], "filters": { "conditions": ["melanoma", "non-small cell lung cancer"], "interventions": ["pembrolizumab"], "sponsors": [], "phases": ["PHASE2", "PHASE3"], "statuses": ["RECRUITING", "ACTIVE_NOT_RECRUITING"], "maxResults": 250 }, "finishedAt": "2026-08-06T11:43:12.909Z" } ``` Read it as three cases. `studiesSaved` equal to `studiesMatched` is complete. `studiesSaved` equal to `maxResults` and below `studiesMatched` means your cap truncated the run, so raise `maxResults` or narrow `filters`. `studiesSaved` below both with `pagesFailed` above zero means a page failed and paging stopped there, so the tail is missing and the run should be repeated. `studiesMatched` can be `null` if the first page never returned. ## Behaviour to plan around - **An empty dataset is a valid, successful run.** Zero matches with no failures exits cleanly with nothing saved. The Actor only throws when it saved nothing *and* had failures. Always check `studiesMatched` before concluding the Actor broke. - **A bad enum value kills the run before any request.** An unrecognised entry in `phases` or `statuses` throws immediately with the valid list in the message. Case is normalised for you, so `recruiting` is fine, `Recruting` is not. - **Setting `phases` silently excludes observational studies.** They carry no phase, so any phase filter drops them. Leave `phases` empty when you want the full picture for an indication. - **`phases` on a record is an array.** Combined-phase trials report two values. Do not index `[0]` and call it the phase. - **Dates are as-filed, not normalised.** `startDate` and `completionDate` can be `YYYY-MM` or `YYYY-MM-DD`. Parse defensively and never assume a day component. - **A failed page stops paging but keeps everything already saved.** The failure is recorded and the run finishes successfully, so a truncated dataset can look complete unless you read `pagesFailed`. - **Transient failures are retried** three times with linear backoff. A rejected query and a malformed response fail fast, since retrying either never helps. - **Pages are 1,000 studies and the last one is trimmed** to land exactly on `maxResults`, so there are no duplicates and no overshoot. - **`locations` is often empty** on large industry trials that list a central contact instead of sites. Absence is not evidence that a trial has no sites. ## Recipes **Indication landscape, unfiltered by phase.** Sizes the field including observational work. ```json { "conditions": ["idiopathic pulmonary fibrosis"], "maxResults": 1000 } ``` Read `studiesMatched` first. Group the records by `leadSponsor` and by `studyType` to see who is active and how. **Competitor pipeline profile.** Mid and late-stage, everything not yet finished. ```json { "sponsors": ["Pfizer"], "phases": ["PHASE2", "PHASE3"], "statuses": ["RECRUITING", "ACTIVE_NOT_RECRUITING", "NOT_YET_RECRUITING"], "maxResults": 2000 } ``` Group by `conditions` to see where the bets are, and by `startDate` to see the cadence. **Trial sites for patient recruitment.** Recruiting only, then explode the sites. ```json { "conditions": ["type 2 diabetes"], "statuses": ["RECRUITING"], "maxResults": 500 } ``` Flatten `locations` into one row per site and filter on `country`, `state` and the per-site `status`, which can differ from the study-level `overallStatus`. **Molecule tracking on a schedule.** One intervention, no other filter, so nothing is excluded by accident. ```json { "interventions": ["semaglutide"], "maxResults": 1000 } ``` Store each run keyed on `nctId`. New ids are new starts, a changed `overallStatus` is a transition or a termination, and a moved `completionDate` is a schedule slip.