--- name: whois-rdap-scraper description: Look up domain registration data through RDAP, the structured successor to WHOIS, via the Apify Actor arman-bd/whois-rdap-scraper. Returns the sponsoring registrar and its IANA id, registration, expiry and last-changed dates, EPP status codes, delegated nameservers, DNSSEC state, registrant organisation and country where a registry publishes them, the registrar abuse contact, and which registry server answered. Use when a task needs portfolio expiry monitoring, domain-age or ownership due diligence, brand-protection sweeps or threat-intelligence enrichment. Not for DNS record lookups, IP or ASN registration data, or registrant identities that registries redact. --- # RDAP Domain Scraper: Modern WHOIS Replacement Apify Actor `arman-bd/whois-rdap-scraper`. Give it domains, get one dataset record per domain, read from the authoritative registry for each one rather than from a WHOIS mirror. Routing comes from IANA's published bootstrap registry, loaded once per run. You supply no credentials. ## When to use it - Expiry monitoring over a domain portfolio, on a schedule, alerting before renewal. - Due diligence: registration age, sponsoring registrar and lock status before a deal. - Brand protection: checking lookalike registrations and finding an abuse contact. - Threat intelligence: enriching indicators with domain age, nameservers and DNSSEC. - Hijack detection, by diffing `status`, `nameservers` and `registrar` between runs. ## When not to use it - DNS resolution. A, AAAA, MX, TXT and the rest are not registration data and are not returned here. - IP address, ASN or entity RDAP lookups. Domains only. - Registrant names, addresses, phone numbers or emails on most generic TLDs. Registries redact them, and nothing is synthesised to fill the gap. - Subdomains. RDAP answers for registrable names, not for hosts below them. ## Call it ```js import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('arman-bd/whois-rdap-scraper').call({ domains: ['apify.com', 'wikipedia.org', 'bbc.co.uk'], includeNameservers: true, includeRawResponse: false, }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); const { value: summary } = await client .keyValueStore(run.defaultKeyValueStoreId) .getRecord('RUN_SUMMARY'); ``` One-shot over HTTP, when you want the rows back in the same request: ```bash curl -X POST "https://api.apify.com/v2/acts/arman-bd~whois-rdap-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{"domains":["apify.com","bbc.co.uk"],"includeNameservers":true}' ``` The Actor is also exposed through Apify's MCP server as `arman-bd/whois-rdap-scraper`, so an MCP-capable agent can call it with no extra wiring. ## Input | Field | Type | Required | Default | Notes | |---|---|---|---|---| | `domains` | string[] | yes | | Bare domains (`apify.com`), hostnames with `www`, or full URLs. Everything is lower-cased and reduced to the host; a leading `www.` , a trailing dot and any path are stripped. Duplicates are removed before any request. Entries with no dot or with whitespace are dropped. | | `includeRawResponse` | boolean | no | `false` | Attach the registry's complete unparsed RDAP object as `rawResponse`. The escape hatch for anything this Actor does not map, at a large cost in record size. | | `includeNameservers` | boolean | no | `true` | Include delegated nameserver hostnames, lower-cased with the root dot stripped so they compare equal across registries. Off makes the array empty. | **Neither flag changes the request count**, which is one per domain plus one routing lookup for the whole run. They only change record size, and `includeRawResponse` changes it dramatically: a raw RDAP object carries the full entity tree, notices and links, and is many times the size of the parsed record. Turn it on for a handful of domains when you need a field that is not mapped, such as DS records or registrar contact details. Leave it off for a portfolio sweep, where the parsed columns are the whole point. ## Output One record per domain the registry answered for. | Field | Type | Notes | |---|---|---| | `domain` | string | The normalised registrable domain. Join on this. | | `registrar` | string \| null | Sponsoring registrar name, falling back to the registry's handle. `null` for registries that publish no registrar entity. | | `registrarIanaId` | string \| null | IANA-assigned registrar number only. Registry-internal identifiers are deliberately not substituted, so this is `null` more often than a registrar id appears in the raw object. | | `createdDate` | string \| null | Registration event, as published, normally ISO 8601. | | `expiryDate` | string \| null | Expiration event. The field for renewal alerting. | | `updatedDate` | string \| null | Last-changed event. Moves on registrar, nameserver or status changes. | | `status` | string[] | EPP status codes as RDAP publishes them: lower case with spaces, e.g. `client transfer prohibited`. Empty array when none. | | `nameservers` | string[] | Delegated nameservers, lower-cased, root dot stripped. **`[]` when `includeNameservers` is off**, never null. | | `dnssec` | boolean \| null | Whether the delegation is signed. `null` means the registry did not state it, which is not the same as `false`. | | `registrantCountry` | string \| null | ISO country code where published. Usually `null` on generic TLDs. | | `registrantOrg` | string \| null | Registrant organisation, falling back to the registrant's formatted name. Usually `null` on generic TLDs. | | `abuseEmail` | string \| null | Registrar's abuse address, read from inside the registrar entity. | | `rdapServer` | string | Origin of the server that actually answered, after redirects. Useful for auditing where a record came from. | | `rawResponse` | object | The complete RDAP object. **The key is absent entirely** when `includeRawResponse` is off, not present as `null`. | | `scrapedAt` | string | Run timestamp, ISO 8601. | A real record: ```json { "domain": "apify.com", "registrar": "Amazon Registrar, Inc.", "registrarIanaId": "468", "createdDate": "2009-06-02T17:14:10Z", "expiryDate": "2035-06-02T17:14:10Z", "updatedDate": "2026-05-16T16:53:04Z", "status": ["client transfer prohibited"], "nameservers": [ "ns-1225.awsdns-25.org", "ns-1928.awsdns-49.co.uk", "ns-449.awsdns-56.com", "ns-839.awsdns-40.net" ], "dnssec": true, "registrantCountry": null, "registrantOrg": null, "abuseEmail": "trustandsafety@support.aws.com", "rdapServer": "https://rdap.verisign.com", "scrapedAt": "2026-08-06T12:00:00.000Z" } ``` ## RUN_SUMMARY Written to the run's key-value store under the key `RUN_SUMMARY`. **Read it.** The distinction between "not registered" and "this TLD cannot be looked up" lives only in the failure messages. ```json { "domainsRequested": 4, "domainsFailed": 2, "failures": [ { "domain": "definitely-not-registered-xyz.com", "error": "not registered, or the registry has no record (RDAP 404)" }, { "domain": "example.tld", "error": "no RDAP service published for .tld" } ], "domainsSaved": 2, "filters": { "includeRawResponse": false, "includeNameservers": true }, "finishedAt": "2026-08-06T12:00:11.412Z" } ``` `domainsRequested` is the count after normalisation and de-duplication, so it can be below the length of your input list. Read `failures[].error` rather than just counting them: "not registered" is a real answer about the domain and often the answer you wanted, while "no RDAP service published" is a statement about the TLD and means you need a different source for that one. The run throws only when every domain failed. ## Behaviour to plan around - **`status` codes are spaced lower case, not the WHOIS spelling.** RDAP publishes `client transfer prohibited`, not `clientTransferProhibited`. Matching on the camel case form silently finds nothing. Normalise before comparing. - **Registrant fields are usually `null` on generic TLDs, by policy.** Registries redact them. Country and organisation are frequently populated for country-code TLDs and frequently absent for `.com`. A null here is the registry's answer, not a parse failure, and no product should promise registrant identity from this Actor. - **`dnssec: null` is not `false`.** It means the registry published no DNSSEC block. Treat the three states separately in any security scoring. - **`nameservers: []` is ambiguous on its own.** It means either the flag was off or the domain genuinely has no delegation. `filters.includeNameservers` in the summary resolves it. - **`rawResponse` is missing rather than null when the flag is off.** Code that reads the key without a guard will see `undefined`, and CSV exports lose the column entirely. Do not test it for null. - **Only registrable names resolve.** A host such as `blog.example.com` is not reduced to its parent (only a leading `www.` is), so it will come back as not registered. Reduce to the registrable domain yourself before passing it in. - **Multi-label country TLDs work.** `bbc.co.uk` routes on the final label, which is what the registry expects, so `.co.uk`, `.com.au` and similar behave normally. - **A null `registrar` does not mean an unregistered domain.** Some registries publish the domain object with no registrar entity at all, while still returning status, nameservers and dates. - **One bad domain never aborts the run**, and transient failures are retried three times with exponential backoff. A response that is not RDAP at all is treated as final for that domain rather than retried. ## Recipes **Portfolio expiry sweep.** Lean records, scheduled weekly. ```json { "domains": ["example1.com", "example2.com", "example3.net"], "includeNameservers": false } ``` Alert on `expiryDate` inside your renewal window, and treat a row moving from a `transfer prohibited` status to none as a lock that came off. **Hijack and drift detection.** Nameservers on, everything else default. ```json { "domains": ["apify.com", "wikipedia.org", "bbc.co.uk"], "includeNameservers": true } ``` Diff `nameservers`, `status`, `registrar` and `updatedDate` against the previous run, keyed on `domain`. A changed nameserver set with a moved `updatedDate` is the signal. **Due diligence on a single name.** Everything the registry published. ```json { "domains": ["lemonde.fr"], "includeRawResponse": true } ``` Use `createdDate` for age, `registrantOrg` and `registrantCountry` where the country TLD publishes them, and read `rawResponse` for entity detail the parsed columns omit. **Abuse contact lookup for a batch.** Defaults are already right for this. ```json { "domains": ["suspicious-lookalike-1.com", "suspicious-lookalike-2.com"] } ``` Group by `abuseEmail` to file one report per registrar rather than one per domain, and keep `rdapServer` in the record so the source of each claim is auditable.