---
name: duckduckgo-search-scraper
description: Search DuckDuckGo across web, news, images and videos via the Apify Actor arman-bd/duckduckgo-search-scraper. Returns one flat record per result with the same 25 keys on all four verticals - title, URL, domain, snippet, rank - plus source, publish date and relative time on news, pixel dimensions, format and full-size image URL on images, duration, uploader, view count and embed URL on videos. Takes a list of queries in one run, supports site:, quoted phrases, -exclusions, intitle: and OR, and 41 verified news locales. Advertising rows are excluded by default and flagged and left unranked (position: null) rather than displacing the organic ranking when kept. Use when a task needs web search results, a non-Google ranking, entity-to-URL enrichment, or an image or video set for a keyword list. Not for Google's ranking specifically, full page text, per-row dates on web results, or safe-search filtering.
---

# DuckDuckGo Scraper

Apify Actor `arman-bd/duckduckgo-search-scraper`. Give it queries and the verticals you
want; get one dataset record per result. No credentials are involved.

## When to use it

- A keyword-to-URLs search tool for an agent, with nothing to provision.
- Enriching a list of entities to a canonical homepage and a one-line description.
- Building an image or video set for a keyword list.
- Rank tracking on an index that is not Google's.

## When not to use it

- **Google's ranking.** This is DuckDuckGo's index, largely Bing-derived, and the rows say so
  in `syndicate`. The ordering, snippets and domain mix are not Google's. This is the
  single most common wrong assumption about this Actor.
- Full page text. Only the result rows are returned; nothing is fetched from the pages.
- A publish date for a web result. `publishedAt` is **always `null`** on the web vertical,
  because that surface returns none. News and video rows carry real ones.
- Locale-targeted web results. `region` applies to news, images and videos only, and web
  rows carry `region: null` rather than pretending.
- Safe-search filtering. There is no such input, because it could not be shown to do
  anything upstream.

## Call it

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('arman-bd/duckduckgo-search-scraper').call({
  queries: ['lithium battery recycling', 'site:reuters.com openai'],
  verticals: ['web', 'news'],
  maxResultsPerVertical: 25,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
const summary = await client.keyValueStore(run.defaultKeyValueStoreId).getRecord('RUN_SUMMARY');
```

One call, no SDK, records straight back:

```bash
curl -X POST "https://api.apify.com/v2/acts/arman-bd~duckduckgo-search-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H 'content-type: application/json' \
  -d '{"queries":["lithium battery recycling"],"verticals":["web"],"maxResultsPerVertical":10}'
```

## Input

| Key | Type | Required | Default | Notes |
|---|---|---|---|---|
| `queries` | array | yes | | Each searched independently on every selected vertical. Operators work (below). |
| `verticals` | array | | `["web"]` | Any of `web`, `news`, `images`, `videos`. Empty is a named error, not a silent web-only run. |
| `maxResultsPerVertical` | integer | | `50` | 1–1000, per query **per vertical**. The main cost control. |
| `region` | string | | `us-en` | 41 locales. Applies to news, images and videos; ignored by web. |
| `timeRange` | string | | `any` | `any`, `day`, `week`, `month`, `year`. Web only. |
| `includeSponsored` | boolean | | `false` | Keep advertising rows, flagged `isSponsored: true`. |
| `proxyConfiguration` | object | | off | Needed for the web vertical past ~6 pages *or* ~6 queries per run. Never needed for news, images or videos. |

**Operators inside a query**: `"quoted phrase"`, `site:reuters.com`, `intitle:changelog`,
`-excluded`, `a OR b`. They compose, and each was verified against the live source.

**Verticals** all return the same record shape. Rows per request differ a lot: web ~10 on
the first page and ~15 after, news ~15–28, videos ~58, images ~95. Plan `maxResultsPerVertical`
around that: 50 on images is half a request, 50 on web is four.

**Regions** are `country-language`, e.g. `us-en`, `uk-en`, `de-de`, `fr-fr`, `jp-jp`,
`br-pt`, `in-en`, `mx-es`, `tr-tr`. An unlisted value fails immediately rather than
quietly serving US English.

## Output

One record per result. Every key is present on every record, whichever vertical it came
from. `null` means the surface does not carry that value.

| Field | Type | Populated on |
|---|---|---|
| `query` | string | all |
| `vertical` | string | all. `web`, `news`, `images`, `videos` |
| `position` | number\|null | all except sponsored rows. Rank within its (query, vertical); **`null` on an advertising row**, and the organic rows are 1..n either way |
| `title` | string | all |
| `url` | string | all. Image rows point at the page hosting the picture |
| `domain` | string | all. Lowercased host of `url`, no `www.`. Group by this. On a sponsored row it names the advertiser, not the ad network the link points at |
| `snippet` | string\|null | web, news, videos. **Always `null` on images** |
| `displayUrl` | string\|null | web. A truncated display string, not a host |
| `isSponsored` | boolean | all. `true` only with `includeSponsored` on, and those rows carry `position: null` |
| `source` | string\|null | news (outlet), videos (platform) |
| `publishedAt` | string\|null | news, videos. **Always `null` on web and images** |
| `relativeTime` | string\|null | news. The source's own wording |
| `thumbnailUrl` | string\|null | images and videos always; news on about a third of rows |
| `imageUrl` | string\|null | images. The full-size picture |
| `width` / `height` | number\|null | images |
| `encodingFormat` | string\|null | images |
| `discoveredAt` | string\|null | images. Indexing date, **not** a publish date |
| `duration` | string\|null | videos, e.g. `11:57` |
| `uploader` | string\|null | videos |
| `viewCount` | number\|null | videos |
| `embedUrl` | string\|null | videos |
| `syndicate` | string\|null | news, images, videos. The index the row is credited to |
| `region` | string\|null | news, images, videos. **Always `null` on web** |
| `scrapedAt` | string | all |

```json
{
  "query": "renewable energy storage",
  "vertical": "news",
  "position": 1,
  "title": "From infrastructure to energy storage: The expanding role of zinc in India's development journey",
  "url": "https://www.thehindubusinessline.com/markets/commodities/from-infrastructure-to-energy-storage-the-expanding-role-of-zinc-in-indias-development-journey/article71319198.ece",
  "domain": "thehindubusinessline.com",
  "snippet": "Discover how zinc is pivotal in enhancing India's infrastructure, energy storage, and agricultural resilience for sustainable development.",
  "displayUrl": null,
  "isSponsored": false,
  "source": "The Hindu BusinessLine",
  "publishedAt": "2026-08-09T09:45:00.000Z",
  "relativeTime": "3 hours ago",
  "thumbnailUrl": "https://bl-i.thgim.com/public/todays-paper/tp-agri-biz-and-commodity/bpf6z6/article70929874.ece/alternates/LANDSCAPE_1200/Supply-side-risGQ9FU0F85.5.jpg.jpg",
  "imageUrl": null,
  "width": null,
  "height": null,
  "encodingFormat": null,
  "discoveredAt": null,
  "duration": null,
  "uploader": null,
  "viewCount": null,
  "embedUrl": null,
  "syndicate": "bing",
  "region": "us-en",
  "scrapedAt": "2026-08-09T12:41:07.000Z"
}
```

## RUN_SUMMARY

Written to the run's key-value store under `RUN_SUMMARY`.

```json
{
  "queriesRequested": ["renewable energy storage"],
  "verticalsRequested": ["web", "news"],
  "rejectedInputs": [],
  "filters": {
    "maxResultsPerVertical": 15, "region": "us-en", "regionAppliedTo": ["news"],
    "timeRange": "any", "timeRangeAppliedTo": [], "includeSponsored": false, "proxy": false
  },
  "requestsMade": { "web": 2, "lookup": 1, "news": 1, "images": 0, "videos": 0, "total": 4 },
  "resultsSaved": 30,
  "resultsByVertical": { "web": 15, "news": 15, "images": 0, "videos": 0 },
  "duplicatesCollapsed": 0,
  "rowsUnusable": 0,
  "sponsoredRowsDropped": 4,
  "sponsoredRowsLabelled": 0,
  "throttleEvents": [],
  "webRequestBudget": { "proxyRotation": false, "limitPerRun": 6, "used": 2, "stoppedEarly": false },
  "emptyResults": [],
  "perQuery": {
    "renewable energy storage": {
      "web": { "returned": 25, "saved": 15, "duplicatesCollapsed": 0, "rowsUnusable": 0, "requests": 2, "stoppedBecause": "cap" },
      "news": { "returned": 27, "saved": 15, "duplicatesCollapsed": 0, "rowsUnusable": 0, "requests": 1, "stoppedBecause": "cap" }
    }
  },
  "malformedFields": {},
  "failures": [],
  "failureReasons": {},
  "finishedAt": "2026-08-09T12:41:12.000Z"
}
```

`stoppedBecause` is the field to read when a query returned fewer rows than asked for:
`cap` means it hit your limit, `exhausted` means the source ran out, `throttled` means the
source stopped answering and there is more to get with proxy rotation, and
`request-budget` means the run reached its own web-request limit, and `error` means a
later page failed after earlier ones had already delivered, in which case the reason sits
on that same entry and the rows already read are kept. `throttleEvents` records
the offset each throttle happened at. `sponsoredRowsDropped` counts advertising rows
excluded by the default, and never charged. `emptyResults` names the (query, vertical) pairs
that produced nothing, so an empty result reads as a filter rather than as a failure.
`malformedFields` is normally empty; entries there mean an upstream shape shifted and
those fields should be treated as unreliable.

## Behaviour to plan around

- **`publishedAt` and `region` are `null` on every web row, and that is not a bug.** The
  web surface publishes neither. Do not filter web rows by date; use the `news` vertical
  when dates matter.
- **`snippet` is `null` on every image row.** There is no text on an image result.
- **Dedupe images on `imageUrl`, not `url`.** One page hosts many pictures, and the Actor
  already collapses on the picture, but so should anything you build downstream.
- **Rows per request vary sixfold between verticals**, so a run's cost is driven by which
  verticals you pick far more than by the query. Image rows are priced separately for
  exactly this reason.
- **Web paging is capped without proxy rotation.** Roughly six pages **per run, shared by
  every query**, then the Actor stops that vertical, keeps what it has, and continues. So an
  unrotated web run over more than about six queries returns rows for the first few and
  nothing for the rest; they appear in `emptyResults` with `stoppedBecause:
  "request-budget"`. Set `proxyConfiguration` for any web run over a long query list or past
  ~6 pages. News, images and videos are unaffected at any depth or query count.
- **Send every keyword in one run.** `queries` is a list, so 10,000 keywords is one run,
  not 10,000 runs. On the `web` vertical turn `proxyConfiguration` on when you do, or the
  run's six-page web allowance is spent on the first few keywords.
- **Charging is per result pushed**, with image rows on their own event. Dropped
  advertising rows, collapsed duplicates and failed queries are never charged.
- **One bad input does not end the run.** It lands in `rejectedInputs` or `failures`.

## Recipes

**Agent web search, cheapest useful shape**

```json
{ "queries": ["who acquired Deepgram"], "verticals": ["web"], "maxResultsPerVertical": 10 }
```

**Entity enrichment over a keyword list**

```json
{ "queries": ["Acme Robotics official site", "Globex Industries official site"],
  "verticals": ["web"], "maxResultsPerVertical": 5 }
```

Take `position: 1` per query and read `domain` and `snippet`.

**An image set for a keyword**

```json
{ "queries": ["gravity battery"], "verticals": ["images"], "maxResultsPerVertical": 400 }
```

Filter on `width`/`height`/`encodingFormat` before downloading anything.

**All four verticals for one query**

```json
{ "queries": ["lithium battery recycling"],
  "verticals": ["web", "news", "images", "videos"], "maxResultsPerVertical": 50 }
```

One table, filterable by `vertical`. Nothing else in this market returns all four as rows.

**Deep rank tracking**

```json
{ "queries": ["\"heat pump\" installer"], "verticals": ["web"],
  "maxResultsPerVertical": 300, "proxyConfiguration": { "useApifyProxy": true } }
```

Without the proxy this stops after roughly six pages and says so in
`webRequestBudget.stoppedEarly`.

**News in another country**

```json
{ "queries": ["Wirtschaft"], "verticals": ["news"], "region": "de-de",
  "maxResultsPerVertical": 30 }
```