--- name: github-advisories-scraper description: Export the GitHub Advisory Database as structured records via the Apify Actor arman-bd/github-advisories-scraper. Returns one row per advisory with the GHSA id, the CVE alias when one exists, severity, CVSS score, vector and revision, CWE classifications, the affected package with its vulnerable version range and first patched version, every other affected package, references and the full Markdown write-up. Use when a task needs vulnerability feeds for chosen ecosystems, alerting on a specific package, CVE cross-referencing or a patched-version lookup. Not for scanning a codebase, resolving a dependency tree, exploit code or advisories outside the GitHub database. --- # GitHub Advisories Scraper Apify Actor `arman-bd/github-advisories-scraper`. Sweeps the GitHub Advisory Database and writes one dataset record per advisory, with package ranges rather than free-text product names. It runs without credentials; a GitHub personal access token is optional and only raises the request budget. ## When to use it - Building a vulnerability feed for the ecosystems you actually ship, on a schedule. - Alerting when a new advisory appears for one named package. - Cross-referencing against another vulnerability database by joining on `cveId`. - Answering "which version fixes this" in bulk, from `firstPatchedVersion`. - Reporting on severity distribution or CWE classes across an ecosystem over a period. ## When not to use it - Scanning your own code or lockfiles. Nothing here reads a repository or resolves a dependency tree; you supply the package names or the ecosystem. - Advisories that exist only in other databases. This is the GitHub set, mapped to registry packages. - Exploit code, proof of concept material or patch diffs. Only advisory metadata and the write-up are returned. - Withdrawn advisories. They are excluded. ## Call it ```js import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('arman-bd/github-advisories-scraper').call({ ecosystems: ['npm', 'pip'], severities: ['critical', 'high'], publishedFrom: '2026-01-01', maxResults: 500, }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); const { value: summary } = await client .keyValueStore(run.defaultKeyValueStoreId) .getRecord('RUN_SUMMARY'); ``` One-shot over HTTP, when you want the rows back in the same request: ```bash curl -X POST "https://api.apify.com/v2/acts/arman-bd~github-advisories-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{"ecosystems":["npm"],"severities":["critical","high"],"publishedFrom":"2026-01-01","maxResults":200}' ``` The Actor is also exposed through Apify's MCP server as `arman-bd/github-advisories-scraper`, so an MCP-capable agent can call it with no extra wiring. ## Input | Field | Type | Required | Default | Notes | |---|---|---|---|---| | `ecosystems` | string[] | no | `[]` | Any of `npm`, `pip`, `rubygems`, `maven`, `nuget`, `composer`, `go`, `rust`, `erlang`, `actions`, `pub`, `swift`, `other`. Case-insensitive, deduplicated. An unrecognised value throws before any request. Empty means all ecosystems in one sweep. | | `severities` | string[] | no | `[]` | Any of `critical`, `high`, `medium`, `low`, `unknown`. `moderate`, `med`, `info` and `none` are accepted and normalised. An unrecognised value throws. Empty means every severity. | | `affectsPackage` | string | no | `""` | Exact registry package name as published: `lodash`, `django`, `org.apache.commons:commons-text`. Lower-cased. Not the repository name. | | `publishedFrom` | string | no | `""` | `YYYY-MM-DD`. Advisories published on or after this date. Any other format throws before any request. | | `githubToken` | string | no | `""` | Optional personal access token. Needs no scopes. Raises the budget from 60 to 5000 requests per hour. Secret field. | | `maxResults` | integer | no | `500` | Cap across all sweeps, minimum `0`, maximum `50000`. `0` means no cap. Advisories page 100 at a time, so 500 costs five requests. | **`ecosystems` and `severities` multiply, everything else is free.** Those two filters are applied one value at a time, so three ecosystems and two severities is six separate paginated sweeps, each spending its own share of the request budget. Without a token that combination will consume the hour on its own. For breadth, run one broad sweep bounded by `publishedFrom` and filter the rows yourself; for depth on one ecosystem, narrowing by severity is cheap. `affectsPackage` and `publishedFrom` cost nothing extra: they are applied inside the same request. ## Output One record per advisory, deduplicated on `ghsaId` across overlapping sweeps. | Field | Type | Notes | |---|---|---| | `ghsaId` | string | GitHub's advisory identifier. Stable primary key. | | `cveId` | string \| null | The CVE this maps to. `null` when none has been assigned yet, which is common on fresh advisories. | | `summary` | string \| null | One-line title. | | `description` | string \| null | Full Markdown write-up: impact, patches, workarounds. Can be several kilobytes. | | `severity` | string \| null | `critical`, `high`, `medium`, `low` or `unknown`. Note `medium`, never `moderate`. | | `cvssScore` | number \| null | Base score. Can be `null`, and can be `0` on rows scored without a vector. | | `cvssVector` | string \| null | Full vector string. `null` when the advisory carries no vector. | | `cvssVersion` | string \| null | Which CVSS revision the vector came from, read from the vector's own prefix: `4.0`, `3.1`, `3.0`. `null` when there is no vector. | | `cweIds` | string[] | Every CWE the advisory is classified under, for example `["CWE-1321"]`. Empty array when unclassified. | | `ecosystem` | string \| null | Registry of the **primary** affected package. See the note below. | | `packageName` | string \| null | Name of the primary affected package. | | `vulnerableVersionRange` | string \| null | Affected range for the primary package, as a range expression. | | `firstPatchedVersion` | string \| null | Version that fixes the primary package. `null` when no fix has shipped. | | `affected` | object[] | **Every** affected package: `{ ecosystem, packageName, vulnerableVersionRange, firstPatchedVersion }`. | | `publishedAt` | string \| null | ISO 8601. | | `updatedAt` | string \| null | ISO 8601. An advisory can be re-scored or re-scoped after publication. | | `references` | string[] | External links: patches, issues, vendor bulletins. | | `url` | string \| null | The human-readable advisory page. | | `scrapedAt` | string | Run timestamp, ISO 8601. | The scalar package fields describe one entry from `affected`. When `affectsPackage` is set, that entry is the package you asked about; otherwise it is the first one listed. A real record, long strings trimmed: ```json { "ghsaId": "GHSA-r5fr-rjxr-66jc", "cveId": "CVE-2026-4800", "summary": "Prototype pollution in lodash template compilation", "description": "### Impact\n\nThe template compiler …", "severity": "high", "cvssScore": 8.1, "cvssVector": "CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:H/A:H", "cvssVersion": "3.1", "cweIds": ["CWE-1321"], "ecosystem": "npm", "packageName": "lodash", "vulnerableVersionRange": ">= 4.0.0, <= 4.17.23", "firstPatchedVersion": "4.18.0", "affected": [ { "ecosystem": "npm", "packageName": "lodash", "vulnerableVersionRange": ">= 4.0.0, <= 4.17.23", "firstPatchedVersion": "4.18.0" }, { "ecosystem": "npm", "packageName": "lodash-es", "vulnerableVersionRange": ">= 4.0.0, <= 4.17.23", "firstPatchedVersion": "4.18.0" }, { "ecosystem": "npm", "packageName": "lodash.template", "vulnerableVersionRange": ">= 4.0.0, < 4.18.0", "firstPatchedVersion": "4.18.0" } ], "publishedAt": "2026-04-01T23:51:12Z", "updatedAt": "2026-04-02T14:02:55Z", "references": [ "https://github.com/lodash/lodash/security/advisories/GHSA-r5fr-rjxr-66jc", "https://nvd.nist.gov/vuln/detail/CVE-2026-4800" ], "url": "https://github.com/advisories/GHSA-r5fr-rjxr-66jc", "scrapedAt": "2026-08-06T12:00:00.000Z" } ``` ## RUN_SUMMARY Written to the run's key-value store under the key `RUN_SUMMARY`. **Read it.** It is where a partial run admits that it was partial. ```json { "sweepsRequested": 2, "sweepsFailed": 0, "failures": [], "advisoriesSaved": 500, "requestsUsed": 5, "rateLimited": false, "rateLimit": { "limit": 60, "remaining": 43, "resetsAt": "2026-08-06T13:12:11.000Z", "authenticated": false }, "filters": { "ecosystems": ["npm"], "severities": ["critical", "high"], "affectsPackage": null, "publishedFrom": "2026-01-01", "maxResults": 500 }, "finishedAt": "2026-08-06T12:00:04.918Z" } ``` Three states look alike in the dataset and are told apart only here. Fewer rows than `maxResults` with `rateLimited: false` and no `failures` means the filters genuinely matched that few. `rateLimited: true` means the budget ran out: the remainder is unfetched and can be resumed after `rateLimit.resetsAt`, ideally with a token. `sweepsFailed` above zero with `rateLimited: false` means a specific filter combination errored and its slice is missing. Exactly `maxResults` rows should always be read as "there may be more", never as "that was all of it". ## Behaviour to plan around - **The middle severity band is `medium` in the data,** although GitHub's own web interface labels it Moderate. The input accepts `moderate` and normalises it; the output never uses that word. - **`cveId` is often `null` on recent advisories.** GitHub publishes on triage and the CVE is assigned separately, sometimes days later. That gap is a feature of this source, not a data error, so never drop rows on a missing CVE. - **The scalar package fields are one entry out of many.** An advisory that covers four packages still produces one flat row. Read `affected` whenever the answer depends on which package is involved, or set `affectsPackage` so the scalars describe the one you care about. - **`cvssVersion` is read from the vector, not assumed.** Much of the database is scored under 3.0 rather than 3.1, and v4 is preferred when a v4 vector exists. Branch on this field before applying a scoring rubric. A `cvssScore` of `0` with a `null` vector means unscored, not harmless. - **Hitting `maxResults` stops the run mid-sweep,** so any sweeps queued behind it never execute even though they are counted in `sweepsRequested`. Raise the cap or narrow the filters rather than assuming the later combinations returned nothing. - **Budget exhaustion stops cleanly,** keeping everything already saved, setting `rateLimited: true` and recording the reset time. The run does not hammer on into refusals. - **Bad filter values fail before any request.** An unknown ecosystem, an unknown severity or a malformed `publishedFrom` throws immediately rather than burning budget. - **Transient failures are retried** three times with linear backoff. A rejected filter, a credential problem or an unexpected shape is final for that sweep and is recorded in `failures`; the next sweep still runs. The Actor only throws when every sweep failed. - **Advisories are sorted newest first within each sweep,** so a truncated run keeps the most recent material rather than a random slice. - **`firstPatchedVersion` of `null` means no fix has shipped** for that package, which is the row to escalate, not the row to skip. ## Recipes **Feed for the ecosystems you ship.** One sweep per ecosystem, bounded by date, run on a schedule with `publishedFrom` set to the previous run's date. ```json { "ecosystems": ["npm", "pip", "go"], "publishedFrom": "2026-07-01", "maxResults": 1000 } ``` Diff on `ghsaId` against what you already hold, then match `affected[].packageName` and `vulnerableVersionRange` to your lockfiles. **Watch one package.** Cheap, precise, and the scalar fields describe the package you named. ```json { "affectsPackage": "lodash", "ecosystems": ["npm"] } ``` Alert on any `ghsaId` you have not seen before; `firstPatchedVersion` is the upgrade target. **Cross-reference against another database.** Everything serious over a period, then join on `cveId`. ```json { "severities": ["critical", "high"], "publishedFrom": "2026-01-01", "maxResults": 2000, "githubToken": "ghp_…" } ``` Rows with `cveId: null` are the ones GitHub has published and the CVE registry has not caught up with. **Bulk export with a token.** Wide and deep, which is only realistic with the raised budget. ```json { "ecosystems": ["npm", "pip", "maven", "go"], "publishedFrom": "2025-01-01", "githubToken": "ghp_…", "maxResults": 20000 } ``` Four sweeps, so check `RUN_SUMMARY.requestsUsed` and `rateLimit.remaining` before scheduling a second run in the same hour. **Rule of thumb for the token.** Under about 5000 advisories in a single sweep, or a few hundred spread over two or three sweeps, you do not need one. Beyond that, pass `githubToken` or expect `rateLimited: true`. A classic token with no scopes ticked is enough.