---
name: github-repo-scraper
description: Read public GitHub repositories as structured records via the Apify Actor arman-bd/github-repo-scraper. Returns stars, forks, watchers, open issues, licence, topics, the full language byte map, the last 10 releases and the top 10 contributors for each repo. Use when a task needs repository metadata, OSS traction tracking, dependency due diligence, release cadence, licence audits or contributor bus-factor checks, especially for many repos at once. Not for issues, pull requests, commits, code search or private repositories without a token.
---

# GitHub Repository Scraper

Apify Actor `arman-bd/github-repo-scraper`. Give it a list of repositories, get one
dataset record per repo. It runs without credentials; a GitHub personal access token
is optional and only raises the request budget.

## When to use it

- A watchlist of repos has to be turned into rows: stars, forks, licence, language mix.
- You need release history or the top contributors for a handful of projects.
- You are auditing dependencies for licence, maintenance status or bus factor.
- Any of the above on a schedule, diffing `stars`, `forks` and `pushedAt` over time.

## When not to use it

- Issues, pull requests, commits, code search, gists, users or organisations. None of
  those are returned.
- Private repositories, unless you pass a token that can already see them.
- Repo-by-repo interactive lookups where one HTTP call of your own would do. This
  Actor earns its keep on lists.

## Call it

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('arman-bd/github-repo-scraper').call({
  repositories: ['facebook/react', 'vercel/next.js'],
  includeReleases: true,
  includeContributors: false,
  includeLanguages: false,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
const { value: summary } = await client
  .keyValueStore(run.defaultKeyValueStoreId)
  .getRecord('RUN_SUMMARY');
```

One-shot over HTTP, when you want the rows back in the same request:

```bash
curl -X POST "https://api.apify.com/v2/acts/arman-bd~github-repo-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"repositories":["facebook/react"],"includeReleases":false,"includeContributors":false,"includeLanguages":false}'
```

The Actor is also exposed through Apify's MCP server as `arman-bd/github-repo-scraper`,
so an MCP-capable agent can call it with no extra wiring.

## Input

| Field | Type | Required | Default | Notes |
|---|---|---|---|---|
| `repositories` | string[] | yes | | `owner/repo`, a `https://github.com/owner/repo` URL, or a git clone string. Mixed forms are fine and are normalised. Duplicates are removed before any request. |
| `includeReleases` | boolean | no | `true` | Last 10 releases plus the newest stable one as `latestRelease`. Costs one extra request per repo. |
| `includeContributors` | boolean | no | `true` | Top 10 contributors by commit count. Costs one extra request per repo. |
| `includeLanguages` | boolean | no | `true` | Full `{ language: bytes }` map. Costs one extra request per repo. |
| `githubToken` | string | no | `""` | Optional personal access token. Needs no scopes. Raises the budget from 60 to 5000 requests per hour. Secret field. |

**The three booleans are the whole cost model.** All off, a repo costs 1 request; all
on, 4. Without a token that is roughly 60 repos an hour versus roughly 15. Stars,
forks, watchers, open issues, licence, topics, the primary language *name*, dates and
archive status all ride along on the base request and are always present, so never
switch an extra on to reach those. Extras on for depth on a short list, off for
breadth on a long one.

## Output

One record per repository that resolved.

| Field | Type | Notes |
|---|---|---|
| `requestedAs` | string | Exactly what you passed in, normalised to `owner/repo`. Join back to your input list on this. |
| `fullName` | string | The canonical `owner/repo` GitHub resolved to. Differs from `requestedAs` for renamed or transferred repos. |
| `owner` | string | Owner login. |
| `description` | string \| null | Repo tagline. |
| `url` | string | Repo URL. |
| `homepage` | string \| null | The project's own site, when it declares one. |
| `stars` | number | Stargazers. |
| `forks` | number | Forks. |
| `watchers` | number | Real subscriber count, not the star count. See below. |
| `openIssues` | number | Open issues, GitHub counts pull requests in this. |
| `license` | string \| null | SPDX identifier where GitHub has one (`MIT`, `Apache-2.0`), otherwise the licence name. |
| `topics` | string[] | Topic tags. |
| `primaryLanguage` | string \| null | Dominant language. Always present, no flag needed. |
| `languages` | object \| null | `{ language: bytes }`, only when `includeLanguages` is on. |
| `defaultBranch` | string | Branch name. |
| `isArchived` | boolean | Archived repos are read-only upstream. |
| `isFork` | boolean | |
| `sizeKb` | number | Checkout size in KB. |
| `createdAt` | string | ISO 8601. |
| `pushedAt` | string | Last push. This, not `createdAt`, is the activity signal. |
| `latestRelease` | object \| null | Newest stable release, only when `includeReleases` is on. Keys: `tagName`, `name`, `publishedAt`, `isPrerelease`, `isDraft`, `url`, `notes`. |
| `releases` | object[] \| null | Last 10, same shape as `latestRelease`. |
| `contributors` | object[] \| null | Top 10, only when `includeContributors` is on. Keys: `login`, `contributions`, `type`, `url`. |
| `scrapedAt` | string | Run timestamp, ISO 8601. |

A real record, long strings trimmed:

```json
{
  "requestedAs": "facebook/react",
  "fullName": "react/react",
  "owner": "react",
  "description": "The library for web and native user interfaces.",
  "url": "https://github.com/react/react",
  "homepage": "https://react.dev",
  "stars": 247059,
  "forks": 51190,
  "watchers": 6600,
  "openIssues": 1248,
  "license": "MIT",
  "topics": ["declarative", "frontend", "javascript", "library", "react", "ui"],
  "primaryLanguage": "JavaScript",
  "languages": { "JavaScript": 5659949, "Rust": 2965252, "TypeScript": 2567778 },
  "defaultBranch": "main",
  "isArchived": false,
  "isFork": false,
  "sizeKb": 1045734,
  "createdAt": "2013-05-24T16:15:54Z",
  "pushedAt": "2026-08-05T19:11:16Z",
  "latestRelease": {
    "tagName": "v19.2.8",
    "name": "19.2.8 (July 21st, 2026)",
    "publishedAt": "2026-07-21T15:49:09Z",
    "isPrerelease": false,
    "isDraft": false,
    "url": "https://github.com/react/react/releases/tag/v19.2.8",
    "notes": "## React Server Components …"
  },
  "contributors": [
    { "login": "sebmarkbage", "contributions": 1939, "type": "User", "url": "https://github.com/sebmarkbage" }
  ],
  "scrapedAt": "2026-08-06T12:00:00.000Z"
}
```

## RUN_SUMMARY

Written to the run's key-value store under the key `RUN_SUMMARY`. **Read it.** It is
where a partial run admits that it was partial.

```json
{
  "reposRequested": 4,
  "reposSaved": 4,
  "reposFailed": 0,
  "reposSkipped": 0,
  "failures": [],
  "skipped": [],
  "requestsUsed": 16,
  "requestsPerRepo": 4,
  "rateLimited": false,
  "rateLimit": { "limit": 60, "remaining": 44, "resetsAt": "2026-08-06T13:12:11.000Z", "authenticated": false },
  "filters": { "includeReleases": true, "includeContributors": true, "includeLanguages": true },
  "finishedAt": "2026-08-06T12:00:07.221Z"
}
```

`reposSaved` short of `reposRequested` is normal and is explained by `failures` plus
`skipped`. If `rateLimited` is `true`, the remainder is in `skipped` and can be
re-run after `rateLimit.resetsAt`.

## Behaviour to plan around

- **`watchers` is not `stars`.** GitHub's raw `watchers_count` is a legacy alias that
  returns the star count. This Actor reports the real subscriber count instead, so the
  two fields differ here where they would agree in a raw API response. Do not
  reconcile them.
- **`fullName` can differ from what you asked for.** Renamed and transferred repos
  redirect, and the Actor follows the redirect. Key your own tables on `requestedAs`,
  or on both.
- **One bad repo never aborts the run.** A typo or an unreachable repo lands in
  `RUN_SUMMARY.failures` and the run continues. The Actor only throws when every repo
  failed, or when the budget was gone before a single repo could be read.
- **Budget exhaustion stops cleanly, before a partial row.** The remaining budget is
  checked before each repo, so you never get a record with the extras silently
  missing. Anything not reached is listed in `skipped`.
- **Transient failures are retried** three times with linear backoff. Not-found and
  permission errors are final for that repo and are not retried.
- **Empty repos are handled.** A repo with no commits returns empty `contributors` and
  `languages` rather than an error.
- **`openIssues` includes pull requests.** GitHub counts them together. There is no
  separate PR count here.
- **`releases` is release objects, not tags.** A project that ships tags without
  creating GitHub releases comes back with `releases: []`.

## Recipes

**Track competitor traction weekly.** Metrics plus release cadence, extras trimmed to
keep a long list inside the unauthenticated budget.

```json
{
  "repositories": ["vercel/next.js", "remix-run/react-router", "nuxt/nuxt", "sveltejs/kit"],
  "includeReleases": true,
  "includeContributors": false,
  "includeLanguages": false
}
```

Diff `stars`, `forks` and `pushedAt` against the previous run, keyed on `requestedAs`.

**Leaderboard over a long list.** Everything off, 1 request per repo, so roughly 60
repos fit in an unauthenticated hour and about 5000 with a token.

```json
{
  "repositories": ["torvalds/linux", "facebook/react", "tensorflow/tensorflow"],
  "includeReleases": false,
  "includeContributors": false,
  "includeLanguages": false
}
```

**Dependency due diligence.** Everything on, short list. `license` for legal,
`pushedAt` and `isArchived` for maintenance, `contributors` for bus factor, `releases`
for cadence.

```json
{
  "repositories": ["expressjs/express", "lodash/lodash"],
  "includeReleases": true,
  "includeContributors": true,
  "includeLanguages": true,
  "githubToken": "ghp_…"
}
```

**Rule of thumb for the token.** Under a dozen repos with extras on, or under 60 with
extras off, you do not need one. Above that, pass `githubToken` or expect `skipped` to
be non-empty. A classic token with no scopes ticked is enough.