# Optional Reddit collection watchdog for n8n

This checker distinguishes **healthy silence**, **failed/partial collection**, **stale sources**, **missed runs**, and **unreadable health**. It monitors an existing collector; it never starts an Actor run or fetches Reddit pages. Zero NEW or keyword matches is not an alarm condition.

Start with the [short troubleshooting guide](https://api.apify.com/v2/key-value-stores/9ZywSAVVA8tMMdMQC/records/reddit-collection-watchdog-v1-troubleshooting.md) if you are investigating a quiet feed.

Import [n8n-reddit-watchdog.json](https://api.apify.com/v2/key-value-stores/9ZywSAVVA8tMMdMQC/records/reddit-collection-watchdog-v1-n8n.json) from file. The workflow is inactive, its five-minute Schedule Trigger is disabled, `deliveryEnabled` is false, and no credentials or destination URLs are included. It is a separate optional template; the two already-public Reddit workflow files are unchanged.

## Setup

1. Use a **dedicated saved Apify task** that always runs one Reddit monitor name and scope. If your current collector uses an Actor endpoint directly, first decide whether to adopt a dedicated task; this template does not search mixed Actor histories or reconfigure your collector. Use the real task ID, not the Actor ID or a schedule ID.
2. In your n8n project, create a Data Table named `suslabs_watchdog_state` with four custom columns:

   | Column | n8n type | Initial value |
   |---|---|---|
   | `watchdogKey` | String | `reddit-hourly-watchdog` |
   | `leaseId` | String | empty string, not null |
   | `leaseUntil` | Number | `0` |
   | `stateJson` | String | `{}` |

   Create **exactly one row** for that watchdog key. The standard n8n `id`, `createdAt` and `updatedAt` system columns stay automatic. Do not reset or delete the row during an incident; it contains notification receipts/pending events. Back up the table with your n8n database.
3. In **Configure watchdog**, fill `taskId`, `actorId`, `monitorName`, and `expectedSources`. Use the exact `source` objects in a known-good `SUMMARY.sourceHealth`, e.g. `{"kind":"subreddit","name":"saas"}`. Matching is by object content, independent of key order. Expected source names are not silently normalized. Set a new key/seeded row for a different monitor scope.
4. Set `expectedIntervalMinutes` to the collector cadence. Defaults are hourly collection, 15-minute grace, maximum expected run of 10 minutes, reminders off (`reminderHours: 0`), and `gapAlertAfterRuns: 2`. Missed runs/source checks become stale after **2 × interval + grace**, not after the checker interval. Slow active runs have their own **max runtime + grace** threshold. Gap confirmation counts distinct completed SUCCEEDED runs observed by the checker, not repeated five-minute reads of the same run; set `gapAlertAfterRuns: 1` for immediate gap alerts, or another integer up to 100 for a longer threshold.
5. On all **three Apify GET nodes**, select your own Header Auth credential: name `Authorization`, value `Bearer YOUR_APIFY_TOKEN`. The credential must read that task's run metadata and the two records in its run stores. Use access scoped appropriately to those resources. Do not put a real token in the shared JSON. No INPUT record or scraped content is read.
6. Run a manual check with delivery still disabled. Read **Watchdog result** and the Data Table. A healthy zero-NEW run should have no active issues. An unhealthy check queues events even in this review mode, so they are visible for inspection. These queued events will be sent later if you enable delivery: review them first. To start with clean test history, use a new test key/row rather than deleting production receipts.
7. Configure `alertUrl` as your HTTPS receiver. It receives `{ "events": [...], "text": "..." }`. Set a **separate receiver credential** on **Send notification batch** if needed; never reuse the Apify credential there. The receiver must durably accept the entire batch before returning 2xx, and deduplicate each `events[].eventId`. A rejected/timeout response leaves the batch pending for the next check. The `Idempotency-Key` header identifies a batch; per-event receipts remain necessary when pending batches grow.
8. After a manual destination test, set `deliveryEnabled: true`, enable the Schedule Trigger, and activate/publish the workflow in your n8n version. Use an Error Workflow/external monitoring for checker failures. This template does not enable anything on import.

## Incident and delivery behavior

- A fresh source failure opens one source incident immediately even when the Apify run is SUCCEEDED. Missing/duplicate expected sources, invalid health fields and stale successful checks also remain immediate incidents.
- Coverage gaps are different: `SUMMARY.possibleGap` and each expected source's `coverageInterrupted` have independent pending counters. By default a gap must stay interrupted on the next distinct completed SUCCEEDED run before opening its gap incident. An initial backfill or a brief gap that clears on the next run produces neither ALERT nor RECOVERY. Re-reading the same run ID counts once; failed/unreadable runs cannot confirm a gap. A persisting `item-limit` gap says **raise maxItems or run more often**. Once a gap has alerted, a clean successful observation sends one recovery. `gapAlertAfterRuns: 1` restores immediate gap alerts.
- Pending gap evidence (first/last run IDs and observed-run count) appears in **Watchdog result** and the saved state. It is not a notification outbox entry. Global and per-source gaps are separate incident keys; persistent flags at both scopes can produce separate events in the same notification batch. The checker samples the latest run history, so it cannot prove what happened in intermediate runs it did not observe.
- A failed/aborted/timed-out collector opens a run incident. A fresh RUNNING run does not hide the last completed failure. The checker reads at most the latest five runs; if all five are active, it waits for completed evidence and cannot establish source recovery.
- Failed metadata/record reads mean UNKNOWN. Existing source incidents stay open until fresh valid source health proves recovery. Task/Actor IDs and `MONITOR_STATE.name` must match configuration.
- One ALERT per incident, one RECOVERY when it clears. Optional reminders apply only after the configured interval and while that incident has no pending delivery. New incidents get new event IDs.
- Pending notifications are persisted **before sending**. After 2xx, their IDs are acknowledged and removed. A crash after receiver acceptance but before acknowledgement replays the same event IDs; delivery is at least once. Direct Slack/Discord incoming webhooks lack a durable receipt store and may duplicate a notification in this crash window. Use an idempotent receiver for that guarantee.
- If enabled notification delivery is rejected or times out, the checker saves the pending outbox, releases its lease, then fails visibly. A configured n8n Error Workflow can therefore report that delivery failure; the next scheduled check retries the same IDs. Use a different destination for that error handler if the primary receiver is unavailable.
- State holds incident metadata and a bounded pending outbox (100 events), not scraped posts. If the outbox fills, the checker fails visibly rather than discarding it. Resolve the receiver failure and inspect state. No automatic post inactivity/empty-fetch heuristic is shipped.

## Independence, overlap and operating cost

A dedicated workflow is independent of the collector logic, but not of its host. If both are on the same n8n installation, an installation outage stops both. Host the watchdog elsewhere for that case; monitor its own heartbeat from another service. This is a practical optional checker, not an uptime SLA.

Persistent state uses Data Table reads and conditional **compare-and-swap** updates against the row ID and previous lease values. The winner owns a five-minute lease; competing executions stop without sending. Subsequent writes require that same lease token. A 120-second workflow timeout and pre-send lease checks bound normal processing. A crashed lease expires after five minutes; platform pauses/process failures are still reasons to monitor the checker. Do not run copies with different tables against the same logical key; their locks cannot coordinate.

Each check makes **one task-run GET**, and, when a completed success is available, **two exact-store record GETs**. At five-minute checks that is up to **864 API GETs/day** plus n8n executions and Data Table operations. No new Actor execution, start fee or source-check event is generated by the watchdog. API/storage usage, n8n hosting and destination costs depend on the user's services; no dollar cost benchmark is claimed. A longer checker interval reduces overhead and increases detection delay.

## Validation and review

Run `node build-workflow.cjs`, then `node --test test-watchdog.cjs`. `prove-n8n.cjs` additionally imports the exact inactive JSON into a fresh isolated local n8n database, then executes **fixture copies** with every HTTP Request replaced by synthetic Code nodes. The fixture copies retain real Data Table lease/stage/acknowledgement nodes. In installed n8n 2.22.6, stock CLI execution omits the server's Data Table context; `cli-data-table-context.cjs` initializes its unchanged standard proxy in the proof process. This initializer is not part of the imported workflow and is unnecessary in a normal server with Data Tables enabled. Results are in `n8n-proof-summary.json` after a successful run.

Fixture execution does not prove your credentials, destination, unattended schedule, or another n8n version. Separately, Claude's read-only review on 24 October verified the required fields/types in two real Reddit Actor runs. That supports field compatibility, not a configured end-to-end watchdog proof. Before enabling, do a small read-only metadata and destination check in your own account. The Actor itself needs no new build.

Official references checked 24 October 2026: [Apify task runs](https://docs.apify.com/api/v2/actor-task-runs-get), [n8n Data Table node](https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-base.datatable/), [HTTP Request](https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-base.httprequest/). Data Tables were chosen because [workflow static data](https://docs.n8n.io/build/code-in-n8n/cookbook/built-in-methods-and-variables-examples/getworkflowstaticdata.md) is not persisted by manual test executions and has frequency-related limits.
