{ "openapi": "3.0.1", "info": { "title": "Datasource Pulse · Actor, API & Scraper Health Monitor", "description": "Scraper monitoring and API monitoring for your whole pipeline: monitor the health of your Actors, scrapers, and API keys. Catches schema drift, empty results, stale data, and quota exhaustion. One alert when something degrades, one notice when it recovers. Learn more: datasourcepulse.com", "version": "0.1", "x-build-id": "wYbEae5K98alefVVf" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/growth_wizard~datasource-pulse/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-growth_wizard-datasource-pulse", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/growth_wizard~datasource-pulse/runs": { "post": { "operationId": "runs-sync-growth_wizard-datasource-pulse", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/growth_wizard~datasource-pulse/run-sync": { "post": { "operationId": "run-sync-growth_wizard-datasource-pulse", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "clientToken": { "title": "Client Token (Mode B · managed clients)", "type": "string", "description": "Enterprise / managed onboarding. Supply only this token; your full connector manifest is loaded from the operator's private KV store. Leave blank for Mode A (self-serve) where you configure checks directly below." }, "checks": { "title": "Connector checks (Mode A · self-serve)", "type": "array", "description": "Array of connector checks to run; billed $0.02 per item. Each item is ONE of two object shapes.\n\nCREDENTIAL CHECK: { \"type\": \"credential\", \"id\": string (unique key), \"label\": string (human name), \"vendor\": one of [\"serpapi\",\"apify\",\"hunter\",\"openai\",\"anthropic\",\"twitter\",\"reddit\",\"github\",\"stripe\",\"resend\",\"producthunt\",\"gemini\",\"groq\",\"mistral\",\"cohere\",\"perplexity\",\"together\",\"openrouter\",\"firecrawl\",\"exa\",\"tavily\",\"brightdata\",\"scrapingbee\",\"zenrows\",\"elevenlabs\",\"deepgram\",\"assemblyai\",\"replicate\",\"huggingface\",\"pinecone\",\"supabase\",\"airtable\",\"stackexchange\",\"google_oauth\",\"youtube\",\"serper\",\"brave\",\"generic_http\"], + credential fields for that vendor.\n\nCredential fields by vendor: \"apiKey\" (serpapi, apify, hunter, openai, anthropic, github, stripe [restricted read-only key only], resend, producthunt, gemini, groq, mistral, cohere, perplexity, together, openrouter, firecrawl, exa, tavily, brightdata, scrapingbee, zenrows, elevenlabs, deepgram, assemblyai, replicate, huggingface, pinecone, airtable, stackexchange, youtube, serper, brave, and supabase management tokens sbp_...); \"clientId\"+\"clientSecret\" (reddit); \"clientId\"+\"clientSecret\"+\"refreshToken\" (google_oauth · validates a Google OAuth refresh token via a free token mint); \"bearerToken\" (twitter); \"url\"+\"apiKey\" (supabase project-key check · url like https://YOURPROJECT.supabase.co); \"url\"+\"authHeader\"+\"successStatusCode\"+optional \"successJsonPath\"+\"successJsonValue\" (generic_http).\n\nACTOR CHECK: { \"type\": \"actor\", \"id\": string (unique key), \"label\": string, \"actorId\": string (e.g. \"clockworks~tiktok-scraper\"), \"apifyToken\": string (Actor read+run-read scope), \"enableVolumeBaseline\": boolean, \"enableCostBaseline\": boolean, optional \"stalenessThresholdDays\": int (default 30), \"volumeDropPct\": int (default 30), \"costSpikePct\": int (default 50), \"zeroYieldRunThreshold\": int (default 3; 0 disables) }.\n\nzeroYieldRunThreshold: alert when this many consecutive most-recent SUCCEEDED runs each produced 0 dataset items (silent zero-yield: run status green, dataset empty · run status alone never reveals this). Needs no baseline, so it works from a source's very first runs. Set 0 for sources where an empty result is routinely legitimate.\n\nSet enableVolumeBaseline/enableCostBaseline to false when each run uses different inputs (varying queries), or you will get false alarms. Optional on BOTH shapes: \"impact\": string (max 200 chars) · what breaks downstream when this connector fails; echoed verbatim in every alert so the operator's first question (\"what breaks?\") is answered in the alert itself. Optional on CREDENTIAL checks: \"quotaSoftCap\": number · YOUR OWN budget line in the vendor's native usage unit (searches for serpapi, USD for apify/openrouter, requests for hunter, credits for firecrawl/scrapingbee, characters for elevenlabs, API calls for tavily/stackexchange); quota warnings and the 100%-crossed warning state are evaluated against your cap, with the vendor-account percentage always shown alongside.\n\nSee README for per-vendor examples." }, "onlyConnectorIds": { "title": "Recheck only these connector ids (subset recheck)", "type": "array", "description": "On-demand subset recheck for preflight gates: when set, ONLY the listed connector ids (from your checks array, or from your managed manifest in Mode B) are probed and billed this run · e.g. recheck a single credential immediately before an expensive job instead of paying for the full roster. Unknown ids stop the run before any connector is charged and write a guidance record listing the known ids. Leave empty for a full run. Works in both Mode A and Mode B.", "items": { "type": "string" } }, "emailTo": { "title": "Alert email address", "type": "string", "description": "Email address to receive state-change alerts. Managed (Mode B) plans get native email delivery; in Mode A this address is stamped into a structured EMAIL_ALERT log event for relaying via an Apify webhook to your email provider (see README, Alerting). Leave blank to skip email alerts." }, "webhookUrl": { "title": "Webhook URL (Slack, Discord, or any HTTP endpoint)", "type": "string", "description": "Webhook endpoint that receives a JSON POST on every state transition. The default body is a Slack/Discord-compatible notification card; set webhookFormat to \"json\" for a raw structured event body instead." }, "webhookSecret": { "title": "Webhook signing secret (optional)", "type": "string", "description": "When set, every webhook POST includes X-DP-Signature: sha256=, so your receiver can verify the alert genuinely came from Datasource Pulse." }, "webhookFormat": { "title": "Webhook body format", "enum": [ "slack", "json" ], "type": "string", "description": "\"slack\" (default) posts a Slack/Discord-compatible notification card · pick this when a person reads the alerts in a chat channel. \"json\" posts the raw structured event object with top-level fields transition, connector_id, connector_label, failure_class, message, last_healthy_at, occurred_at, recommended_action, impact (present when the check declares one), and metadata · pick this when a program consumes your alerts and indexes them by field. X-DP-Signature signs the raw body in both formats.", "default": "slack" }, "webhookExpectJsonOk": { "title": "Require {\"ok\":true} webhook acknowledgement", "type": "boolean", "description": "When enabled, a 2xx webhook response must carry the JSON body {\"ok\":true} to count as delivered. Guards against catch-all 200 pages (framework fallback routes) silently swallowing alerts. Leave off for Slack/Discord · they acknowledge with plain \"ok\".", "default": false }, "quotaWarningPct": { "title": "Quota warning threshold (%)", "minimum": 1, "maximum": 99, "type": "integer", "description": "Fire a warning alert when a credential's usage exceeds this percentage of its quota limit. Default: 80.", "default": 80 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } }