Crawl websites and extract video transcripts. Domain-specific parsers for legal, news, real estate, finance. RAG-ready chunks with metadata.
# Crawl any website → structured data
import httpx
resp = httpx.post("https://api.crawlkit.ai/v1/scrape", json={
"url": "https://example.com/article",
"chunk": True
}, headers={"Authorization": "Bearer ck_xxx"})
data = resp.json()["data"]
print(data["title"]) # → "Article Title"
print(data["chunks"]) # → RAG-ready chunks
Auto-detects static vs JS-heavy pages. Handles SPAs and dynamic content.
Extract transcripts from YouTube, TikTok, Facebook. No download needed.
Specialized parsers for legal, news, real estate, finance domains.
Detects content type automatically. Applies the right parser.
Markdown, JSON output. Chunks with metadata for vector databases.
Crawl hundreds of URLs. Discover content from sitemaps.