Firecrawl
First met in aicp-c3-basecamp-03 · Scraping & data sources
FirecrawlFirst met in aicp-c3-basecamp-03 · Scraping & data sources
Scraping built for AI — hand it a URL and get clean markdown or structured JSON back, with an LLM doing the extraction.
You need page content in a shape a model can use, and you don't want to write or maintain selectors.
AI extraction: give it a prompt and a JSON schema and it returns structured fields, not HTML
Native n8n nodes with quick-connect, plus 100k free credits on the hobby plan
Open source — self-host it when the credits run out
Scrapes single pages (/scrape) and crawls sites, returning markdown and/or JSON
Extracts to a schema you define using a prompt
Offers proxy and stealth options for sites that resist
Runs as native nodes inside n8n with a credential connect flow
Doesn't handle every hostile target — geo-blocked and heavily defended sites want a proxy provider like Bright Data
Doesn't stay free forever, though self-hosting is the escape hatch
Doesn't do the platform-specific structured feeds Apify specializes in
The ingestion end of any content workflow: scrape → structure → process with an agent → publish.
Scraping & data sources
The pipeline reads local files, not the web — but the LMS-facing fetch scripts are the same problem shape.
If Outskill's LMS ever needs scraping for materials the API doesn't expose, this is the tool, and its schema-extraction mode maps neatly onto the manifest structure.
current
apify: Apify for platform-specific scrapers (LinkedIn, Instagram); Firecrawl for general pages
bright-data: geo-proxies in front of it for ban-resistant scraping
n8n: native nodes; the BC4 newsletter workflow's front end