JavaScript minimal site crawler (Crawlee + Cheerio)
A minimal site crawler for developers who already know the Apify SDK and Crawlee.
src/main.js
src/routes.js
1import { CheerioCrawler } from '@crawlee/cheerio';2import { Actor } from 'apify';3
4// this is ESM project, and as such, it requires you to specify extensions in your relative imports5// read more about this here: https://nodejs.org/docs/latest-v18.x/api/esm.html#mandatory-file-extensions6import { router } from './routes.js';7
8await Actor.init();9
10// `checkAccess` flag ensures the proxy credentials are valid, but the check can take a few hundred milliseconds.11// Disable it for short runs if you are sure your proxy configuration is correct12const proxyConfiguration = await Actor.createProxyConfiguration({ checkAccess: true });13
14const crawler = new CheerioCrawler({15 proxyConfiguration,16 requestHandler: router,17});18
19await crawler.run(['https://example.com']);20
21await Actor.exit();This is a project skeleton to help you bootstrap CheerioCrawler web scraping projects in JavaScript faster. It will always use the most up-to-date configuration and include all the common files. It's made for developers already familiar with Apify SDK and Crawlee libraries.
If you're looking for examples or want to learn how to use Apify, Apify SDK, or Crawlee, check out the other templates .
- Video tutorial on building a scraper using CheerioCrawler
- Written tutorial on building a scraper using CheerioCrawler
- How to scrape a dynamic page using Cheerio
- Video guide on getting data using Apify API
- Integration with GitHub, Zapier, Make, Google Drive and others
- A short guide on how to create Actors using code templates
JavaScript site crawler (Crawlee + Cheerio)
A fast crawler that follows links and gets data from static pages, with Crawlee handling retries, queues, and proxies. Good for blogs, news, or product listings, but it can't run client-side JavaScript.
JavaScript one-page scraper (Cheerio)
A scraper that gets data from one web page with Cheerio. The simplest way to start scraping.
JavaScript browser crawler (Crawlee + Puppeteer + Chrome)
A browser crawler that renders JavaScript in headless Chrome, with Crawlee handling retries, queues, and proxies. Good for social feeds, dashboards, or single-page apps.
JavaScript browser crawler (Crawlee + Playwright + Chrome)
A browser crawler that renders JavaScript, with Crawlee handling retries, queues, and proxies. Good for social feeds, dashboards, or single-page apps.
JavaScript stealth crawler (Crawlee + Playwright + Camoufox)
A browser crawler built to get past bot protection. Uses Camoufox, a Firefox build that looks like a real user.
JavaScript test runner (Cypress)
A cloud test runner for Cypress that saves results and videos.