website-content-crawler avatar
website-content-crawler

Deprecated

Pricing

Pay per usage

See alternative Actors
Go to Store
website-content-crawler

website-content-crawler

Deprecated

See alternative Actors

Developed by

Matěj Šesták

Matěj Šesták

Maintained by Community

0.0 (0)

Pricing

Pay per usage

1

Total users

2

Monthly users

2

Last modified

2 years ago

Automation

You can access the website-content-crawler programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

Python

JavaScript

JavaScript

CLI

CLI

OpenAPI

OpenAPI

HTTP

HTTP

MCP

MCP

$echo '{
<  "startUrls": [
<    {
<      "url": "https://docs.apify.com/academy/web-scraping-for-beginners/"
<    }
<  ],
<  "proxyConfiguration": {
<    "useApifyProxy": true
<  },
<  "removeElementsCssSelector": "header, nav, footer"
<}' |
<apify call acceptable_seahorse/website-content-crawler --silent --output-dataset

website-content-crawler API through CLI

The Apify CLI is the official tool that allows you to use website-content-crawler locally, providing convenience functions and automatic retries on errors.

Install the Apify CLI

$npm i -g apify-cli
$apify login

Other API clients include:

Python

website-content-crawler API in Python

JavaScript

website-content-crawler API in JavaScript

OpenAPI

website-content-crawler OpenAPI definition

HTTP

website-content-crawler API

CP

Crawler Puppeteer

jancurn/crawler-puppeteer

Copy of https://github.com/apifytech/actor-crawler-puppeteer

User avatar

Jan Curn

3

LP

Legacy Phantomjs Crawler

apify/legacy-phantomjs-crawler

User avatar

Apify Technologies

3

5.0

CS

Crawler Stats

drobnikj/crawler-stats

User avatar

Jakub Drobník

3

CR

Crawler

mtrunkat.com/crawler

Experimental implementation of Apify crawler in act.

User avatar

Marek Trunkát

2

CC

Check Crawler Results

drobnikj/check-crawler-results

Act checks first 100 results from crawler execution agains json schema. It returns validation errors to act log file. It sends mail if one or more results is not valid. ==Environment vars== - JSON_SCHEMA json schema for validation - MAILGUN_API_KEY mailgun api key for sending mail from act - MAI...

User avatar

Jakub Drobník

1

CI

Crawler In Act Test

drobnikj/crawler-in-act-test

User avatar

Jakub Drobník

1

Screenshot Url avatar

Screenshot Url

apify/screenshot-url

Actor serving as an example of Input Schema. Takes URL of website and screenshot configuration parameters as input and outputs a screenshot of the website into Key-Value store.

User avatar

Apify Technologies

25

Test-title avatar

Test-title

honzakirchner43/screenshot-url-mine

Actor serving as an example of Input Schema. Takes URL of website and screenshot configuration parameters as input and outputs a screenshot of the website into Key-Value store.

User avatar

Jan Kirchner

1

RAG Web Browser avatar

RAG Web Browser

apify/rag-web-browser

Web browser for OpenAI Assistants, RAG pipelines, or AI agents, similar to a web browser in ChatGPT. It queries Google Search, scrapes the top N pages, and returns their content as Markdown for further processing by an LLM. It can also scrape individual URLs. Supports Model Context Protocol (MCP).

User avatar

Apify Technologies

1

Screenshot Url 21 avatar

Screenshot Url 21

honzakirchner43/screenshot-url-21

Actor serving as an example of Input Schema. Takes URL of website and screenshot configuration parameters as input and outputs a screenshot of the website into Key-Value store.

User avatar

Jan Kirchner

1

Screen URL Duplicate avatar

Screen URL Duplicate

matej-test/screenshot-url-duplicate

Actor serving as an example of Input Schema. Takes URL of website and screenshot configuration parameters as input and outputs a screenshot of the website into Key-Value store.

User avatar

Matej Hamaš

1