My Actor 291
Try for free
No credit card required
Go to Store
Do you want to learn more about this Actor?
Get a demo.actor/Dockerfile
1# Specify the base Docker image. You can read more about
2# the available images at https://docs.apify.com/sdk/js/docs/guides/docker-images
3# You can also use any other image from Docker Hub.
4FROM apify/actor-node:18
5
6# Copy just package.json and package-lock.json
7# to speed up the build using Docker layer cache.
8COPY package*.json ./
9
10# Install NPM packages, skip optional and development dependencies to
11# keep the image small. Avoid logging too much and print the dependency
12# tree for debugging
13RUN npm --quiet set progress=false \
14 && npm install --omit=dev --omit=optional \
15 && echo "Installed NPM packages:" \
16 && (npm list --omit=dev --all || true) \
17 && echo "Node.js version:" \
18 && node --version \
19 && echo "NPM version:" \
20 && npm --version \
21 && rm -r ~/.npm
22
23# Next, copy the remaining files and directories with the source code.
24# Since we do this after NPM install, quick build will be really fast
25# for most source file changes.
26COPY . ./
27
28
29# Run the image.
30CMD npm start --silent.actor/actor.json
1{
2 "actorSpecification": 1,
3 "name": "my-actor-29",
4 "title": "Scrape single page in JavaScript",
5 "description": "Scrape data from single page with provided URL.",
6 "version": "0.0",
7 "meta": {
8 "templateId": "js-start"
9 },
10 "input": "./input_schema.json",
11 "dockerfile": "./Dockerfile"
12}.actor/input_schema.json
1{
2 "title": "Scrape data from a web page",
3 "type": "object",
4 "schemaVersion": 1,
5 "properties": {
6 "pageFunction": {
7 "title": "Page function",
8 "type": "string",
9 "description": "JavaScript (ES6) function that is executed in the context of every page loaded in the Chrome browser. Use it to scrape data from the page, perform actions or add new URLs to the request queue.<br><br>For details, see <a href='https://apify.com/apify/web-scraper#page-function' target='_blank' rel='noopener'>Page function</a> in README.",
10 "prefill": "// The function accepts a single argument: the \"context\" object.\n// For a complete list of its properties and functions,\n// see https://apify.com/apify/web-scraper#page-function \nasync function pageFunction(context) {\n // This statement works as a breakpoint when you're trying to debug your code. Works only with Run mode: DEVELOPMENT!\n // debugger; \n\n // jQuery is handy for finding DOM elements and extracting data from them.\n // To use it, make sure to enable the \"Inject jQuery\" option.\n const $ = context.jQuery;\n const pageTitle = $('title').first().text();\n\n // Print some information to actor log\n context.log.info(`URL: ${context.request.url}, TITLE: ${pageTitle}`);\n\n // Manually add a new page to the queue for scraping.\n await context.enqueueRequest({ url: 'http://www.example.com' });\n\n // Return an object with the data extracted from the page.\n // It will be stored to the resulting dataset.\n return {\n url: context.request.url,\n pageTitle\n };\n}",
11 "editor": "javascript"
12 }
13 },
14 "required": ["pageFunction"]
15}src/main.js
1// Axios - Promise based HTTP client for the browser and node.js (Read more at https://axios-http.com/docs/intro).
2import axios from 'axios';
3// Cheerio - The fast, flexible & elegant library for parsing and manipulating HTML and XML (Read more at https://cheerio.js.org/).
4import * as cheerio from 'cheerio';
5// Apify SDK - toolkit for building Apify Actors (Read more at https://docs.apify.com/sdk/js/).
6import { Actor } from 'apify';
7// this is ESM project, and as such, it requires you to specify extensions in your relative imports
8// read more about this here: https://nodejs.org/docs/latest-v18.x/api/esm.html#mandatory-file-extensions
9// import { router } from './routes.js';
10
11// The init() call configures the Actor for its environment. It's recommended to start every Actor with an init().
12await Actor.init();
13
14// Structure of input is defined in input_schema.json
15const input = await Actor.getInput();
16const { url } = input;
17
18// Fetch the HTML content of the page.
19const response = await axios.get(url);
20
21// Parse the downloaded HTML with Cheerio to enable data extraction.
22const $ = cheerio.load(response.data);
23
24// Extract all headings from the page (tag name and text).
25const headings = [];
26$("h1, h2, h3, h4, h5, h6").each((i, element) => {
27 const headingObject = {
28 level: $(element).prop("tagName").toLowerCase(),
29 text: $(element).text(),
30 };
31 console.log("Extracted heading", headingObject);
32 headings.push(headingObject);
33});
34
35// Save headings to Dataset - a table-like storage.
36await Actor.pushData(headings);
37
38// Gracefully exit the Actor process. It's recommended to quit all Actors with an exit().
39await Actor.exit();.dockerignore
1# configurations
2.idea
3
4# crawlee and apify storage folders
5apify_storage
6crawlee_storage
7storage
8
9# installed files
10node_modules
11
12# git folder
13.git.gitignore
1# This file tells Git which files shouldn't be added to source control
2.DS_Store
3.idea
4dist
5node_modules
6apify_storage
7storage/*
8!storage/key_value_stores
9storage/key_value_stores/*
10!storage/key_value_stores/default
11storage/key_value_stores/default/*
12!storage/key_value_stores/default/INPUT.jsonpackage.json
1{
2 "name": "js-scrape-single-page",
3 "version": "0.0.1",
4 "type": "module",
5 "description": "This is an example of an Apify actor.",
6 "engines": {
7 "node": ">=18.0.0"
8 },
9 "dependencies": {
10 "apify": "^3.1.10",
11 "axios": "^1.5.0",
12 "cheerio": "^1.0.0-rc.12"
13 },
14 "scripts": {
15 "start": "node ./src/main.js",
16 "test": "echo \"Error: oops, the actor has no tests yet, sad!\" && exit 1"
17 },
18 "author": "It's not you it's me",
19 "license": "ISC"
20}Developer
Jan Kirchner Maintained by Apify
Actor Metrics
1 monthly user
-
1 star
>99% runs succeeded
Created in Oct 2023
Modified a year ago
Categories
