Actor with output schema

Under maintenance

Pricing

$14.00 / 1,000 results

Try for free

Go to Apify Store

Actor with output schema

Under maintenance

Try for free

Pricing

$14.00 / 1,000 results

Rating

5.0

(1)

Developer

Daniel Wébr

Maintained by Community

Actor stats

Bookmarked

Total users

Monthly active users

2 months ago

Last modified

.actor/Dockerfile

# Specify the base Docker image. You can read more about
# the available images at https://docs.apify.com/sdk/js/docs/guides/docker-images
# You can also use any other image from Docker Hub.
FROM apify/actor-node:18

# Copy just package.json and package-lock.json
# to speed up the build using Docker layer cache.
COPY package*.json ./

# Install NPM packages, skip optional and development dependencies to
# keep the image small. Avoid logging too much and print the dependency
# tree for debugging
RUN npm --quiet set progress=false \
    && npm install --omit=dev --omit=optional \
    && echo "Installed NPM packages:" \
    && (npm list --omit=dev --all || true) \
    && echo "Node.js version:" \
    && node --version \
    && echo "NPM version:" \
    && npm --version \
    && rm -r ~/.npm

# Next, copy the remaining files and directories with the source code.
# Since we do this after NPM install, quick build will be really fast
# for most source file changes.
COPY . ./


# Run the image.
CMD npm start --silent

.actor/actor.json

{
    "actorSpecification": 1,
    "name": "my-actor-4",
    "title": "Scrape single page in JavaScript",
    "description": "Scrape data from single page with provided URL.",
    "version": "0.0",
    "storages": {
		"dataset": "./dataset_schema.json"
	},
    "meta": {
        "templateId": "js-start"
    },
    "input": "./input_schema.json",
    "dockerfile": "./Dockerfile"
}

.actor/dataset_schema.json

{
    "actorSpecification": 1,
    "fields": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "type": "object",
        "properties": {
            "analysis": {
                "type": "array",
                "items": {
                    "type": "object",
                    "properties": {
                        "ticker": {
                            "type": "string"
                        },
                        "error": {
                            "type": "string"
                        },
                        "sentiment_analysis": {
                            "type": "object",
                            "properties": {
                                "google": {
                                    "type": "object",
                                    "properties": {
                                        "category": {
                                            "type": "string"
                                        },
                                        "confidence": {
                                            "type": "number"
                                        },
                                        "sentiment_score": {
                                            "type": "number"
                                        },
                                        "reasoning": {
                                            "type": "string"
                                        }
                                    }
                                },
                                "twitter": {
                                    "type": "object",
                                    "properties": {
                                        "category": {
                                            "type": "string"
                                        },
                                        "confidence": {
                                            "type": "number"
                                        },
                                        "sentiment_score": {
                                            "type": "number"
                                        },
                                        "reasoning": {
                                            "type": "string"
                                        }
                                    }
                                }
                            }
                        },
                        "market_summary": {
                            "type": "object",
                            "properties": {
                                "current_situation": {
                                    "type": "string"
                                },
                                "top_discussed_topics": {
                                    "type": "array"
                                },
                                "recent_information": {
                                    "type": "array"
                                },
                                "investor_interest": {
                                    "type": "string"
                                }
                            }
                        },
                        "personalized_recommendation": {
                            "type": "object",
                            "properties": {
                                "persona_analysis": {
                                    "type": "string"
                                },
                                "action": {
                                    "type": "string"
                                },
                                "summary": {
                                    "type": "string"
                                },
                                "reasoning": {
                                    "type": "string"
                                },
                                "potential_risks": {
                                    "type": "array"
                                }
                            }
                        }
                    }
                }
            }
        }
    },
    "views": {}
}

.actor/input_schema.json

{
    "title": "Scrape data from a web page",
    "type": "object",
    "schemaVersion": 1,
    "properties": {
        "url": {
            "title": "URL of the page",
            "type": "string",
            "description": "The URL of website you want to get the data from.",
            "editor": "textfield",
            "prefill": "https://www.apify.com/"
        }
    },
    "required": ["url"]
}

src/main.js

1// Axios - Promise based HTTP client for the browser and node.js (Read more at https://axios-http.com/docs/intro).
2import axios from 'axios';
3// Cheerio - The fast, flexible & elegant library for parsing and manipulating HTML and XML (Read more at https://cheerio.js.org/).
4import * as cheerio from 'cheerio';
5// Apify SDK - toolkit for building Apify Actors (Read more at https://docs.apify.com/sdk/js/).
6import { Actor } from 'apify';
7// this is ESM project, and as such, it requires you to specify extensions in your relative imports
8// read more about this here: https://nodejs.org/docs/latest-v18.x/api/esm.html#mandatory-file-extensions
9// import { router } from './routes.js';
10
11// The init() call configures the Actor for its environment. It's recommended to start every Actor with an .
12await Actor.init();
13
14// Structure of input is defined in input_schema.json
15const input = await Actor.getInput();
16const { url } = input;
17
18// Fetch the HTML content of the page.
19const response = await axios.get(url);
20
21// Parse the downloaded HTML with Cheerio to enable data extraction.
22const $ = cheerio.load(response.data);
23
24// Extract all headings from the page (tag name and text).
25const headings = [];
26$("h1, h2, h3, h4, h5, h6").each((i, element) => {
27    const headingObject = {
28        level: $(element).prop("tagName").toLowerCase(),
29        text: $(element).text(),
30    };
31    console.log("Extracted heading", headingObject);
32    headings.push({
33        "_id": "6613b694115ce0746526cace",
34        "index": 0,
35        "guid": "6b641c15-4525-4821-9e13-a1b80687a0c1",
36        "isActive": true,
37        "balance": "$2,274.85",
38        "picture": "http://placehold.it/32x32",
39        "age": 38,
40        "eyeColor": "green",
41        "name": "Thomas Carver",
42        "gender": "male",
43        "company": "ORBOID",
44        "email": "thomascarver@orboid.com",
45        "phone": "+1 (812) 467-3843",
46        "address": "310 Apollo Street, Strong, Missouri, 6644",
47        "about": "Adipisicing nisi ex magna nisi esse do non ea nulla tempor anim voluptate aliquip. Ut commodo ex magna velit. Aliqua do eiusmod qui mollit dolore dolor nulla voluptate voluptate ullamco ullamco quis. Voluptate amet ea culpa ut laborum enim elit cupidatat labore non consequat. Ut anim aliquip nulla ullamco reprehenderit enim ex.\r\n",
48        "registered": "2024-02-15T09:59:11 -01:00",
49        "latitude": 34.573829,
50        "longitude": -16.193091,
51        "tags": [
52        "exercitation",
53        "est",
54        "consequat",
55        "esse",
56        "mollit",
57        "reprehenderit",
58        "incididunt"
59        ],
60        "friends": [
61        {
62            "id": 0,
63            "name": "Kelsey Hayes"
64        },
65        {
66            "id": 1,
67            "name": "Kimberly Simmons"
68        },
69        {
70            "id": 2,
71            "name": "Nelda Pittman"
72        }
73        ],
74        "greeting": "Hello, Thomas Carver! You have 1 unread messages.",
75        "favoriteFruit": "strawberry"
76    });
77});
78
79// Save headings to Dataset - a table-like storage.
80await Actor.pushData(headings);
81
82// Gracefully exit the Actor process. It's recommended to quit all Actors with an exit().
83await Actor.exit();

.dockerignore

# configurations
.idea

# crawlee and apify storage folders
apify_storage
crawlee_storage
storage

# installed files
node_modules

# git folder
.git

.gitignore

# This file tells Git which files shouldn't be added to source control
.DS_Store
.idea
dist
node_modules
apify_storage
storage/*
!storage/key_value_stores
storage/key_value_stores/*
!storage/key_value_stores/default
storage/key_value_stores/default/*
!storage/key_value_stores/default/INPUT.json

package.json

{
    "name": "js-scrape-single-page",
    "version": "0.0.1",
    "type": "module",
    "description": "This is an example of an Apify actor.",
    "engines": {
        "node": ">=18.0.0"
    },
    "dependencies": {
        "apify": "^3.1.10",
        "axios": "^1.5.0",
        "cheerio": "^1.0.0-rc.12"
    },
    "scripts": {
        "start": "node ./src/main.js",
        "test": "echo \"Error: oops, the actor has no tests yet, sad!\" && exit 1"
    },
    "author": "It's not you it's me",
    "license": "ISC"
}

Output schema

honzakirchner43/output-schema

Jan Kirchner

Actor With Ui2

mtrunkat.com/actor-with-ui2

Marek Trunkát

Actor with synthetic event

knowing_didgeridoo/actor-with-synthetic-event

Jan Novotny

Output example that fails

botuitestsadmin/output-example-that-fails

Kacka Test

My Actor

rezaczu/actor-name

Zuzana Řezáčová

My Actor

test-fest/my-actor

test-fest

4.1

Test Public With Env Vars

someuser/test-public-with-env-vars

fwef wefwef

Test Input Fields

gip/test-input-fields

Actor with different input fields from input schema

Jaroslav Hejlek

3.3

Testing Actor With Hidden Sourceasd

pentest_citadelo/my-actor-2

asdasd

Pentestx Citadelox

5.0

PPR Actor

ruly_villa/ppr-actor

Zuzana Štětinová

4.7

# Specify the base Docker image. You can read more about # the available images at https://docs.apify.com/sdk/js/docs/guides/docker-images # You can also use any other image from Docker Hub. FROM apify/actor-node:18 # Copy just package.json and package-lock.json # to speed up the build using Docker layer cache. COPY package*.json ./ # Install NPM packages, skip optional and development dependencies to # keep the image small. Avoid logging too much and print the dependency # tree for debugging RUN npm --quiet set progress=false \ && npm install --omit=dev --omit=optional \ && echo "Installed NPM packages:" \ && (npm list --omit=dev --all || true) \ && echo "Node.js version:" \ && node --version \ && echo "NPM version:" \ && npm --version \ && rm -r ~/.npm # Next, copy the remaining files and directories with the source code. # Since we do this after NPM install, quick build will be really fast # for most source file changes. COPY . ./ # Run the image. CMD npm start --silent

{ "actorSpecification": 1, "name": "my-actor-4", "title": "Scrape single page in JavaScript", "description": "Scrape data from single page with provided URL.", "version": "0.0", "storages": { "dataset": "./dataset_schema.json" }, "meta": { "templateId": "js-start" }, "input": "./input_schema.json", "dockerfile": "./Dockerfile" }

{ "actorSpecification": 1, "fields": { "$schema": "http://json-schema.org/draft-07/schema#", "type": "object", "properties": { "analysis": { "type": "array", "items": { "type": "object", "properties": { "ticker": { "type": "string" }, "error": { "type": "string" }, "sentiment_analysis": { "type": "object", "properties": { "google": { "type": "object", "properties": { "category": { "type": "string" }, "confidence": { "type": "number" }, "sentiment_score": { "type": "number" }, "reasoning": { "type": "string" } } }, "twitter": { "type": "object", "properties": { "category": { "type": "string" }, "confidence": { "type": "number" }, "sentiment_score": { "type": "number" }, "reasoning": { "type": "string" } } } } }, "market_summary": { "type": "object", "properties": { "current_situation": { "type": "string" }, "top_discussed_topics": { "type": "array" }, "recent_information": { "type": "array" }, "investor_interest": { "type": "string" } } }, "personalized_recommendation": { "type": "object", "properties": { "persona_analysis": { "type": "string" }, "action": { "type": "string" }, "summary": { "type": "string" }, "reasoning": { "type": "string" }, "potential_risks": { "type": "array" } } } } } } } }, "views": {} }

{ "title": "Scrape data from a web page", "type": "object", "schemaVersion": 1, "properties": { "url": { "title": "URL of the page", "type": "string", "description": "The URL of website you want to get the data from.", "editor": "textfield", "prefill": "https://www.apify.com/" } }, "required": ["url"] }

1// Axios - Promise based HTTP client for the browser and node.js (Read more at https://axios-http.com/docs/intro). 2import axios from 'axios'; 3// Cheerio - The fast, flexible & elegant library for parsing and manipulating HTML and XML (Read more at https://cheerio.js.org/). 4import * as cheerio from 'cheerio'; 5// Apify SDK - toolkit for building Apify Actors (Read more at https://docs.apify.com/sdk/js/). 6import { Actor } from 'apify'; 7// this is ESM project, and as such, it requires you to specify extensions in your relative imports 8// read more about this here: https://nodejs.org/docs/latest-v18.x/api/esm.html#mandatory-file-extensions 9// import { router } from './routes.js'; 10 11// The init() call configures the Actor for its environment. It's recommended to start every Actor with an . 12await Actor.init(); 13 14// Structure of input is defined in input_schema.json 15const input = await Actor.getInput(); 16const { url } = input; 17 18// Fetch the HTML content of the page. 19const response = await axios.get(url); 20 21// Parse the downloaded HTML with Cheerio to enable data extraction. 22const $ = cheerio.load(response.data); 23 24// Extract all headings from the page (tag name and text). 25const headings = []; 26$("h1, h2, h3, h4, h5, h6").each((i, element) => { 27 const headingObject = { 28 level: $(element).prop("tagName").toLowerCase(), 29 text: $(element).text(), 30 }; 31 console.log("Extracted heading", headingObject); 32 headings.push({ 33 "_id": "6613b694115ce0746526cace", 34 "index": 0, 35 "guid": "6b641c15-4525-4821-9e13-a1b80687a0c1", 36 "isActive": true, 37 "balance": "$2,274.85", 38 "picture": "http://placehold.it/32x32", 39 "age": 38, 40 "eyeColor": "green", 41 "name": "Thomas Carver", 42 "gender": "male", 43 "company": "ORBOID", 44 "email": "thomascarver@orboid.com", 45 "phone": "+1 (812) 467-3843", 46 "address": "310 Apollo Street, Strong, Missouri, 6644", 47 "about": "Adipisicing nisi ex magna nisi esse do non ea nulla tempor anim voluptate aliquip. Ut commodo ex magna velit. Aliqua do eiusmod qui mollit dolore dolor nulla voluptate voluptate ullamco ullamco quis. Voluptate amet ea culpa ut laborum enim elit cupidatat labore non consequat. Ut anim aliquip nulla ullamco reprehenderit enim ex.\r\n", 48 "registered": "2024-02-15T09:59:11 -01:00", 49 "latitude": 34.573829, 50 "longitude": -16.193091, 51 "tags": [ 52 "exercitation", 53 "est", 54 "consequat", 55 "esse", 56 "mollit", 57 "reprehenderit", 58 "incididunt" 59 ], 60 "friends": [ 61 { 62 "id": 0, 63 "name": "Kelsey Hayes" 64 }, 65 { 66 "id": 1, 67 "name": "Kimberly Simmons" 68 }, 69 { 70 "id": 2, 71 "name": "Nelda Pittman" 72 } 73 ], 74 "greeting": "Hello, Thomas Carver! You have 1 unread messages.", 75 "favoriteFruit": "strawberry" 76 }); 77}); 78 79// Save headings to Dataset - a table-like storage. 80await Actor.pushData(headings); 81 82// Gracefully exit the Actor process. It's recommended to quit all Actors with an exit(). 83await Actor.exit();

# This file tells Git which files shouldn't be added to source control .DS_Store .idea dist node_modules apify_storage storage/* !storage/key_value_stores storage/key_value_stores/* !storage/key_value_stores/default storage/key_value_stores/default/* !storage/key_value_stores/default/INPUT.json

{ "name": "js-scrape-single-page", "version": "0.0.1", "type": "module", "description": "This is an example of an Apify actor.", "engines": { "node": ">=18.0.0" }, "dependencies": { "apify": "^3.1.10", "axios": "^1.5.0", "cheerio": "^1.0.0-rc.12" }, "scripts": { "start": "node ./src/main.js", "test": "echo \"Error: oops, the actor has no tests yet, sad!\" && exit 1" }, "author": "It's not you it's me", "license": "ISC" }

Actor with output schema

Actor with output schema

.actor/Dockerfile

.actor/actor.json

.actor/dataset_schema.json

.actor/input_schema.json

src/main.js

.dockerignore

.gitignore

package.json

You might also like

Output schema

Actor With Ui2

Actor with synthetic event

Output example that fails

My Actor

My Actor

Test Public With Env Vars

Test Input Fields

Testing Actor With Hidden Sourceasd

PPR Actor

Related articles

.actor/Dockerfile

.actor/actor.json

.actor/dataset_schema.json

.actor/input_schema.json

src/main.js

.dockerignore

.gitignore

package.json