October 1st 2021 new story
Using Node v12, we will build a simple scraper and crawler from scratch using Javascript.
We’ll use scrapeme.live as an example, a fake website prepared for scraping. We use Axios to get the HTML. Then we will pass the HTML to cheerio and query it as we would in a browser environment. We’ll query for the two things we want right now: paginator links and products.
For cases when we want to run JS, Playwright will do. Once everything is working fine, we will scale it by launching crawls async.
Ander
Web developer who has been working for startups for +10 years. Engineer turned entrepreneur.
Tags
Join Hacker Noon
Create your free account to unlock your custom reading experience.
Web Scraping with Javascript and Node.js
Source: Pinay Tube PH
0 Comments