Screen Scraping with Node.js
You may have used NodeJS as a web server, but did you know that you can also use it for web scraping? In this tutorial, we'll review how to scrape static web pages - and those pesky ones with dynamic content - with the help of NodeJS and a few helpful NPM modules. Web scraping has always had a negative connotation in the world of web development - and for good reason. In modern development, APIs are present for most popular services and they should be used to retrieve data rather than scraping. The inherent problem with scraping is that it relies on the visual structure of the page being scraped. Despite these flaws, it's important to learn a bit about web scraping and some of the tools available to help with this task. Note: If you can't get the information you require through an API or a feed, it's a good sign that the owner does not want that information to be accessible. Scrapers can be written in any language, really. Let's start with the simple use-case: static web pages.