Scraping with nodejs

Jul 1, 2017

In the context of summer, a newsroom coworker was seeking advice and information to go abroad with the intention of making news, and which countries were at risk of malaria, yellow fever, etc. She found a web application from the Spanish government where you could not download the information to make an exhaustive analysis.

Spanish Ministry Of Health doesn't have a way to extract the data with the information web app. We asked them for a csv or any other format to obtain the information. However, they won't facilite any document as they told us that the information is already on the website. Then, I made this scrape.

There is a select on the landing page I use to fill and array with all the links for each country I will scrape later.

To avoid being banned by making multiple requests to the same page until I have the code that would extract the information from each country page I used Google Chrome snippets thanks to which you can run small pieces of code on client side.

Take a look to the code at github.

nodejs