The application is a desktop scraping application with a user-friendly interface, MS SQL database and a crawler to scrape professionals related data from http://www.houzz.com/.
Available professionals related data is scraped and saved into the database of the application with the possibility to export the scraped into CSV format with only several; clicks of a button.
Additionally, the professionals’ website urls are visited with the aim to scrape further not available at http://www.houzz.com/ contact details with the help of Regular Expressions from there. The big amount of scraped data is saved into the database of the application and further exported into CSV.
For scanning the website TOR network Proxy addresses were used.
The complexity of the project was due to the big number of urls it was necessary to regularly scan with high efficiency.
Tools and Technologies