You Can contact us by filling the form below:

Thank you, your message was sent successfully!

Please, wait our respond.

Sorry, the error has occured.


You Can contact us by filling the form below:

Thank you, your message was sent successfully!

Please, wait our respond.

Sorry, the error has occured.


Jobs

ABOUT


The application is a desktop scraping application with a user-friendly interface, MS SQL database and a Jobs crawler for scraping Jobs related data from https://www.glassdoor.com/. The data is scraped into the database of the application and output into .CSV format.

 

The project was quite complex, as at https://www.glassdoor.com/ browser identification is used, and in case of a big number of queries from the same user, the server replies with a Captcha. The problem was solved with the help of IP rotating and client identification in case of having to deal with Captcha. A special service was created to realize this. The service scans Free Proxy available on the web with the frequency predefined, checks the possibility of them to be used at each target website and saves these Proxy addresses into the database for further use.

 

 

PROPERTIES


Solution type: Desktop App, WPF
Items in Database: ~ 3.5 mln
Database size: 1Gb
Built on: dynamic .dll library
Crawler business logic: described in crawler core

FEATURES


Data search
Data crawling
Anti-captcha
Data collection
Multithreading
xml parsing
html parsing
csv export

 

EFFORT


80 man-hours

 

Services


Web Crawling
Data Collection
Data export
Jobs

PROJECT SCREENS


Tools and Technologies


Contact us
Feel free to contact us.
Our work hours:
Monday-Friday 8am - 8pm

Thank you, your message was sent successfully!

Please, wait our respond.


Sorry, the error has occured.


Our location:

Megapolis Office Center, Office 607,
Moskovskiy av. 179-B Kharkiv, 61068, Ukraine