Smart Data Scraping System for Digital Marketing Agency
Our client is a market-leading digital marketing agency that provides digital advertisers around the world with efficient ways and tools to hit the right audience. Using innovative algorithms, a team of professionals performs in-depth analysis and evaluation of the respective platforms to make their customers get only high traffic websites with relevant target audience.
Our client was looking for a software solution capable of downloading advertisement information from the various classified websites. The classified websites contained advertisements on the vast amount of different products, but the client needed the software to extract information only from the “Cars” category.
Our client also needed the system to provide the function of placing their watermark logo on the images downloaded.
Due to the excessive amount of data placed on the classified websites, it was impossible to process all the relevant data at once, therefore, the software was expected to be able to restart downloading the ads from the same point it finished downloading during the last session.
Db Devs engineers developed a desktop scraping solution, that comprised of three powerful crawlers with intuitive user-friendly interface that could scrape the data from the three large classified platforms. After collecting the relevant data from the websites, the crawlers stored it to the MS SQL database and then output the materials into csv format.
Furthermore, the Db Devs team enabled crawlers to place watermark logo of our client on the pictures downloaded from the websites.
In order to enable automatic uploading of the data scraped from the ads and images onto the client’s WordPress website on the regular basis, Db Devs engineers designed and developed a data connector for Wordpress. Thanks to this connector, our client could upload the data from the application’s MS SQL database directly to their WordPress website database and have the relevant information always at their disposal.
Db Devs engineers developed and delivered a cutting-edge all-round web crawling solution which was created with top-notch technologies and could easily extract valuable data even from the deepest data layers, protected with complex captchas. The implemented web crawling software allowed our client to improve their business processes by making grounded decisions, based on the retrieved data. These improvements helped our client keep leading positions on the market of digital advertising by providing their customers with the best services and exceptional experience.