Context

One of our clients, a large real estate group, contacted us to build a software program that could automatically crawl/parse and fetch data from a significantly large real estate listings website on a daily basis.

The client had tried a few web development firms prior to hiring us, and lacked the in-house knowledge. We’d worked with a number of firms and freelancers previously, but with unsatisfactory results.

This project was relatively challenging because our client had tried a few web development firms prior to hiring us, but they could not achieve the desired results.

Solution

My Web Programmer developed a data mining program that crawled and parsed required data from a real estate listings website. The front-end of the listings website was built using AngularJS, which added to the complexity of the project.

We chose Python for crawler development, and Laravel for the backend application.

The data mining software that we developed collects and stores website data, and automatically sends a daily report via email. Our client can also log in into a secured Laravel backend application to view, and download the collected data.

Results

The custom data crawling software we developed has had a very positive impact for our client’s real estate group. The software has helped our client’s company to improve their sales, and overall operations.