Scrimmo: A Real-Time Web Scraper Monitoring the Belgian Real Estate Market

Web scraping (or Web crawling), a technique for automated data extraction from websites, has emerged as a valuable tool for scientific research and data analysis. This paper presents a comprehensive exploration of Web scraping, its methodologies and challenges. The discussion revolves around a concr...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:2023 IEEE International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) s. 335 - 338
Hlavní autoři: Barzin, Felix, Yernaux, Gonzague, Vanhoof, Wim
Médium: Konferenční příspěvek
Jazyk:angličtina
japonština
Vydáno: IEEE 26.10.2023
Témata:
Data analysis
Data collection
Data extraction
Data gathering
Data mining
Ethics
Intelligent agents
Iterative methods
Real-time systems
Web crawling
Web scraping
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Web scraping (or Web crawling), a technique for automated data extraction from websites, has emerged as a valuable tool for scientific research and data analysis. This paper presents a comprehensive exploration of Web scraping, its methodologies and challenges. The discussion revolves around a concrete application, namely the automatic extraction of data concerning the Belgian real estate market. We introduce a real-time Web scraper called SCRIMMO and tailored to collect data from websites containing real estate classified ads. The tool is developed in a continuous iterative process and based on an innovative cloud architecture. The paper also briefly addresses the ethical aspects of Web scraping. By integrating insights from previous research and ethical guidelines, this study provides researchers with a comprehensive understanding of Web scraping and its potential benefits, while promoting responsible and ethical practices in data collection and analysis.
DOI:10.1109/WI-IAT59888.2023.00054