University of Oulu

H. Mehmood et al., "Implementing Big Data Lake for Heterogeneous Data Sources," 2019 IEEE 35th International Conference on Data Engineering Workshops (ICDEW), Macao, Macao, 2019, pp. 37-44. doi: 10.1109/ICDEW.2019.00-37

Implementing big data lake for heterogeneous data sources

Saved in:
Author: Mehmood, Hassan1; Gilman, Ekaterina1; Cortes, Marta1;
Organizations: 1University of Oulu, Finland
2Dell EMC, Ireland
3Draxis Environmental S.A, Greece
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 0.6 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2019
Publish Date: 2019-08-20


Modern connected cities are more and more leveraging advances in ICT to improve their services and the quality of life of their inhabitants. The data generated from different sources, such as environmental sensors, social networking platforms, traffic counters, are harnessed to achieve these end goals. However, collecting, integrating, and analyzing all the heterogeneous data sources available from the cities is a challenge. This article suggests a data lake approach built on Big Data technologies, to gather all the data together for further analysis. The platform, described here, enables data collection, storage, integration, and further analysis and visualization of the results. This solution is the first attempt to integrate a diverse set of data sources from four pilot cities as part of the CUTLER project (Coastal urban development through the lenses of resiliency). The design and implementation details, as well as usage scenarios are presented in this paper.

see all

Series: IEEE International Conference on Data Engineering workshop
ISSN: 1943-2895
ISSN-E: 2473-3490
ISSN-L: 1943-2895
ISBN: 978-1-7281-0890-2
ISBN Print: 978-1-7281-0891-9
Pages: 37 - 44
DOI: 10.1109/ICDEW.2019.00-37
Host publication: IEEE 35th International Conference on Data Engineering Workshops (ICDEW)
Conference: IEEE International Conference on Data Engineering Workshops
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
Funding: This research has been financially supported by EU Horizon 2020 project CUTLER: Coastal Urban developmenT through the LEnses of Resiliency, under contract no. 770469 ( ), ERDF project A71720, ‘Big Data for 5G’, governed by the HILLA program ( ), and by Academy of Finland 6Genesis Flagship (grant 318927).
EU Grant Number: (770469) CUTLER - Coastal Urban developmenT through the LEnses of Resiliency
Academy of Finland Grant Number: 318927
Detailed Information: 318927 (Academy of Finland Funding decision)
Copyright information: © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.