Implementing big data lake for heterogeneous data sources |
|
Author: | Mehmood, Hassan1; Gilman, Ekaterina1; Cortes, Marta1; |
Organizations: |
1University of Oulu, Finland 2Dell EMC, Ireland 3Draxis Environmental S.A, Greece |
Format: | article |
Version: | accepted version |
Access: | open |
Online Access: | PDF Full Text (PDF, 0.6 MB) |
Persistent link: | http://urn.fi/urn:nbn:fi-fe2019082024798 |
Language: | English |
Published: |
Institute of Electrical and Electronics Engineers,
2019
|
Publish Date: | 2019-08-20 |
Description: |
AbstractModern connected cities are more and more leveraging advances in ICT to improve their services and the quality of life of their inhabitants. The data generated from different sources, such as environmental sensors, social networking platforms, traffic counters, are harnessed to achieve these end goals. However, collecting, integrating, and analyzing all the heterogeneous data sources available from the cities is a challenge. This article suggests a data lake approach built on Big Data technologies, to gather all the data together for further analysis. The platform, described here, enables data collection, storage, integration, and further analysis and visualization of the results. This solution is the first attempt to integrate a diverse set of data sources from four pilot cities as part of the CUTLER project (Coastal urban development through the lenses of resiliency). The design and implementation details, as well as usage scenarios are presented in this paper. see all
|
Series: |
IEEE International Conference on Data Engineering workshop |
ISSN: | 1943-2895 |
ISSN-E: | 2473-3490 |
ISSN-L: | 1943-2895 |
ISBN: | 978-1-7281-0890-2 |
ISBN Print: | 978-1-7281-0891-9 |
Pages: | 37 - 44 |
DOI: | 10.1109/ICDEW.2019.00-37 |
OADOI: | https://oadoi.org/10.1109/ICDEW.2019.00-37 |
Host publication: |
IEEE 35th International Conference on Data Engineering Workshops (ICDEW) |
Conference: |
IEEE International Conference on Data Engineering Workshops |
Type of Publication: |
A4 Article in conference proceedings |
Field of Science: |
113 Computer and information sciences |
Subjects: | |
Funding: |
This research has been financially supported by EU Horizon 2020 project CUTLER: Coastal Urban developmenT through the LEnses of Resiliency, under contract no. 770469 ( http://www.cutler-h2020.eu/ ), ERDF project A71720, ‘Big Data for 5G’, governed by the HILLA program ( www.hilla.center ), and by Academy of Finland 6Genesis Flagship (grant 318927). |
EU Grant Number: |
(770469) CUTLER - Coastal Urban developmenT through the LEnses of Resiliency |
Academy of Finland Grant Number: |
318927 |
Detailed Information: |
318927 (Academy of Finland Funding decision) |
Copyright information: |
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. |