Big Data-Oriented PaaS Architecture with Disk-as-a-Resource Capability and Container-Based Virtualization
Use this link to cite
http://hdl.handle.net/2183/21763Collections
- GI-GAC - Artigos [192]
Metadata
Show full item recordTitle
Big Data-Oriented PaaS Architecture with Disk-as-a-Resource Capability and Container-Based VirtualizationDate
2018-12Citation
Enes, J., Cacheiro, J.L., Expósito, R.R. et al. J Grid Computing (2018) 16: 587. https://doi.org/10.1007/s10723-018-9460-4
Abstract
[Abstract] With the increasing adoption of Big Data technologies as basic tools for the ongoing Digital Transformation, there is a high demand for data-intensive applications. In order to efficiently execute such applications, it is vital that cloud providers change the way hardware infrastructure resources are managed to improve their performance. However, the increasing use of virtualization technologies to achieve an efficient usage of infrastructure resources continuously widens the gap between applications and the underlying hardware, thus decreasing resource efficiency for the end user. Moreover, this scenario is especially troublesome for Big Data applications, as storage resources are one of the most heavily virtualized, thus imposing a significant overhead for large-scale data processing. This paper proposes a novel PaaS architecture specifically oriented for Big Data where the scheduler offers disks as resources alongside the more common CPU and memory resources, looking forward to provide a better storage solution for the user. Furthermore, virtualization overheads are reduced to the bare minimum by replacing heavy hypervisor-based technologies with operating-system-level virtualization based on light software containers. This architecture has been deployed on a Big Data infrastructure at the CESGA supercomputing center, used as a testbed to compare its performance with OpenStack, a popular private cloud platform. Results have shown significant performance improvements, reducing the execution time of representative Big Data workloads by up to 4.5×.
Keywords
Big Data
Platform as a Service (PaaS)
Cloud computing
Disk-as-a-resource scheduling
Operating-system-level virtualization
Platform as a Service (PaaS)
Cloud computing
Disk-as-a-resource scheduling
Operating-system-level virtualization
Description
This is a post-peer-review, pre-copyedit version of an article published in Journal of Grid Computing. The final authenticated version is available online at: https://doi.org/10.1007/s10723-018-9460-4
Editor version
ISSN
1570-7873
1572-9184
1572-9184