BDWatchdog: real-time monitoring and profiling of Big Data applications and frameworks
Ver/Abrir
Use este enlace para citar
http://hdl.handle.net/2183/21722
Excepto si se señala otra cosa, la licencia del ítem se describe como Atribución-NoComercial-SinDerivadas 3.0 España
Colecciones
- GI-GAC - Artigos [192]
Metadatos
Mostrar el registro completo del ítemTítulo
BDWatchdog: real-time monitoring and profiling of Big Data applications and frameworksFecha
2018-10Cita bibliográfica
Jonatan Enes, Roberto R. Expósito, Juan Touriño, BDWatchdog: Real-time monitoring and profiling of Big Data applications and frameworks, Future Generation Computer Systems, Volume 87, 2018, Pages 420-437, ISSN 0167-739X, https://doi.org/10.1016/j.future.2017.12.068.
Resumen
[Abstract] Current Big Data applications are characterized by a heavy use of system resources (e.g., CPU, disk) generally distributed across a cluster. To effectively improve their performance there is a critical need for an accurate analysis of both Big Data workloads and frameworks. This means to fully understand how the system resources are being used in order to identify potential bottlenecks, from resource to code bottlenecks. This paper presents BDWatchdog, a novel framework that allows real-time and scalable analysis of Big Data applications by combining time series for resource monitorization and flame graphs for code profiling, focusing on the processes that make up the workload rather than the underlying instances on which they are executed. This shift from the traditional system-based monitorization to a process-based analysis is interesting for new paradigms such as software containers or serverless computing, where the focus is put on applications and not on instances. BDWatchdog has been evaluated on a Big Data cloud-based service deployed at the CESGA supercomputing center. The experimental results show that a process-based analysis allows for a more effective visualization and overall improves the understanding of Big Data workloads. BDWatchdog is publicly available at http://bdwatchdog.dec.udc.es.
Palabras clave
Big Data
Monitoring
Profiling
Time series
Flame graphs
Process-based analysis
Monitoring
Profiling
Time series
Flame graphs
Process-based analysis
Descripción
This is a post-peer-review, pre-copyedit version of an article published in Future Generation Computer Systems. The final authenticated version is available online at: https://doi.org/10.1016/j.future.2017.12.068
Versión del editor
Derechos
Atribución-NoComercial-SinDerivadas 3.0 España
ISSN
0167-739X
1872-7115
1872-7115