BDWatchdog: real-time monitoring and profiling of Big Data applications and frameworks
Ver/ abrir
Use este enlace para citar
http://hdl.handle.net/2183/21722
A non ser que se indique outra cousa, a licenza do ítem descríbese como Atribución-NoComercial-SinDerivadas 3.0 España
Coleccións
- GI-GAC - Artigos [192]
Metadatos
Mostrar o rexistro completo do ítemTítulo
BDWatchdog: real-time monitoring and profiling of Big Data applications and frameworksData
2018-10Cita bibliográfica
Jonatan Enes, Roberto R. Expósito, Juan Touriño, BDWatchdog: Real-time monitoring and profiling of Big Data applications and frameworks, Future Generation Computer Systems, Volume 87, 2018, Pages 420-437, ISSN 0167-739X, https://doi.org/10.1016/j.future.2017.12.068.
Resumo
[Abstract] Current Big Data applications are characterized by a heavy use of system resources (e.g., CPU, disk) generally distributed across a cluster. To effectively improve their performance there is a critical need for an accurate analysis of both Big Data workloads and frameworks. This means to fully understand how the system resources are being used in order to identify potential bottlenecks, from resource to code bottlenecks. This paper presents BDWatchdog, a novel framework that allows real-time and scalable analysis of Big Data applications by combining time series for resource monitorization and flame graphs for code profiling, focusing on the processes that make up the workload rather than the underlying instances on which they are executed. This shift from the traditional system-based monitorization to a process-based analysis is interesting for new paradigms such as software containers or serverless computing, where the focus is put on applications and not on instances. BDWatchdog has been evaluated on a Big Data cloud-based service deployed at the CESGA supercomputing center. The experimental results show that a process-based analysis allows for a more effective visualization and overall improves the understanding of Big Data workloads. BDWatchdog is publicly available at http://bdwatchdog.dec.udc.es.
Palabras chave
Big Data
Monitoring
Profiling
Time series
Flame graphs
Process-based analysis
Monitoring
Profiling
Time series
Flame graphs
Process-based analysis
Descrición
This is a post-peer-review, pre-copyedit version of an article published in Future Generation Computer Systems. The final authenticated version is available online at: https://doi.org/10.1016/j.future.2017.12.068
Versión do editor
Dereitos
Atribución-NoComercial-SinDerivadas 3.0 España
ISSN
0167-739X
1872-7115
1872-7115