Enabling Hardware Affinity in JVM-Based Applications: A Case Study for Big Data

Use this link to cite
http://hdl.handle.net/2183/35754Collections
- Investigación (FIC) [1618]
Metadata
Show full item recordTitle
Enabling Hardware Affinity in JVM-Based Applications: A Case Study for Big DataDate
2020Abstract
[Abstract]: Java has been the backbone of Big Data processing for more than a decade due to its interesting features such as object orientation, cross-platform portability and good programming productivity. In fact, most popular Big Data frameworks such as Hadoop and Spark are implemented in Java or using other languages designed to run on the Java Virtual Machine (JVM) such as Scala. However, modern computing hardware is increasingly complex, featuring multiple processing cores aggregated into one or more CPUs that are usually organized as a Non-Uniform Memory Access (NUMA) architecture. The platform-independent features of the JVM come at the cost of hardware abstraction, which makes it more difficult for Big Data developers to take advantage of hardware-aware optimizations based on managing CPU or NUMA affinities. In this paper we introduce jhwloc, a Java library for easily managing such affinities in JVM-based applications and gathering information about the underlying hardware topology. To demonstrate the functionality and benefits of our proposal, we have extended Flame-MR, our Java-based MapReduce framework, to provide support for setting CPU affinities through jhwloc. The experimental evaluation using representative Big Data workloads has shown that performance can be improved by up to 17% when efficiently exploiting the hardware. jhwloc is publicly available to download at https://github.com/rreye/jhwloc.
Keywords
Big Data
Java Virtual Machine (JVM)
Hardware affinity
MapReduce
Performance evaluation
Java Virtual Machine (JVM)
Hardware affinity
MapReduce
Performance evaluation
Description
This version of the article has been accepted for publication, after peer review and is subject to Springer Nature’s AM terms of use, but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: https://doi.org/10.1007/978-3-030-50371-0_3.
Editor version
Rights
Todos os dereitos reservados. All rights reserved.
ISSN
0302-9743
1611-3349
1611-3349
ISBN
978-3-030-50370-3 978-3-030-50371-0