Scalable processing and autocovariance computation of big functional data
Use this link to cite
http://hdl.handle.net/2183/34611Collections
- GI-LBD - Artigos [51]
Metadata
Show full item recordTitle
Scalable processing and autocovariance computation of big functional dataDate
2018Citation
Brisaboa NR, Cao R, Paramá JR, Silva-Coira F. Scalable processing and autocovariancecomputation of big functional data. Softw Pract Exper. 2018; 48: 123–140. https://doi.org/10.1002/spe.2524
Is version of
https://doi.org/10.1002/spe.2524
Abstract
[Abstract]: This paper presents 2 main contributions. The first is a compact representation of huge sets of functional data or trajectories of continuous-time stochastic processes, which allows keeping the data always compressed even during the processing in main memory. It is oriented to facilitate the efficient computation of the sample autocovariance function without a previous decompression of the data set, by using only partial local decoding. The second contribution is a new memory-efficient algorithm to compute the sample autocovariance function. The combination of the compact representation and the new memory-efficient algorithm obtained in our experiments the following benefits. The compressed data occupy in the disk 75% of the space needed by the original data. The computation of the autocovariance function used up to 13 times less main memory, and run 65% faster than the classical method implemented, for example, in the R package.
Keywords
Big data
compact representation
autocovariance function
efficient computation
functional data
R package
compact representation
autocovariance function
efficient computation
functional data
R package
Description
This is the peer reviewed version of the following article: Brisaboa NR, Cao R, Paramá JR, Silva-Coira F. Scalable processing and autocovariance computation of big functional data. Softw Pract Exper. 2018; 48: 123–140 which has been published in final form at https://doi.org/10.1002/spe.2524 . This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions. This article may not be enhanced, enriched or otherwise transformed into a derivative work, without express permission from Wiley or by statutory rights under applicable legislation. Copyright notices must not be removed, obscured or modified. The article must be linked to Wiley’s version of record on Wiley Online Library and any embedding, framing or otherwise making available the article or pages thereof by third parties from platforms, services and websites other than Wiley Online Library must be prohibited.
Editor version
Rights
Copyright © 2017 John Wiley & Sons, Ltd