Scalable processing and autocovariance computation of big functional data
Citations Over Time
Abstract
Summary This paper presents 2 main contributions. The first is a compact representation of huge sets of functional data or trajectories of continuous‐time stochastic processes, which allows keeping the data always compressed even during the processing in main memory. It is oriented to facilitate the efficient computation of the sample autocovariance function without a previous decompression of the data set, by using only partial local decoding. The second contribution is a new memory‐efficient algorithm to compute the sample autocovariance function. The combination of the compact representation and the new memory‐efficient algorithm obtained in our experiments the following benefits. The compressed data occupy in the disk 75% of the space needed by the original data. The computation of the autocovariance function used up to 13 times less main memory, and run 65% faster than the classical method implemented, for example, in the R package.
Related Papers
- → Derivation of the theoretical autocovariance and autocorrelation function of autoregessive moving average processes(1988)6 cited
- → Finite collection time effects in autocovariance function measurements(2003)2 cited
- → Investigations of errors in sample autocovariance functions and their corresponding power spectra*(1978)9 cited
- Neural networks method for computation of autocovariance distance(2003)
- Robust Estimation of Power Spectra via the Autocovariance Function(2015)