Addressing the big-earth-data variety challenge with the hierarchical triangular mesh
Citations Over TimeTop 18% of 2016 papers
Abstract
We have implemented an updated Hierarchical Triangular Mesh (HTM) as the basis for a unified data model and an indexing scheme for geoscience data to address the variety challenge of Big Earth Data. In the absence of variety, the volume challenge of Big Data is relatively easily addressable with parallel processing. The more important challenge in achieving optimal value with a Big Data solution for Earth Science (ES) data analysis, however, is being able to achieve good scalability with variety. With HTM unifying at least the three popular data models, i.e. Grid, Swath, and Point, used by current ES data products, data preparation time for integrative analysis of diverse datasets can be drastically reduced and better variety scaling can be achieved. HTM is also an indexing scheme, and when applied to all ES datasets, data placement alignment (or co-location) on the shared nothing architecture, which most Big Data systems are based on, is guaranteed and better performance is ensured. With HTM most geospatial set operations become integer interval operations with further performance advantages.
Related Papers
- → Trends and Future Perspective Challenges in Big Data(2021)246 cited
- → Statistical Perspectives on “Big Data”(2015)53 cited
- Recent Advances in Big Data: Features, Classification, Analytics, Research Challenges, and Future Trends(2020)
- → Big data and regional science: Opportunities, challenges, and directions for future research(2018)9 cited