Leveraging LLMs for Semantic Correlation Enhancement in Spatial-temporal Imputation

Abstract

Spatial-temporal imputation remains a challenging problem in transportation, environment and healthcare, where the missing value is filled based on spatial, temporal, and cross correlations. Previous research mainly focused on feature-level correlation integration and comprehension with the hand-crafted enhancement strategy. Meanwhile, the recently prevalent large language models (LLMs) provide token-level understanding for language linguistics, and whether they could be applied for spatial-temporal correlation enhancement is under exploration. To this end, we proposed an LLM-native framework STOMA to fully utilize the intrinsic relevance. We designed semantic enhancing methods by converting the complex correlations, e.g. spatial correlation in network, temporal correlation with periodicity and cross correlation from human behavior, into the embedded tokens. Specifically, we reform dynamic time warping as an asymmetric correlation constructor for complex dynamics. We adapt the proposed backbone along with the spatial-temporal fine-tuning technique, and the empirical results demonstrate the effectiveness of our methods over recent LLM-inspired methods evaluating on real-world datasets.