Evaluating top-k queries over incomplete data streams

Citations Over TimeTop 10% of 2009 papers

Abstract

We study the problem of continuous monitoring of top-k queries over multiple non-synchronized streams. Assuming a sliding window model, this general problem has been a well addressed research topic in recent years. Most approaches, however, assume synchronized streams where all attributes of an object are known simultaneously to the query processing engine. In many streaming scenarios though, different attributes of an item are reported in separate non-synchronized streams which do not allow for exact score calculations. We present how the traditional notion of object dominance changes in this case such that the k dominance set still includes all and only those objects which have a chance of being among the top-k results in their life time. Based on this, we propose an exact algorithm which builds on generating multiple instances of the same object in a way that enables efficient object pruning. We show that even with object pruning the necessary storage for exact evaluation of top-k queries is linear in the size of the sliding window. As data should reside in main memory to provide fast answers in an online fashion and cope with high stream rates, storing all this data may not be possible with limited resources. We present an approximate algorithm which leverages correlation statistics of pairs of streams to evict more objects while maintaining accuracy. We evaluate the efficiency of our proposed algorithms with extensive experiments.

Related Papers

• → A Sliding Window Method for Finding Recently Frequent Itemsets over Online Data Streams(2004)138 cited
• → An Approximate Approach for Mining Recently Frequent Itemsets from Data Streams(2006)17 cited
• Mining the Frequent Patterns in an Arbitrary Sliding Window over Online Data Streams(2008)
• An Approximate Approach for Mining Recently Frequent Itemsets from Data Streams*(2015)
• → Trends in Data Stream Mining(2023)