K Nearest Neighbor Join Algorithms for Spatial Data Using Compact and Plain Data Structures in Main Memory

Abstract

[Abstract] The K Nearest Neighbors Query (KNNQ) is a fundamental distance-based query with applications in Geographic Information Systems (GIS) and many other domains. The K Nearest Neighbors Join Query (KNNJQ) combines KNNQ with the join operation to find the K nearest neighbors from a dataset P to every object in another dataset Q. Due to the large number of distance computations involved, KNNJQ is an expensive operation that poses a challenge for in-memory systems. In this paper, we propose and evaluate novel in-memory algorithms for KNNQ and KNNJQ using the compact data structure k2-tree. The proposed algorithms exploit hierarchical spatial decomposition together with grouping, pre-processing and progressive pruning techniques. In addition, we develop two new plane-sweep algorithms for KNNQ and KNNJQ, that improve the brute-force algorithms by reducing redundant computations. We conduct an extensive experimental evaluation using real-world spatial datasets to evaluate the performance of our KNNQ and KNNJQ algorithms, comparing them with KD-tree and R*-tree based algorithms. Our results highlight the efficiency of pruning techniques in k2-trees, which reduce distance computations and response time by up to an order of magnitude in KNNJQ, in comparison with the equivalent KD-tree, R*-tree and plane-sweep algorithms.