Technical reports

A unified graph query layer for multiple databases

Eiko Yoneki, Amitabha Roy

August 2012, 22 pages

DOI: 10.48456/tr-820

Abstract

There is increasing demand to store and query data with an inherent graph structure. Examples of such data include those from online social networks, the semantic web and from navigational queries on spatial data such as maps. Unfortunately, traditional relational databases have fallen short where such graph structured data is concerned. This has led to the development of specialised graph databases such as Neo4j. However, traditional databases continue to have a wide usage base and have desirable properties such as the capacity to support a high volume of transactions while offering ACID semantics. In this paper we argue that it is in fact possible to unify different database paradigms together in the case of graph structured data through the use of a common query language and data loader that we have named Crackle (a wordplay on Gra[ph]QL). Crackle provides an expressive and powerful query library in Clojure (a functional LISP dialect for JVMs). It also provides a data loader that is capable of interfacing transparently with various data sources such as PostgreSQL databases and the Redis key-value store. Crackle shields programmers from the backend database by allowing them to write queries in Clojure. Additionally, its graph-focused prefetchers are capable of closing the hitherto large gap between a PostgreSQL database and a specialised graph database such as Neo4j from as much 326x (with a SQL query) to as low as 6x (when using Crackle). We also include a detailed performance analysis that identifies ways to further reduce this gap with Crackle. This brings into question the performance argument for specialised graph databases such as Neo4j by providing comparable performance on supposedly legacy data sources.

Full text

PDF (0.3 MB)

BibTeX record

@TechReport{UCAM-CL-TR-820,
  author =	 {Yoneki, Eiko and Roy, Amitabha},
  title = 	 {{A unified graph query layer for multiple databases}},
  year = 	 2012,
  month = 	 aug,
  url = 	 {https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-820.pdf},
  institution =  {University of Cambridge, Computer Laboratory},
  doi = 	 {10.48456/tr-820},
  number = 	 {UCAM-CL-TR-820}
}