Indices and Refactoring JRDF

Staring back at me today is something that Paul probably saw last year. But it's something that I found pretty cool because initially it looked like a mistake. Almost as cool as Tom starting to get SPARQL going.

Basically, there are three indices in JRDF: (0,1,2), (1,2,0) and (2,0,1). With the 0 equating to the subject, the 1 the predicate and 2 to the object.

Looking at a refactoring between these internal indices and triples (ordered by subject, predicate and object) there are these three mappings:
* (0,1,2) maps to (s,p,o) using (0,1,2),
* (1,2,0) maps to (s,p,o) using (2,0,1), and
* (2,0,1) maps to (s,p,o) using (1,2,0).

Using the second index as an example, it means that the internal representation maps the third element to the subject, the first element to predicate and the second element to the object.

The surprising thing I noticed, beside the other two indices not being tested correctly, was that the other indices ((2,1,0), (1,0,2), (0,2,1) - these are mentioned in Paul's post) don't seem to have this property. They all seem to map (s,p,o) like (0,1,2) - i.e. using themselves. Whereas, (1,2,0) mapped to itself gives (2,0,1) and (2,0,1) mapped to itself gives (1,2,0).

Comparing YARS and Kowari

Cold, Mastering OWL, Scalability, Indexing "There are two differences in their implementation to our own. The first is that they used B-trees rather than the AVL trees that Kowari uses. That is not a significant difference, and I blogged about this several times before. The main advantage of the AVL tree is that they are cheaper to write to the structure than B-trees are, though a little slower to read. They also pointed out that they wanted to use an existing library for the index, and while B-tree libraries are common, we never found a completely working AVL tree library (deletions were always buggy).

The second difference is a set of special statements which count the number of triples in parts of each of the indexes. This is certainly novel, but not an approach I would use. I believe that counting is still very efficient in Kowari (O(log(n)), and the space overhead they incurred would seem prohibitive. More importantly, writing is always the slowest operation, and their system would incur a large writing penalty for using this scheme."

Just Ask for Help

Came across Yet Another RDF Store: Perfect Index Structures for Storing Semantic Web Data With Contexts "We tried to install Kowari, but failed to get a running version. In one of our installations, inserting a 1 MB N-triples file via the Jena interface resulted in 30 minutes processing time before the process threw an exception because of a full disk (there were 200MB disk space available before starting the Kowari server). On another installation, we got a core dump of the JVM when running Kowari. Therefore we concentrated in our efforts on Sesame 1.1RC2 and Redland 0.9.18."

This is the first record of anyone having a problem with Kowari in this respect and without any idea of what version of Kowari or Java or the OS or anything else I went and downloaded their example code. Using Kowari 1.0.5 (they mentioned Sesame 1.1RC2 which was released after 1.0.5) with the given code you get an exception as NTriples is not supported as a parameter. NTriples is a subset of N3. So this code doesn't work. However, if the code is modified so that the last lines become:

  model.read(new FileReader(file), file.toURI().toString(), "N3");
  model.close();
  database.close();

And run the code you get:

...
 INFO [main] (AbstractDatabaseSession.java:699) - Loading 
file:/.../University1_0.nt into rmi://.../server1#camera
 INFO [main] (AbstractDatabaseSession.java:770) - Loaded 7304 
statements
...

If people have any problems running Kowari, before they write it up as a paper maybe they should ask someone how to get it running first. Although, this isn't the first time.