Announcing Andrae

lca: Andrae Muys on RDF "On the first day of linux.conf.au, I ran into Andrae Muys. He hacks Java and RDF for clients who want semantic web hackery done. I have to admit that early Semantic Web hype put me off: it sounded too much like 1970s AI hype. Andrae was interesting, though, and completely free of the wide-eyed uncritical enthusiasm that characterized a lot of my early RDF engagement."

"Andrae runs the Mulgara project, a Java RDF store. His goal is to be able to deal with 1E13 statements (aka tuples, facts, assertions) in three years. It'll do 1E9 right now, next stop is 1E11. He refers to this goal as "3 Ts: three trillion triples". A consortium is forming around Mulgara to make this happen: if it coalesces, Andrae will be the coder to make it happen."

"The next version of Mulgara, 1.3, will ship in February and have this relational mapping in it. A quick Google search shows a lot of RDF-relational mappings going on, but the list of other mappings he had impressed me: Lucene, RSS, mbox, ID3."

"I think it's time I looked again at the world of RDF. They may yet be doing interesting things. I said as much to Andrae and he replied, "I am an engineer. In the early days it was scientists and logicians in RDF. Now the engineers have arrived, and we just want it to work and to scale." Bold claim! If you have a favourite RDF package or practice, let me know in the comments."

Andrae also announced the paper presented at linux.conf.au. Among other things it references David Wood's paper presented in 2004 "Scaling the Kowari Metastore" ("Makepeace" is a good search term).

Things I've Read

* Evangelical Scientists Refute Gravity With New 'Intelligent Falling' Theory related to Boing Boing's $250,000 Intelligent Design challenge (UPDATED: $1 million) and Flying Spaghetti Monster.
* JUnit-Addons and GSBase so at least only two software projects have to reinvent the wheel.
* Non-blocking Data Sharing in Multiprocessor Real-Time Systems "In this paper, we present an efficient non-blocking solution to the general readers/writers inter-task communication problem our solution allows any arbitrary number of readers and writers to perform their respective operations."
* Why I Prefer SOA to REST "The problem is that although it is easy to model resources as services as shown in the example in many cases it is quite difficult to model a service as a resource. For example, a service that validates a credit card number can be modeled as a validateCreditCardNumber(string cardNumber) service. On the other hand it is unintuitive how one would model the service as a resource. For this reason I prefer to think about distributed applications in terms of services as opposed to resources." Now people can just argue about what is intuitive (credit card gateways on the Web existed before SOAP - must have been REST I guess).
* I Need a New Language: Rel? "I want a language for table programming. I think you can write programs in this language that do everything we expect of an application programming language -- building GUIs, reacting to mouse events, listening to sockets -- everything. Don't model your domain as objects. Model it as relations...I can imagine programs that have a relvar (Date's term for a relational variable: essentially a table or a view) for MouseState."
* Package Scoping And Unit Testing "Package scoping particularly shines during unit testing. Some programmers argue that you should only test through the public API. Don't be silly. Limiting your tests to the public API contradicts the spirit of unit testing and subjects you to unnecessary dependency pain. I prefer to isolate and limit the amount of code I test at one time, and test as close to the code as possible." Somewhat related, JSR 277 - Java Module System
* Web as Platform Mash-Ups "There have been a lot of excellent posts and articles this week about APIs, the Web as Platform, web sites as software companies, and so forth..."
* Ruby, Python, "Power" "There are different opinions on the relative power of Ruby and Python. I'm not much more authoritative than other resources (though I'm not less authoritative either; most comparisons between the two languages are flawed). Ultimately I don't believe there are many (any?) places where one language is more "powerful" than the other (and not just in the "they are both Turing complete" sense)"

A World Without Locks

Wikipedia defines lock-free and wait-free algorithms as allowing "...multiple threads to read and write shared data concurrently without corrupting it. "Lock-free" refers to the fact that a thread cannot lock up: every step it takes brings progress to the system."

LOCK-FREE LINKED LISTS AND SKIP LISTS "Developing a correct and efficient memory management scheme is important to make a data structure practical. Developing such a scheme for a lock-free data structure is often quite a challenging task. The difficulty lies in determining how and when memory that was once occupied by parts of the data structure (e.g. nodes of a linked list), can be freed and reused, so that the processes that might still be accessing those parts are able to complete their operations correctly...We presented new algorithms implementing a lock-free linked list and a lock-free skip list. We proved their correctness and lock-freedom."

Lock-Free Reference Counting The goal of this work, therefore, is to allow programmers to exploit the advantages of GC in designing their lock-free data structure implementations, while avoiding its drawbacks. To this end, we provide a methodology that allows programmers to first solve the easier problem of designing a GC-dependent implementation, and to then apply our methodology in order to achieve a GC-independent one.

An older article: Lock-free Parallel Garbage Collection by Mark&Sweep.

Related to Lock Free Programming.

JRDF for Learning

In the past I used another project to practically use trendy new things like patterns, XML, Swing, etc. Similarly, I'm going to use JRDF for the same purpose. Kowari is a bit too big for things like going over to Java 1.5, IoC, mocking (real unit tests), lock free alogirthms (including B-Trees) and a few other things that I want to try. I'm not sure it's possible to have a system that doesn't have transactions but it would be interesting to find out. So basically this is just to let people know to expect some changes in JRDF.

Practically, it might mean an RDF/XML based pull parser, persistent JRDF, and more interesting APIs. I'm convinced that developing web services is too expensive and may implode under its own weight - so maybe something based on netKernel or a REST based framework would be a good idea. At the moment I'm just using it to see how much I can get out of IntelliJ.

Lock Free Programming

Java theory and practice: Going atomic "Until JDK 5.0, it was not possible to write wait-free, lock-free algorithms in the Java language without using native code. With the addition of the atomic variables classes in the java.util.concurrent.atomic package, that has changed. The atomic variable classes all expose a compare-and-set primitive (similar to compare-and-swap), which is implemented using the fastest native construct available on the platform (compare-and-swap, load linked/store conditional, or, in the worst case, spin locks). Nine flavors of atomic variables are provided in the java.util.concurrent.atomic package (AtomicInteger; AtomicLong; AtomicReference; AtomicBoolean; array forms of atomic integer; long; reference; and atomic marked reference and stamped reference classes, which atomically update a pair of values)."

See also, More flexible, scalable locking in JDK 5.0, Atomic Javadoc and an interesting article The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software.

And Java 1.5 Update 1 is out too.