JRDF 0.4.1.1 Released

JRDF 0.4.1.1. A bug fix release for an error in the wiring. It was wiring up the join code rather than the union code when it was suppose to do union operations. Both Derby and Hadoop were removed for this release (which reduces the GUI jar by 2 MB).

Why Fahrenheit 451 is not like 1984

Ray Bradbury: Fahrenheit 451 Misinterpreted

Now, Bradbury has decided to make news about the writing of his iconographic work and what he really meant. Fahrenheit 451 is not, he says firmly, a story about government censorship. Nor was it a response to Senator Joseph McCarthy, whose investigations had already instilled fear and stifled the creativity of thousands.

This, despite the fact that reviews, critiques and essays over the decades say that is precisely what it is all about. Even Bradbury’s authorized biographer, Sam Weller, in The Bradbury Chronicles, refers to Fahrenheit 451 as a book about censorship.

Points to Ray Bradbury at Home with movies about many topics including stories and politics.

4:34

I was struck by an interview with Ayaan Hirsi Ali. In it she says a moderate version of Islam cannot exist without disregarding parts of the Koran (and without thinking, questioning and debating). I'd always assumed this was true and thought it wouldn't bother me but I was surprised when it did. About half way through the interview she was prompted to give support for her claim that Islam does promote violence. She says there is a criteria for determining whether something is true and she quoted Sura (4:34). Without dancing around the subject it says when faced with a disobedient wife: shout at her, ignore her, and finally beat her. And as far as I can tell this is not up for dispute. It's a bit hard to describe and I'm sure I'd heard it before but for some reason it finally got to me.

So like, Timothy 2, 12 this is a religious statement, without allegory, that is clearly in conflict with modern laws and morals (assuming that morals can exist outside of religion).

I guess I'd always assumed people ignore bits that don't make sense. But as someone reading Timothy said, "What do you do when you are confronted with a finding in scripture that either goes against what you've always believed or at least contradicts what you would like to believe? There are really only two choices. Understand it, accept it and conform to it or reject it and go on doing whatever you want."

The Road After

D Conference Videos of Bill and Steve. All are from brightcove. The missing intro, complete with MacIntosh dating game is on YouTube (there's also a full transcript).

Oh the Waste

Running the Numbers

This new series looks at contemporary American culture through the austere lens of statistics. Each image portrays a specific quantity of something: fifteen million sheets of office paper (five minutes of paper use); 106,000 aluminum cans (thirty seconds of can consumption) and so on. My hope is that images representing these quantities might have a different effect than the raw numbers alone, such as we find daily in articles and books. Statistics can feel abstract and anesthetizing, making it difficult to connect with and make meaning of 3.6 million SUV sales in one year, for example, or 2.3 million Americans in prison, or 426,000 cell phones retired every day. This project visually examines these vast and bizarre measures of our society, in large intricately detailed prints assembled from thousands of smaller photographs.

It also includes, ".5 feet wide by 10.5 feet tall in three horizontal panels, Depicts 125,000 one-hundred dollar bills ($12.5 million), the amount our government spends every hour on the war in Iraq."

Via, Statistical visualization.

MapReduce SPARQL

Compositional Evaluation of W3C SPARQL Algebra via Reduce/Map "Tested against older DAWG testsuite. Implemented using functional programming idioms: fold (reduce) / unfold (map)

Does that suggest parallelizable execution?"

Thanks to SPARQL's adoption of compositional semantics (algebra) it's possible. From what I can remember, while there is an implicit left to right evaluation of OPTIONAL, which does limit the top level execution, the partial results are parallelizable. Being able to execute order independent OPTIONAL while retaining the correct results for OPTIONAL is still an open question (although feasible I think).

Available from SVN here. More musings, Musings of a Semantic / Rich Web Architect: What's Next?.

One More Pragmatic Language

Pragmatic Haskell

You see, I’ve been talking to the good folks over at Pragmatic Programmers about the possibility of doing a Haskell book. All of my writing effort has been going into that book, and as I didn’t know what the consequences would be of posting potential book content to the net, I elected to keep my mouth shut.

Well, they have agreed.

Via.

Where's the Evil?

I like bats much better than bureaucrats. I live in the Managerial Age, in a world of "Admin." The greatest evil is not now done in those sordid "dens of crime" that Dickens loved to paint. It is not done even in concentration camps and labour camps. In those we see its final result. But it is conceived and ordered (moved, seconded, carried, and minuted) in clean, carpeted, warmed, and well-lighted offices, by quiet men with white collars and cut fingernails and smooth-shaven cheeks who do not need to raise their voice. Hence, naturally enough, my symbol for Hell is something like the bureaucracy of a police state or the offices of a thoroughly nasty business concern.

From, Bullying of Academics in Higher Education. An interesting review, Notes on The Screwtape Letters.

The Health Benefits of the Semantic Web

A page about the HCLS Demo given in Banff at WWW2007 Some interesting demos. Using OpenLink's Virtuoso to store and query about 350 million statements (and many more required). Part of the demo used Armed Bear Common Lisp, which is Lisp for the JVM (compiles Lisp to bytecode).

Defeasible Logic Plus Time

Temporal extensions to Defeasible Logic. Non-monotonic reasoning is about adding more information over time to reach different conclusions. Rather than adding information adding temporal extensions actually removes when this information applies.

Even the Banner Ads are More Interesting

I was reading, Sun Tells Java Plans, of which I got to the second paragraph before noticing an Apple ad (in Flash with sound). Listened to the commercial, closed the tab, I only barely care what the article was about. So who does marketing better (actually yesterday there was one with the PC guy banging his head against the banner which was better)? Of course, I could just be responding in a Pavlovian (hmm dessert) way to the background jingle.

Romulus and Remus - C# and Java

Ted Neward talks about C# and Java. Also mentions the possible .NET backlash, Scala and LINQ. There's also a very impressive demo of LINQ (about 1/4 of the way through the code demo starts) - the struggle from imperative to declarative.

An Efficient Link Store

• Another web of data store that produces a subset of RDF/XML, Astoria, is from an unlikely source, Microsoft. Instead of a proprietary Semantic Web, Danny sees it as going to town with URIs and REST.


• Silverlight was the other surprising Microsoft development, nothing beats running code - except maybe browser-based dynamic code 2000 times faster. Applets are cool again.


• Some ideas for static triple indexing "Most mature triplestores also index a 4th query element ‘graph’ or ‘context’. I intend to support this query type without expanding the index by using a trick: In my triples format the fact that the subjects are auto-generated and local to the graph means I can choose them to be sequential and effectively re-use them as graph indexes..."


• Plugged In/Invisible Worlds/Tucana/Northrop/TKS/TMex/Kowari/Mulgara podcast (links to the Talis page).


• PAGE a distributed triple store using DHT and YARS (the original). It does seem to miss the DELIS work on P2P RDF which scaled up to 64 nodes.


• Haskell and the Faith of Programming Languages Phillip Wadler gives a rather brilliant talk on programming languages. Covers Haskell, Java generics, combining different typed languages (weak, strong, very strong) as well as monads and Links.

YARS Revenge

With little fanfare the folks at DERI have announced YARS2. I know of at least 4 next generation RDF stores (you know who you are) with a few others on the drawing board. Storing data is cool again.

To save disk space for the on-disk indices, we compress the individual blocks using Huffman coding. Depending on the data values and the sorting order of the index, we achieve a compression rate of ≈ 90%. Although compression has a marginal impact on performance, we deem that the benefits of saved disk space for large index files outweighs the slight performance dip.

Figure 4 shows the correspondence between block size and lookup time, and also shows the impact of Huffman coding on the lookup performance; block sizes are measured pre-compression. The average lookup time for a data file with 100k entries (random lookups for all subjects in the index) using a 64k block size is approximately 1.1 ms for the uncompressed and 1.4 ms for the compressed data file. For 90k random lookups over a 7 GB data file with 420 million synthetically generated triples (more on that dataset in Section 7), we achieve an average seek time of 23.5 ms.

I still wonder how the DERI guys can make the claim about it being their indexing scheme especially when Kowari was open sourced before YARS or the original paper came out. Maybe it's who publishes first? See Paul's previous discussion about it in 2005 (under the title "Indexing"). I mind that this hasn't been properly attributed as I'd like Paul and any others to get the attribution they deserve. On the other hand, I'm glad that people are taking this idea and running with it.

It's good to see that text searching on literals now seems like a standard feature too. They used a sparse index to create all 6 indices. They also hint out how reasoning is going to be performed by linking to, "Unifying Reasoning and Search to Web Scale", which suggests a tradeoff over time and trust.

You Are Here, Now

• Mammal rise 'not linked' to dinos Perhaps the best part is the beautifully rendered result of evolution for the last 166 million years. Good global warming? "However, the supertree shows that the placental mammals had already split into these sub-groups by 93 million years ago, long before the space impact and at a time when dinosaurs still ruled the planet."


• Guice Talk by Google Listing all the good ways DI makes your code better - and yes testing is the main one, more OO is another, better than service locater (J2EE) etc. Not much new here, except for no XML, for people who have been doing this for a while.


• Another Google talk by the Cyc people referenced in Priming the Pump and Threshold Conditions . With many examples that will be familiar for Semantic Web proponents.


• Closures for Java JSR Scala gets a mentions as does the BGGA proposal.


• Three reasons that REST is not RPC "Being able to do state transition processing at disparate locations is hugely powerful...A single process can span machines offering differing levels of scalability, reliability and security."


• Six of One, a Half Dozen of the Other "Closures vs. objects should really be a koan". I think this was off a reddit post at some stage. Leads to a purely functional OO system in Scheme.

Just Fearful Enough

Profile of Jon Stewart

It's sort of this odd and I've always had this problem with the rationality of it. That the President says, "We are in the fight for a way of life. This is the greatest battle of our generation, and of the generations to come. "And, so what I'm going to do is you know, Iraq has to be won, or our way of life ends, and our children and our children's children all suffer. So, what I'm gonna do is send 10,000 more troops to Baghdad."

So, there's a disconnect there between — you're telling me this is fight of our generation, and you're going to increase troops by 10 percent. And that's gonna do it. I'm sure what he would like to do is send 400,000 more troops there, but he can't, because he doesn't have them. And the way to get that would be to institute a draft. And the minute you do that, suddenly the country's not so damn busy anymore. And then they really fight back, and then the whole thing falls apart. So, they have a really delicate balance to walk between keeping us relatively fearful, but not so fearful that we stop what we're doing and really examine how it is that they've been waging this.

And there was you know, this enormous amount of space and coverage to Virginia Tech, as there should have been. And I happened to catch, sort of a headline lower down, which was 200 people killed in four bomb attacks in Iraq. And I think my focus on what was happening here versus sort of this peripheral vision thing that caught my eye about, "Oh, right, there are lives--"

Planet RDF Roundup

• Research Project: Pig Given Yahoo's usage of Hadoop it's good to see them building a query layer on top of it. And it's not SQL - hence the name - it's relationally based (it uses bags and still has DISTINCT) because that scales. Hah! Some of the documentation is gold, "In a conventional database management system, SQL queries are translated into relational algebra expressions, which are in turn translated into physical evaluation plans. Pig Latin queries are already an algebra, so we're bypassing the first layer." It even has nested relations and a flatten operation (which removes the nesting). A good read at "Yahoo Pig and Google Sawzall". Notes that Google's Sawzall looks like Scala.


• REST Compile/Describe & WADL together with I finally get REST. Wow. all link to WADL. In what seems like an age ago, I remember several enterprise architects asking what the REST equivalent to WSDL was. The later has one of the best one liners to describe REST, "state machine as node graph traversed via URI". An example of how to use is give in "REST Describe first working Beta released"


• Stavanger, oil, and the Semantic Web Talks about the "Norwegian Semantic Web Days" which covered things such as the OBO foundary.

Jobs for my Dog

So I'm idly looking at the escalating real estate market and was considering ways my dog could finally make his living. So I thought back across jobs that I'd seen that could be done equally well by him:

1. A station master. While this sounds like a job requiring thumbs, really not so much. The station being mastered no longer received any trains but still took truck deliveries. Not very many though. Now, the station master is unable to load or unload these deliveries - this is done by the drivers. So the job mainly consisted of watching people drop things off and pick things up; a job for my dog.


2. Train driver. In recent memory an accident occurred that had the distinct posibility of being caused by both drivers doing things other than driving the train. The introduction of an automated systems was suggested to prevent it happening again. Driving an automated train is something that my dog could do quite well. Actually, he could do it twice as well and he wouldn't join the union. I'm not sure he'd stick to it as it would bore the heck out of him too.


3. Doorman. Now normally this would require the ability to open and close doors. But more recently, I noticed there was a guy at a building watching an automatic door open and close. Maybe he gave directions or something but the door watching seemed to be his prime activity. I've seen my dog look attentively at the front door - he's very qualified.


4. Network Administrator. Maybe the frontal lobes need to be developed further for this one but then again maybe not. The key activities undertaken for this position, that I've seen, revolved around attending meetings and playing Solitaire. I think my dog could do this job and he wouldn't even need a computer. Apparently, this is not an uncommon job amongst dogs.


5. Phone Handler (?). These are the people I've found at the front of buildings saying, "Please use that phone to contact people before going up". Usually replaced by a sign, I think an indicative paw would do as well.

Mistakes I've Made a Few

So the ghosts of implementations past have come to haunt me in recent weeks mostly around Kowari/Tucana (now Mulgara).

Firstly, there was exclude and NAF in iTQL:

> It was introduced to provide
> a limited form of negation, and one that interacts poorly with the
> open-world assumption. We also now have minus, which is well
> defined, corresponds closely to our intuitive understanding of the
> operation, and is (I am told) what was actually required.
>
> If my memory is correct we should probably at least deprecate, if
> not remove exclude entirely from mulgara.
>
> Do we agree that exclude should be removed?

Well *I* agree anyway. As you say, it doesn't do what anyone thinks
it does.

> If it should be removed, when should this occur?

Yesterday.

And now adding the Jena API to Kowari cost customers:

Another possible reason is Tucana/Kowari/Mulgara’s Jena support - originally put in to provide a migration path for companies looking to move on from research projects to scalable infrastructure - which as Jena is the defacto semweb tool of choice, people used to evaluate Kowari’s scalability. Jena’s lack of scaling hurt us several times, I can remember lots of frantic calls as some company wrote us off because of our Jena API.

I'm current still working in this area (maybe somewhat surprisingly) and Jena still dominates (all of the tools I'm currently looking at are Jena based). And I still haven't seen a Jena implementation that scales (see page 10). Maybe the decision to open source Kowari cost another round of funding too. Maybe this is why Garlick or Radar Network's triple stores are still behind closed doors.

Extending SPARQL

A few papers related to extending SPARQL, I may update this as I find more.

• Apart from, SPARQ2L and PSPARQL there's also at the ESWC 2007 conference: iSPARQL (which uses SimPack) and SPARQLeR (which supports some quite sophisticated path queries)


• SPARQL-DL: SPARQL Query for OWL-DL An enhancement of SPARQL to support DL semantics. And they note: "SPARQL-DL is a step between RDF QLs that are too unstructured w.r.t. OWL-DL and DL QLs which are not as expressive. We believe SPARQL-DL would help interoperability on the Semantic Web as it bridges this gap. As part of future work, we intend to investigate other possible extensions to SPARQL-DL including (but not limited to) aggregation operators, epistemic operators (and negation as failure), and regular expressions on OWL properties."


• SPARQL/Update Adding INSERT, MODIFY, DELETE and UPDATE to SPARQL. I'd seen this before but hadn't linked to it.

State of Origin

Same-Origin Policy Part 1: Why we’re stuck with things like XSS and XSRF/CSRF

In my experience most developers—and even many security people—don’t really know what the same-origin policy is. Worse yet, the rise of AJAX and mash-ups seems to have turned same-origin into something developers are trying to break. Complicating the issue further are the weaknesses in most browsers’ implementations of same-origin, leaving open questions about the effectiveness of the policy itself.

I was surprised that not everyone understood this recently talking to Web 2.0 gurus. Points to the very clear definition of what exactly is the same origin. See also: Subverting Ajax, which includes things like using XSS to extend the XMLHttpRequest object to capture calls and to record the data being transmitted.

Update: Just noticed that the lastest Crypto-gram highlights a similar attack overriding object creation.

Ontologies in 3D

"OntoSphere3D is a Protégé plug-in for ontologies navigation and inspection using a 3-dimensional hyper-space where information is presented on a 3D view-port enriched by several visual cues (as the colour or the size of visualized entities)."

While I'm not usually a fan of 3D interfaces this is a fairly intuitive approach. It's certainly seems better than the neighbourhood view and hyperbolic view that they talk about in their paper. They offer a "global view" (projected onto a sphere) and "tree focus".

Java 3D can be pain to install, at least it was for me, as it wouldn't detect Protégé's JRE to install into.

Lucene for the Semantic Web

Hbase

Google's [WWW] Bigtable, a distributed storage system for structured data, is a very effective mechanism for storing very large amounts of data in a distributed environment.

Just as Bigtable leverages the distributed data storage provided by the [WWW] Google File System, Hbase will provide Bigtable-like capabilities on top of Hadoop.

Data is organized into tables, rows and columns, but a query language like SQL is not supported. Instead, an Iterator-like interface is available for scanning through a row range (and of course there is an ability to retrieve a column value for a specific key).

Any particular column may have multiple values for the same row key. A secondary key can be provided to select a particular value or an Iterator can be set up to scan through the key-value pairs for that column given a specific row key.

From the Hbase/HbaseArchitecture page:

HBase uses a data model very similar to that of Bigtable. Users store data rows in labelled tables. A data row has a sortable key and an arbitrary number of columns. The table is stored sparsely, so that rows in the same table can have crazily-varying columns, if the user likes.

A column name has the form ":" where and can be any string you like. A single table enforces its set of s (called "column groups"). You can only adjust this set of groups by performing administrative operations on the table. However, you can use new strings at any write without preannouncing it. HBase stores column groups physically close on disk. So the items in a given column group should have roughly the same write/read behavior.

Writes are row-locked only. You cannot lock multiple rows at once. All row-writes are atomic by default.

All updates to the database have an associated timestamp. The HBase will store a configurable number of versions of a given cell. Clients can get data by asking for the "most recent value as of a certain time". Or, clients can fetch all available versions at once.

The example tables given are very similar to untyped relations. This has only just become part of the nightly build.

Via, Data Parallel.

RDF Path Queries

SPARQ2L: Towards Support For Subgraph Extraction Queries in RDF Databases

Many applications in analytical domains often have the need to connect the dots i.e., query about the structure of data. In bioinformatics for example, it is typical to want to query about interactions between proteins. The aim of such queries is to extract relationships between entities i.e. paths from a data graph. Often, such queries will specify certain constraints that qualifying results must satisfy e.g. paths involving a set of mandatory nodes. Unfortunately, most present day Semantic Web query languages including the current draft of the anticipated recommendation SPARQL, lack the ability to express queries about arbitrary path structures in data.

Implemented using Java and Berkley DB and the memory store Brahms. Also mentions PSPARQL (part of Exmo), which in February reached version complete status.

The Killer Demo

What a relief there's a demo that can turn someone onto the “semantic” Web, this has obviously been a long time coming. As long as you believe the hype, the way you get the Semantic Web is one A-list blogger at a time.

Other comments: "Scoble Gets the Semantic Web", "Scoble Gets the Semantic Web", "Nova Spivack sees to it that Robert Scoble finally gets the Semantic Web", "You say Tomato...", "Describing the Semantic Data Web (Take 2)" and "QOTD : Scobleized".

A New SQL

ETech '07 Summary - Part 2 - MegaData.

Here's the thing, we need a new kind of data store, a new kind of SQL, something that does for storing and querying large amounts of data what SQL did for normalized data.

Sure you can store a lot of data in a relational database, but when I say large, I mean really large; a billion or more records. I know we need this because I keep seeing people build it.

All this talk about making SPARQL behave like SQL maybe for nothing if people realize that's not what they need after all.

The back of the envelope scalability for an RDF store would be potentially 100s of billions of statements.

The key requirements highlighted are: distributed, joinless (no referential integrity at the store level), denormalized and transactionless.

I was aware of this because a comment linked to one of my previous posts about Kowari scalability (which I must of snuck through at some stage). Kowari got up to 10,000 triples/second later on its life.

What no MOOSE?

I promised myself I wasn't going to do this again, but I have one more, one more reason why the option of even having DISTINCT/LOOSE/CHOOSE in SPARQL is a bad idea. Part of this is stimulated because once more I'm sitting next to people trying to make the Semantic Web work and from my perspective SPARQL is letting them down.

It's not a new reason, it's one I wrote in 2004 which offers a pretty good reason why having this as an optional feature doesn't make sense for RDF:

"The other issue with the SPARQL is the lack of an implicit distinct. In my understanding of SQL, DISTINCT is optional because if your queries work on normalized data and joins are based on distinct keys then the returned results cannot be duplicated. If your query works on rows with repeated values on the same column then you apply DISTINCT.

In RDF's data model there isn't really this problem of duplicated data and normalization. SPARQL has the idea of matching statements in the graph. From my understanding, RDF's data model doesn't support the idea of multiple subject, predicates and/or objects with the same values.

In other words, it only seems valid that if a query matches one result in the graph it should return that one unique result not repeated multiple results."

This is on top of the other reasons I came up in "Bagging SPARQL". This could actually be seen as further discussion from the the initial response I got. Among other things, it was said that duplicates could arise by querying multiple graphs. I'd argue that forced distinct values provide the context to effectively count (or perform other aggregate values) across these multiple graphs.

It's three years on and they couldn't even allows users to declaratively count the number of statements in this mystical, future web of data.

Patterns In Software - Part 4

Patterns of Software: Tales from the Software Community (freely available in PDF). The chapter I'll be covering this time is: "The Failure of Pattern Languages".

So it begins with the punch that the last few chapters have been building up to:

Alexander’s story does not end with the publication of A Pattern Language in 1977. It went on and still goes on. And Alexander did not sit still after he wrote the patterns. Like any scientist he tried them out.

And they did not work.

The reason, quoted by Gabriel is that other processes other than architecture play a more fundamental role. That a building is not just the process of slapping together a bunch of very well thought out patterns based on emperical evidence. It's the result of many other processes like finance, zoning, construction, etc.

...one problem with the building process is lump-sum development. In such development few resources are brought to bear on the problems of repair and piecemeal growth. Instead, a large sum of money is dedicated to building a large artifact, and that artifact is allowed to deteriorate somewhat, and anything that is found lacking in the design or construction is ignored or minimally addressed until it is feasible to abandon the building and construct a replacement.

Gabriel gives an example from a previous chapter about how these processes can destroy the otherwise good qualities of development. For example, the process of getting a mortgage and paying it off directly influences the types of buildings built and used. Generally, you invest a large amount of money in a property, so large, that you usually can't afford to make piecemeal modifications because you can barely afford paying it off. This is okay as long as people are jumping from house to house. This avoids fixing up the problems there maybe with these houses until things get really bad and they are knocked down and rebuilt. Perhaps losing what was wrong with them in the first place.

So if you retain these old processes you still get the old results. Now it's clear in software that this is also the case - a nicely architected and controlled software project does not necessarily lead to high quality software. How often have you worked on a software project that you knew was going to be re-written in a few years anyway?

Gabriel suggest that the answer lies in good code and coders not in the typical separation of analysis, design and implementation:

And isn’t the old-style software methodology to put design in the hands of analysts and designers and to put coding in the hands of lowly coders, sometimes offshore coders who can be paid the lowest wages to do the least important work?

Methodologists who insist on separating analysis and design from coding are missing the essential feature of design: The design is in the code, not in a document or in a diagram. Half a programmer’s time is spent exploring the code, not in typing it in or changing it. When you look at the code you see its design, and that’s most of what you’re looking at, and it’s mostly while coding that you’re designing.

And finally, that the typical software patterns are not following Christopher Alexander's original concept of a pattern language:

When I look at software patterns and pattern languages, I don’t see the quality without a name in them, either. Recall that Alexander said that both the patterns themselves and the pattern language have the quality. In many cases today the pattern languages are written quickly, sort of like students doing homework problems. I heard one neophyte pattern writer say that when writing patterns he just writes what he knows and can write four or five patterns at a sitting.

The answer is to build software piecemeal, incrementally, partially designed and reflecting - much more like a Turkish rug and that's the next chapter, "The Bead Game, Rugs, and Beauty".

Defenders of the Web

Tim Berners Lee goes postal on spam "So how are you going to stop the Semantic Web being poisoned?"

TBL, the GLB, replied:

Well, everybody who's building the semantic web pretty much that I know are building systems take data from lots of places, but take data with an awareness of where those places are. So for example, suppose you're getting Geotags and the OS runs a service, lots of people in this country might trust the OS to say this point has a church with a spire - other people might say it's a great church to go to, other people might say it's a heathen church to go to... those are the other sources of data...

There was no let up from the press:

"But that was the basis for Google, and Google got poisoned... "

Shadbolt and Hendler stepped in to shield Sir Tim, but he was seething at the impertinence:

I remember a conference, we were discussing the Semantic Web, and someone asked what do you think is the worst thing that can happen and all the pencils come out. I know you two have been asking about "Woargh - I know the one about... what about the bad guys? Won't we be phished" There's a temptation to give readers about all the terrible things out there OK, and all the ways the web can become less usable.

At this point, your reporter wanted to remind Sir Tim that of all the problems the web has, a hostile press is not one of them. In fact, you can't pick up a newspaper or magazine without reading about how it's ushering in a New Age of Enlightenment. Time magazine gave "Person Of The Year" to every web user in America - or at least every one who looked at the mirror Time placed on its front cover.

He continued, cryptically:

Yes you'll find a bank that's less usable - ... I've never been phished.

So the Greatest Living Briton has never been phished, which is a relief. His answer to the Semantic Web didn't inspire much confidence for the rest of us: it would be used within the firewall, amongst trusted groups, "areas where one is much less worrying about the bad guys"."

Testing and AJAX

Two completely unrelated areas of interest.

The first is Unitils which automates a number of testing tasks. It includes:
* Equality assertion through reflection, with options like ignoring Java default/null values and ignoring order of collections.
* Support for database testing involving test data management with DbUnit, automatic maintenance of unit test databases and automatic constraints disabling
* Hibernate integration features such as session management and testing the mapping with the database
* Integration with Spring, involving ApplicationContext management and injection of spring managed beans
* Integration with EasyMock and injection of mocks into other objects

The cookbook offer concrete examples to do things like mock object injection and the like.

The second is Slingshot: Desktop Apps via Rails, "Slingshot looks like it wraps up a Rails app and packages it in such a way that the user is opening up a Site Specific Browser that accesses your world". It's seen as an alternative to the Firefox 3 or Apollo.

Just One More Layer of Abstraction

" Open Source, Open data, Process Models" has Sean McGrath linking to "Open Data matters more than Open Source" which is a comment on "Open Source is Dead".

Sean writes:

Traditionally, reference implementations (i.e. traditional source code) has been the way to do this. "Running code" is the final arbiter.

Maybe this is as good as it gets? Unfortunately, a fully blown word processor runs to many, many thousands of lines of code and the semantic devil is buried way down in the details...

Dave writes:

Until we can convince (or force) web sites to embrace and standardize on Open Data formats — XML, JSON, or even CSV, as appropriate — we will be in some ways even more locked in than we were in the bad old desktop days.

Dare writes:

Similarly, how much value do you think there is to be had from a snapshot of the source code for eBay or Facebook being made available? This is one area where Open Source offers no solution to the problem of vendor lock-in. In addition, the fact that we are increasingly moving to a Web-based world means that Open Source will be less and less effective as a mechanism for preventing vendor-lockin in the software industry. This is why Open Source is dead, as it will cease to be relevant in a world where most consumers of software actually use services as opposed to installing and maintaining software that is "distributed" to them.

The point the Sean is making is that even if we achieve what Dave is suggesting we still haven't solved the semantic problem. Making it explicit and non-proprietary is not found in XML, JSON or CSV - these just aren't descriptive enough. And having running code is all fine but it's not generic enough - it will be tied to Java or C# or whatever.

The answer is of course both, but both a data format that is descriptive enough (like RDF/OWL) and open source stores that have the ability to process large quantities of it (because you will have vaste quantities of your own data in the future and you won't want one company to own it).

MPTStore

Presentation Summary on MPTStore. A summary of an interesting approach by the Fedora guys to storing lots of triples, fast.

The real motivation behind experimenting with a new triplestore, however, was the NSDL use case. The National Science Digital Library5 (NSDL) is a moderately large repository (4.7 million objects, 250 million triples) with a lot of write activity (driven by periodic OAI harvests; primarily mixed ingests and datastream modifications). The NSDL data model also includes existential/referential integrity constraints that must be enforced. Querying the RI to determine correct repository state proved to be difficult: Kowari is aggressively buffering triple, sometimes on the order of seconds, before writing them to disk. Flushing the buffer after every write is also computationally expensive (hence the drive to use buffers in the first place).

Based on this observation, their solution, called “Mapped Predicate Tables,” creates a table for every predicate in the triplestore. This has several advantages: a low computational cost for triple adds and deletes, queries for known predicates are fast, complex queries benefit from the relatively mature RDBMS planner having finer-granularity statistics and query plans, and flexible data partitioning to help address scalability. This solution comes with several disadvantages, however: one needs to manage predicate to table mapping, complex queries crossing many predicates require more effort to formulate, and with a naive approach simple unbound queries scale linearly with the number of predicates.

They achieved basically the same performance with either asynchronous or synchronous modification.

The project is available on Sourceforge, including slides and javadoc (which has a similar design to JRDF except no blank nodes).

Reliving Old Times

SketchFighter 4000. If you're anything like me, and I assume you are, you will have done something like this. Or played one of the games inspired by thrust such as Oids.

Via, An open letter to Ambrosia Software: Please bring Sketch Fighter to the DS.

Scaling to Hundreds of Billions of Triples

Nova Spivack Talks with Talis (in MP3 format) talks about the recent work by Radar Networks and how they require a triple store to scale to hundreds of billions (from about 45 minutes in). Mentions an interesting business opportunity (48 minutes in) to build a scalable triple store (mentions Tucana and Kowari and how it disappeared) that handles lots of writes and federated queries.

News from Oracle

Oracle 11g Gains Native OWL Support and Oracle 11g to support some OWL inferencing. Following the links to it leads to, "Semantic Technologies Center" and an announcement about TopQuadrant TopBraid Composer supporting Oracle. Apart from the native OWL inferencing, faster querying and faster bulk loading the PDF referenced talks about the various scalability achieved (600 million triples), clustering, failover, concurrent read/write and query features such as a semantic relationship operators: sem_related and sem_distance.

On the Web No One Can Agree on the Schema

• "Why the Semantic Web will Fail" responses include "Business Doesn’t Cooperate? Are You Kidding?" and "Why the Semantic Web Marketing Message Has Failed". Which businesses or organisations cooperate internally, let alone externally? Couldn't this have been the same argument as to why TCP/IP, HTML, RSS, XML, relational databases, or any technology would've failed?
• Ajax Anti-Patterns and more includes: reduce polling, not inspecting callbacks, using XML, and reducing work on the server.
• "What's wrong with Ruby?" Suggests that Ruby won't make programming easier and doesn't include a substantially better way of handling abstractions, has non-intuitive operator overloading, and that dynamic modification of code is evil. The real answer is freedom from coding for CPUs.
• "Microcultures and the WII" "When I visit a friend, we listen to a radio station only airing electro remixes of C64 songs, which you can receive on the net. See, its the dude from Kaiserslautern who enjoys electro remixes of C64 songs who is the market of tomorrow." Maybe microcultures are bigger than you think. For all those that dig things like a capella versions of IK+ (the video is interesting).

A Better NetInfo

Sick of searching for unused GID and manually adding properties using NetInfo? A recent Peachpit artcle, "Master Mac OS X Users and Groups by Making Your Mac Think It's a Server" suggests downloading the Server Admin tool. It can run on any OS X machine by just hitting cancel initially and then connecting to the local directory (Apple+D). It suggests using GID from 600 so it doesn't clash with others.

Wondering About Transactions

Transactionless

A couple of years ago I was talking to a couple of friends of mine who were doing some work at eBay. It's always interesting to hear about the techniques people use on high volume sites, but perhaps one of the most interesting tidbits was that eBay does not use database transactions.

My immediate follow-up to the news of transactionless was to ask what the consequences were for the application programmer, in particular the overall feeling about transactionlessness. The reply was that it was odd at first, but ended up not being a big deal - much less of a problem than you might think. You have to pay attention to the order of your commits, getting the more important ones in first. At each commit you have to check that it succeeded and decide what to do if it fails.

I've been wondering about the usefulness of transactions for sometime and ACID in particular. One of the easiest to dismiss, suprisingly perhaps, is isolation. For example, skip list implementations ruin it as the data may change during iteration and it can easily be avoided by writing the information of one transaction to a file to be read by something else in another (either by accident or on purpose). The alternative might be something like CAS and RDF triples. Expecting application programmers to do it sounds awful however.

Also, noticed by, "Living Without Database Transactions".

Self Selection

I'm a bit late with this one but was interesting none the less, "The Power of the Marginal":

If it's corrupt enough, a test becomes an anti-test, filtering out the people it should select by making them to do things only the wrong people would do. Popularity in high school seems to be such a test. There are plenty of similar ones in the grownup world. For example, rising up through the hierarchy of the average big company demands an attention to politics few thoughtful people could spare. [3] Someone like Bill Gates can grow a company under him, but it's hard to imagine him having the patience to climb the corporate ladder at General Electric—or Microsoft, actually.

It's kind of strange when you think about it, because lord-of-the-flies schools and bureaucratic companies are both the default. There are probably a lot of people who go from one to the other and never realize the whole world doesn't work this way.

I think that's one reason big companies are so often blindsided by startups. People at big companies don't realize the extent to which they live in an environment that is one large, ongoing test for the wrong qualities.

If you're an outsider, your best chances for beating insiders are obviously in fields where corrupt tests select a lame elite. But there's a catch: if the tests are corrupt, your victory won't be recognized, at least in your lifetime.

He also contradicts the Bible:

Almost everyone makes the mistake of treating ideas as if they were indications of character rather than talent—as if having a stupid idea made you stupid. There's a huge weight of tradition advising us to play it safe. "Even a fool is thought wise if he keeps silent," says the Old Testament (Proverbs 17:28).

And Yoda:

The word "try" is an especially valuable component. I disagree here with Yoda, who said there is no try. There is try. It implies there's no punishment if you fail. You're driven by curiosity instead of duty. That means the wind of procrastination will be in your favor: instead of avoiding this work, this will be what you do as a way of avoiding other work. And when you do it, you'll be in a better mood.

Also the entire movie file is available as are the others from that conference (Martin Fowler deconstructing Rails is good too) in mp4.

Radiant RDF

A new and hopefully readable article on XML.com, "A Relational View of the Semantic Web". The main reason is to illicit some feedback and to make a more understanble version of things like this. It also discusses whether or not to use blank nodes when mapping RDF to the relational model (which is something I first published in my thesis). It includes my take, which is no doubt influenced by Simon and working on Kowari, on the DISTINCT issue amongst others.

Update: Well if any feedback is good feedback, having positive feedback is awesome: "But it begs the question: given the relative proximity of RDF/SPARQL to the relational model, why are the Semantic Web standards not closer both syntactically and semantically to it? This is, for me, the major reason which prevents its adoption...Nice to see such a straightforward and well-referenced article outlining the Semantic Web and relational worlds in any case. Thank you."

The question raised in the comment, why not keep pushing SQL, is one I wish I had a short answer for. Often SPARQL syntax has been chosen to look like SQL. But obviously it's still different enough because it is a different domain. I still favour a clean break from SQL for that reason.

I wrote about SQL not being relational enough quite a while ago now (nearly 3 years). At the time, I think I was unaware of some of the operations in SQL like INTERSECT and EXCEPT (set difference).

Update 2: Also noted by Nova Spivacks, "Excellent Overview of Benefits of RDF and SPARQL".

Guice Boost

Guice "Guice injects constructors, fields and methods (any methods with any number of arguments, not just setters). Guice includes advanced features such as custom scopes, circular dependencies, static member injection, Spring integration, and AOP Alliance method interception, most of which you can ignore until you need it."

The moment I saw IoC wiring in code I thought of one man's wish for better IoC wiring in code (we were using Spring at the time).

Behold:

 public class MyModule extends AbstractModule {
  protected void configure() {i
    bind(Service.class)
      .to(ServiceImpl.class)
      .in(Scopes.SINGLETON);
  }
}

And how to inject the Service implementation:

public class Client {

  private final Service service;

  @Inject
  public Client(Service service) {
    this.service = service;
  }

  public void go() {
    service.go();
  }
}

Now you can test drive IoC wiring without feeling fruity (test driving XML always felt weird).

Which reminded me of a quote by Victoria Livschitz:

We now have a generation of young programmers who think of software in terms of angle brackets. An enormous mess of XML documents that are now being created by enterprises at an alarming rate will be haunting our industry for decades. With all that excitement, no one seems to have the slightest interest in basic computer science.

She was talking about web services but it seems applicable to IoC wiring too.

Via, Guice 1.0 Released.

Update: Similarly, Configuration in Java - It sure beats XML!. Mentions, using Java 6's hotpatch to reload the configuration too.

Navigating Java Collections

As a followup to a posting a while back two articles about SortedSet and SortedMap made easier and the interfaces they implement, NavigableSet and NavigableMap. They are implemented by skip lists, Concurrent Skipt List Set and Concurrent Skip List Map.

The first article describes in more detail the poll, iteration and LHCF (lower (L), higher (H), ceiling (C) and floor (F)) methods. The later is useful returning elements in a sorted map less than, greather than, less than or equal to and greater than or equal to.

The second article, notes the lack of returning subsets based on values rather than keys: "The disadvantage with NavigableMap is that the subMap method has no provision for returning a map based on a range of values rather than on keys. Therefore, implementing something similar to database views is laborious."

The thing to remember with these skip list data structures is: "Iterators are weakly consistent, returning elements reflecting the state of the map at some point at or since the creation of the iterator."

Also, there's a recent article about being able to hotpatch in Java 6 (which Symantec Visual Cafe was doing back in 1997). Visual Cafe was an excellent IDE in its day, where many IDEs are still catching up although it was behind in things like refactoring.

Offensive Erlang

erlErr

Handling errors in Erlang is very different from handling errors in all other programming languages [...]. One of the most important rules we'll learn is not to program defensively. The idea of defensive programming—checking all arguments to a function—is alien to Erlang. So much so that we say that if the arguments to a function are incorrect then you should just let your program crash."

"This approach will seem very strange at first, but don't worry. The benefits are worth the effort. To start with, your programs will be a lot shorter. Sometimes as much as 30% of a conventional program can be devoted to defensive testing of function arguments—there is no such code in an Erlang program.

Incremental Adobe

Adobe edits the development cycle

The change we made was going from a traditional waterfall method to an incremental development model. Before, we would specify features up front, work on features until a "feature complete" date, and then (supposedly) revise the features based on alpha and beta testing and fix bugs. But we were scrambling so hard to get all the committed features in by the feature complete date - working nights and weekends up to the deadline - that the program was always very buggy at that point. We'd be desperately finding and fixing bugs, with little time to revise features based on tester feedback.

At the end of every cycle, we faced a huge "bugalanch" that required us to work many nights and weekends again. Of the three variables: features, schedule, and quality, the company sets the schedule and it's only slightly negotiable. Until feature complete, we could adjust the feature knob. But when we hit that milestone, quality sucked and we had only a fixed amount of time until the end. From there to the end, cutting features was not an option and all we could do was trade off our quality of life to get the quality of the product to the level we wanted by the ship date. We've never sacrificed product quality to get the product out the door, but we've sacrificed our home lives.

The quality of the program was higher throughout the development cycle, and there have been fewer total bugs. Instead of the bug count climbing towards a (frighteningly high) peak at "feature complete", it stayed at a much lower plateau throughout development. And we were able to incorporate more feedback from outside testers because we didn't switch into "frantic bug fix mode" so early.

Alphabetti Spaghetti!

Triple Soup "Apache TripleSoup is an effort started within the Apache Incubator to create a SPARQL endpoint that is easy to set up, fast maintainable, and such and so forth. We're only just getting started..."

The Triple Soup Wiki has more information: "TripleSoup is the simplest thing that you can do to turn your apache web server into a SPARQL endpoint. TripleSoup will be an RDF [2] store [3], tooling to work with that database, and a REST [4] web interface to talk to that database using SPARQL [5], implemented as an apache webserver module."

It's to be implemented in C and the authors of Redland (Dave Beckett) and B or lib B, a storage backend for RDF developed at Joost (Andrea Marchesini) are involved. Via Simon.

Also mentioned, "SPARQL Via HTTP Methods".

RDF is DOOMed and other links

• The Future of the World Wide Web, testimony of Sir Timothy Berners-Lee, via Seth Ladd. Some commentry by Nate Anderson.
• Peak Code "At some point, half of the total lines of code for that platform will be written. The further a platform gets past its peak, the less attractive adding new features to it becomes since there is less and less potential return on investment due to the dwindling number of programs that could benefit from the improvements."
• REST Issues, Real and Imagined False Choice: “High” vs. “Low” REST, Red Herring: The Evils of POST, Red Herring: URI Design, Red Herring: Cookies Are Bad and False Choice: Machines vs. People. Sure to be answered in, "REST Books Galore (Sam Ruby also mentions the recent flood of REST books).
• RDFRoom DOOM rendering of FOAF via Semantic Desktop Workshop wrapup
• What engineering has in common with manufacturing and why it matters "Engineering – software engineering in our case – is remarkably like manufacturing, once we shift to view the unvalidated decision as the unit of internal inventory."
• It’s Not RDF versus Microformats "If your world is limited to business cards, social events and reviews then I can see where you might favour expression in HTML. Humans want to read those things too. But the world is bigger than social networking and there are other types of first-class data that people want to exchange with one another."
• Ruby on Rails IDE comparison. Favours NetBeans.
• The State of Swing Mentions one of the small Java 6 enhancements, SwingWorker.

JRDF 0.4.1 and GUI 0.3.1 Released

JRDF 0.4.1 and the GUI are now available.

The main things changes from the last release:

• Initial SPARQL support that's been available in the JRDF GUI.
• RDF/XML writer.
• Push and pull RDF NTriples parser. In the future an RDF pull parser will also implement the RDFEventReader interface (which is similar to the StAX XMLEventReader).
• Allows sorted results from a graph find() to be returned.

SVG Support

Recently, Adobe announced the End Of Life (EOL) for their SVG plugin (the way that most people view SVG). And more importantly, the plugin is not certified to be installed on Vista.

From what I can tell that's because of their acquisition of Macromedia and Flash:

The bomb, though expected by the people in the know, was Adobe's discontinuation of its SVG viewer plug-in, the dominant SVG viewer. It was the way to show SVG in any browser (which for most users means the Internet Explorer browser), and though as Adobe had paid good money to buy Flash, the terms struck everyone as stunningly harsh, if not directly hostile to SVG: By the first announcement the plug-in would be gone and unavailable by the end of this year. Later Adobe relented giving the plug-in, a longer lease of life, you can download the unsupported plus-in next year too. This was obviously taken as bad news, but personally I think this is advantageous in the long run, much like when AOL Netscape ceased ownership of Mozilla. This was commonly seen as the end of Mozilla, but I thought it was more likely to be the beginning and was proven right. Sugar daddies are only advantageous to a point. Worse than the loss of ASV is that authoring products like Illustrator are unlikely to provide useful SVG support.

For cross browser rendering of SVG (or something SVG like) some alternatives include:

• Embedding Mozilla into IE to render SVG.
• Use a Javascript library that renders graphics either using VML (IE), SVG (Firefox, Opera) or Canvas (Safari, Firefox, Opera) these include: Dojo, ExplorerCanvas and Plotkit (which also has a good list of browsers supporting various SVG like technology).
• Use Batik as a Java applet.
• Use another plugin such as Rensis (0.5 released this month).

You Had Me at Hello

iPhone advert

Patterns In Software - Part 3

Patterns of Software: Tales from the Software Community (freely available in PDF). I was going to talk about, "The Failure of Pattern Languages" but I thought that a little discussion on the preceding chapter, "Pattern Languages", is required. This is somewhat of a generalisation of the preceding chapters and a discussion on Alexandar's "A Pattern Language", which offered the template for the software pattern books.

A comparison is made between art and science - art is creation, science is description - and that there are few activities that combine both. The exceptions given are computer science and architecture (I'd suggest maybe biological science is another). Trying to avoid the separation of construction and design is a mistake in computer science. Much or almost all of computer science is design and that it is expensive because it does combine both aspects for example, "What is Software Design?".

So this is specifically what all the discussion was a few years ago about TDD - being Test Driven Design (not Development). And the focus on behaviour rather than testing, for example Tom's recent Instinct project.

I want to set the stage for the accepted view of how architecture is done, at least in the pre-Alexander world. Architects are hired to solve the problem of how to construct a building or buildings that meet certain constraints as specified by the future inhabitants or people in a position to specify what those inhabitants will need. The architect generally interviews those future inhabitants, studies the site, refers to the local building code, observes neighboring buildings, considers the building materials and construction protocols in the area, and then is inspired, in the manner of all artists, to create a set of drawings which a general contractor, in occasional conference with the architect, reduces to a physical object.

Following from the suggestion given in the book, that the inhabitants are the programmers, it's obvious that "software architects" should ask the inhabitants of the software the best way to solve their problems. But that's not enough, more often than not they dictate solutions without knowledge of any of the complexity that their "solutions" might bring. This aligns with Alexander's suggestion for architects.

The first of Alexander’s contributions to architecture was to reject the separate architect and builder model and to posit user-centered design—in which users (inhabitants) design their own buildings—and the architect-builder who would blend the activities of design and construction. This was viewed by architects as high treason enough, let alone the curious quest for an elusive quality that Alexander cannot name.

The mechanism he proposed to accomplish his new model was the pattern language. A pattern language is a set of patterns used by a process to generate artifacts. These artifacts can be considered complexes of patterns. Each pattern is a kind of rule that states a problem to be solved and a solution to that problem.

I recently started to read a copy of "A Pattern Language" and I was struck by the same thing that Gabriel was, especially in comparison with the GoF book:

One thing that strikes many readers of the pattern language is the degree to which Alexander talks about people and their activities in the patterns — the patterns are a response to his arguments about how life is best lived. Because of this, many regard "A Pattern Language" as a manual of how to live and how physical surroundings can support and enhance living.

The patterns discussed in Alexander's book are all about what makes an area good for people - side walks (foot paths), water features, stairs, the height of crossings, making a city accessible to bicycle and foot traffic, etc - all these things make the environment livable and most ideas are supported by empirical evidence. The world of software patterns, by comparison, seems much more barren and arbitrary.

So there are some obviously good things about patterns:

Patterns seem to be a partial solution to the over abstraction problem I talked about in “Abstraction Descant.” They are a way to take advantage of common patterns without building costly, confusing, and unnecessary abstractions when the goal is merely to write something understandable. That is, when there are more idioms to use, using them is far better than inventing a new vocabulary.

Patterns in software are a good idea, but I have rarely seen the attention to the human side—to living within software—that I see in Alexander’s patterns. One architect commented on Alexander that he regarded the patterns as providing information, sort of heuristics, about architecture, and I have to admit the patterns I’ve seen about software do that—they provide information that the software practitioner just plain ought to know.

But this also highlights one of the problems with patterns. They discuss the context of the solution to a problem without first defining and more closely examining the problem itself, especially in a more general way. While it's often discussed in this chapter that patterns interact with other patterns and that they must be free of "systematic variations" the problems themselves don't seem to require this property. When people are critical of patterns, this is what I think they are objecting to, rather than the patterns themselves. He quotes Alexander saying: "You see that the patterns are very much alive and evolving."

And in a foreshadowing of what's to come:

The worst thing is this: Alexander had the chance in the 1970s to try out his pattern language. He observed the results of others trying it out and he even tried it out himself several times. And guess what: It didn’t work.

The next chapter, "The Failure of Pattern Languages", discusses the failure to apply these pattern but also a proposed solution to the problem: that the process should also have "the quality without a name".

Ruby Interfaces and Design By Contract

Interfaces in Ruby (attempted summary)

Technically speaking Ruby does not need interfaces since it is dynamically typed. Any argument list that at least satisfies the arity condition can be handed into a method.

Since people appreciate the usage of interfaces and even more Design by Contract (DbC) some mechanics were developed that mimic interfaces and certain aspects of DbC. I list them in order of increasing complexity.

(1) Document which methods a method argument must implement or its expected type.

(2) Define a module containing the methods that all raise an exception like this:

module FooInterface
def bar(a,b) raise "bar(a,b) must be overridden"; end
end

class FooClass
include FooInterface

def bar(a,b) a+b; end
end

(3) Create a framework (like the one Paul Brannan suggested here [1]) for defining interface methods and for checking that a particular instance implements these methods.

(4) Try to support even more of DbC by dealing with preconditions and postconditions like I tried in [2].

Recently announced 0.1 version of Ruby Design by Contract and Java style interfaces for ruby.

Stone Soup

I was looking at "Unnatural postscript hacks" (which points to the original 1988 version of the postscript ray tracer) where various crazy hacks exist including one of the Mandelbrot Set with a familiar quote about the difficulty of simplicity.

A co-worker said that the image rendered reminded him of the Buddha (turn the output to portrait and see what he means). So I explained what it was, a bit about complex systems, fractals, the length of a coastline, Lorenz attractors, etc and that reminded me of Fractint which I played with ages ago and I was surprise to see that it is still maintained. It was written by the "Stone Soup Group" which I vaguely remembered was a fable.

Sure enough, Stone Soup is on Wikipedia and C2 Wiki and it's often used as a metaphor for some aspects of open source software. Which raises the question of deception:

The villagers deceive the soldiers (OK - they lie to them), and that's bad. The soldiers deceive the villagers (tricking them to bring out the hidden food) and everyone benefits - the soldiers get to eat, but so do the villagers (in the version I know, the villagers hide their food not just from the soldiers but also from themselves). So, at a deeper level, the story raises questions of morality. When is deception acceptable? What gives anyone the authority to claim to have 'the big picture?'

I doubt that the villagers are hiding a shared collection of food. If they were, when each of them were getting items for the soup you'd think they'd see other villagers getting other items and discover what the visitors were up to. If the villagers are individually hoarding the food then it makes more sense that they are unaware that by working together they could produce a meal. In this version, the villages aren't being deceptive and the visitors are allowing them to work together to produce something they couldn't otherwise.

To Coin a Word

Tony makes Reddit again with a familiar term, Refunctoring.

Why ‘Refunctoring’? First, I refuse to reuse euphemistic terms that have served only to denigrate the standard of the software development industry, therefore, I cannot use ‘Refactoring’ (oooh! did I just say that!!?). To show this, consider the fact that the entire notion of ‘Refactoring’ is done away with one simple phrase ‘Functional Programming’ in the case shown and hundreds more. Of course, you might decide to redefine Refactoring to a different context, but then, redefining terms often leads to confusion (the term ‘function’ is itself a perfect example). To further demonstrate this point, open the book or catalog on Refactoring and turn to the section titled, ‘Replace Parameter with Method’. Then say out loud, ‘partial application’ — if you don’t know why you’re saying that, learn what partial application is, then perhaps it might dawn on you.

Anyway, Refunctoring is what most people really are doing. They are converting their code that is written in not-too-powerful programming languages to mimic the power that is already available in functional programming languages. They are representing higher-order functions with (importantly co-variant) polymorphic type parameters, refactoring out what partial application already provides and much more. This is being done, despite having been invented many, many years ago, but the fact that it is being done mandates a name, albeit how absurd it might be. I use the term Refunctoring, for no other reason than it was suggested to me by a bemused colleague (you know who you are :) ) one day and upon reflection, I have decided that it is entirely appropriate.

You don’t think the map function is a special case do you?

It's good to be known as bemused.

All Annotations

JSR-311: a Java API for RESTful Web Services?

I am still not so sure what they want to do exactly, but it seems to be based on JRA which has annotations of the form:

@Get
@HttpResource(location="/customers")
public List getCustomers();

For me a more interesting JSR would be one that would standardise the @rdf annotations the way so(m)mer is doing it.

The RESTlet framework was created because:

"I noted the lack of a REST framework in Java. The only project that came close to it was 1060 NetKernel developed by 1060 Research but it had too many features for my needs and I found that the mapping of REST concepts was not as direct as I was expecting."

It seems that annotations are an exceedingly useful tool, especially having used Elmo's annotations of interfaces to generate beans from RDF (link to PDF documentation). Annotating interfaces rather than classes, seems to be the way to go, given that interfaces can represent relationships like multiple inheritance. Elmo was inspired by subject oriented programming that provides a way "to group common behaviours and separate these different roles into unique interfaces and reusable classes." It's especially useful for unplanned extensions (through composition).

Googling for the various projects brought up, Tripresso looking at mapping RDF to OO (including Java, Ruby, Python and PHP projects). If object to relational mapping is the Vietnam of computer science, I wonder what object to RDF mapping is.

See the previous discussion, RDF on Rails ( mentioning ActiveRDF, RdfReactor, etc) and Henry Story's original, Java Annotations & the Semantic Web.

Behind Blue Links

• The other DL, defeasible logic.
• SableVM: A Research Framework for the Efficient Execution of Java Bytecode "...using our bidirectional layout, a gc only accesses the following memory locations while tracing: reference fields and lock word, for all instances (objects and arrays), and at most three additional accesses for objects with many fields..."
• Speaking of things Sable, Martin Fowler on SableCC "So far I'm not convinced by the approach of removing parser actions and automatically generating a parse tree. Since it's a parse tree, you have to walk it to do anything useful...One plus in tree walking is that if I'm making changes to the tree walker I don't need to re-generate - which keeps me in IntelliJ."
• Evolution rapidly evolving, Horizontal Gene Transfer Adds To Complexity, Speed Of Evolution, "...the speed of evolution has increased over time because bacteria and viruses constantly exchange transposable chunks of DNA between species...", via "The Coming Revolution in Biology".
• And the song to which or rather who the title alludes to.

The Death of Applets and AJAX

Hybridizing Java

So here’s my question. Allow for a moment the possibility that, after 10 years, Java is not going to take over the world of RIAs. Further allow that Ajax is just “how JavaScript was supposed to work in the first place,” but that the limitations imposed by browsers, HTML and CSS committees seem unlikely to let it expand beyond its current bounds. What are we going to use to build RIAs?

And the answer is?

The only obvious solution is Flash. Flash has always been all about cross-platform multimedia experiences and user interfaces. People are very familiar and comfortable with Flash, and it is installed on almost all machines in the world. It’s trusted, stable and reliable.

Do we just forget every 10 years what's going on, does anyone remember Flash Generator from 1998? And I still remember the horrible, inconsistent Flash UIs from that era. Has anything changed with Flex?

Via, Bruce Eckel: Use Flex as the UI for Java Applications

Why I May Never be Good at Scripting

In an effort to highlight my extreme ignorance I'll point out a recent bug I raised in the excellent Mochikit framework. In the end I came to the conclusion that Javascript libraries, while they can handle a great deal of incompatibilities between browsers, can't do it all.

I wrote the following Javascript:listMin([10,9,8,7,]);

This finds the minimum value in the list. In IE the result is undefined, in Firefox it is 7 and in Safari it's a syntax error. As I've found out, an array in Javascript as written above does a different thing in Firefox (an array containing 10, 9, 8, 7), IE (an array containing 10, 9, 8, 7, null) and Safari (syntax error). Checking this code against the specification (using something like JSLint) shows that the above code is really a syntax error (the extra comma after 7).

Part of the problem is that when comparing values in Mochikit a null or undefined is less than any other value. My expectation is either an exception or trying to make all browsers work the same way (not possible in Safari of course).

Now the result of these arrays were being passed around, in a functional way (which is part of why I liked Mochikit), and a more typical example is the second one given in the bug report:
map(function(x) {
if (!isUndefined(x)) return x[0];
}, [[10,10],[9,9],[8,8],[7,7],]).length;

Again, IE is creating an extra null object and returns 5 and it returns 4 in Firefox. The actual code using this was getting an exception in IE but not in Firefox because it was trying to treat a null object like an Array.

This is perhaps an example too, where functional passing style was more difficult to debug than if it was in a loop. But then you'd expect arrays to work the same (or at least I did) and that types are maintained (null not being an Array).

Now I'm generating code and allowing commas after each element in a list is a pretty familiar concept in heaps of languages. I can see why Firefox allowed this behaviour. I can also understand Safari following the specification. IE's behaviour though, doesn't make sense to me at all. It seems to have been purposefully created to cause problems.

All of this conspired to take many hours trying to debug code from a very lower level in several libraries, all the way back up to the highest where the arrays were being created.

A good API prevents these kinds of things from being processed - fail fast. I like Safari's behaviour the most - it would've saved me the most time (of course I was on a Windows machine so no help there).

Compliance Pays

Washington's $8 Billion Shadow "Another failed effort involves the F.B.I., which paid SAIC $124 million to bring the bureau, whose computer systems are among the most primitive in American law enforcement, into at least the late 20th century. The lack of information-sharing is one reason why the F.B.I. failed to realize that in the year leading up to 9/11 two of the future hijackers—including one with known "jihadist connections"—were actually living in the San Diego home of an F.B.I. informant. SAIC set to work on a system called the Virtual Case File. V.C.F. was supposed to become a central repository of data (wiretap transcripts, criminal records, financial transactions) from which all F.B.I. agents could draw. Three years and a million lines of garbled computer code later, V.C.F. has been written off by a global publication for technology professionals as "the most highly publicized software failure in history." The failure was due in part to the bureau's ever shifting directives, which points up the perverse nature of government-by-contract. When the government makes unrealistic demands, the contractors go along anyway: they are being paid not to resist but to comply."

Relational SPARQL?

ARQ : what next? "What next? This release matches the published DAWG working draft but the working group is considering a SPARQL algebra the formalization of the semantic of SPARQL to be based on some like the relational algebra."

Within Specification

The definition of quality is that it meets a certain specification. What a bunch of bollocks! I have a Holden Vectra (a rebadged Opel which people say is a very bad brand in its own country, Germany) - it requires more oil (about 1 litre) every 3,000-4,000 kilometer ever since I've bought it. It has only done 30,000 kilometers in 3 years but in that time it's run out of oil 4 times (between services). Now I ring up Holden support and say that this is an unacceptable specification. They say the car is fine, it is within the oil consumption specification.

Quality is excellence and that's not Holden.

Update: So here's a bit more, based on what Matthew wrote in the comments.

So the quality of the specification comes out of a process. And if that process doesn't have "the quality of excellence" in it, at all levels, then you're not achieving quality yet.

One of these aspects to this quality is a sort of self similarity - like well designed code - each bit of code does a small bit and does it in an understandable way and is roughly the same as the other bits - independent of which level of the system you're at.

If you can't raise an "IT Strategy Defect" then your organizational processes (which your lower processes are a part of) lack that aspect of quality. Now maybe that has to come out of high costs, failed projects or whatever.

From the perspective of the software industry, it seems that we've got a much higher expectation on what can be done. There aren't that many industries where the customer, project manager, support, maintenance, engineer, tester, etc. are even able to say (let alone expected or encouraged) "there's something wrong, fix it".

The guy at Holden support says, "I wouldn't buy a car with those specs" and so on - they know it sucks but they can't change it.

Now that isn't to say software should produce the equivalent of bubble domed cars that play "La Cucaracha". I'd like a flying car but at the moment I'd settle for something that doesn't need to be checked every 3,000 km and isn't going to destroy the planet.

Need for Speed

Bosworth On Why AJAX Failed, Then Succeeded links to which isn't the first time he has examined AJAX's success and failures or why other technologies fail such as RDF. Some commentary was interesting in the comments and a few blogs, including this summary:

All in all the speaker (intentionally) kept his talk simple and stupid but here are some points I think are worth noting:

• Products fail because of [lack of] human psychology not poor engineering
• Top 3 keypoints of UI:
1. Speed
2. Speed
3. Speed
• Your app should be a tool, it shouldn't leave the user feeling like one
• Search isn't perfect but it is far better than any human can do so it is tolerable.
• Analyze the frequency of use of feature and implement those first
• If your app takes more than 2 seconds to load - forget about it
• The Motorola RAZR was cool enough so the poor UI (narrow keypad) didnt matter