What no MOOSE?

I promised myself I wasn't going to do this again, but I have one more, one more reason why the option of even having DISTINCT/LOOSE/CHOOSE in SPARQL is a bad idea. Part of this is stimulated because once more I'm sitting next to people trying to make the Semantic Web work and from my perspective SPARQL is letting them down.

It's not a new reason, it's one I wrote in 2004 which offers a pretty good reason why having this as an optional feature doesn't make sense for RDF:

"The other issue with the SPARQL is the lack of an implicit distinct. In my understanding of SQL, DISTINCT is optional because if your queries work on normalized data and joins are based on distinct keys then the returned results cannot be duplicated. If your query works on rows with repeated values on the same column then you apply DISTINCT.

In RDF's data model there isn't really this problem of duplicated data and normalization. SPARQL has the idea of matching statements in the graph. From my understanding, RDF's data model doesn't support the idea of multiple subject, predicates and/or objects with the same values.

In other words, it only seems valid that if a query matches one result in the graph it should return that one unique result not repeated multiple results."

This is on top of the other reasons I came up in "Bagging SPARQL". This could actually be seen as further discussion from the the initial response I got. Among other things, it was said that duplicates could arise by querying multiple graphs. I'd argue that forced distinct values provide the context to effectively count (or perform other aggregate values) across these multiple graphs.

It's three years on and they couldn't even allows users to declaratively count the number of statements in this mystical, future web of data.

Patterns In Software - Part 4

Patterns of Software: Tales from the Software Community (freely available in PDF). The chapter I'll be covering this time is: "The Failure of Pattern Languages".

So it begins with the punch that the last few chapters have been building up to:

Alexander’s story does not end with the publication of A Pattern Language in 1977. It went on and still goes on. And Alexander did not sit still after he wrote the patterns. Like any scientist he tried them out.

And they did not work.

The reason, quoted by Gabriel is that other processes other than architecture play a more fundamental role. That a building is not just the process of slapping together a bunch of very well thought out patterns based on emperical evidence. It's the result of many other processes like finance, zoning, construction, etc.

...one problem with the building process is lump-sum development. In such development few resources are brought to bear on the problems of repair and piecemeal growth. Instead, a large sum of money is dedicated to building a large artifact, and that artifact is allowed to deteriorate somewhat, and anything that is found lacking in the design or construction is ignored or minimally addressed until it is feasible to abandon the building and construct a replacement.

Gabriel gives an example from a previous chapter about how these processes can destroy the otherwise good qualities of development. For example, the process of getting a mortgage and paying it off directly influences the types of buildings built and used. Generally, you invest a large amount of money in a property, so large, that you usually can't afford to make piecemeal modifications because you can barely afford paying it off. This is okay as long as people are jumping from house to house. This avoids fixing up the problems there maybe with these houses until things get really bad and they are knocked down and rebuilt. Perhaps losing what was wrong with them in the first place.

So if you retain these old processes you still get the old results. Now it's clear in software that this is also the case - a nicely architected and controlled software project does not necessarily lead to high quality software. How often have you worked on a software project that you knew was going to be re-written in a few years anyway?

Gabriel suggest that the answer lies in good code and coders not in the typical separation of analysis, design and implementation:

And isn’t the old-style software methodology to put design in the hands of analysts and designers and to put coding in the hands of lowly coders, sometimes offshore coders who can be paid the lowest wages to do the least important work?

Methodologists who insist on separating analysis and design from coding are missing the essential feature of design: The design is in the code, not in a document or in a diagram. Half a programmer’s time is spent exploring the code, not in typing it in or changing it. When you look at the code you see its design, and that’s most of what you’re looking at, and it’s mostly while coding that you’re designing.

And finally, that the typical software patterns are not following Christopher Alexander's original concept of a pattern language:

When I look at software patterns and pattern languages, I don’t see the quality without a name in them, either. Recall that Alexander said that both the patterns themselves and the pattern language have the quality. In many cases today the pattern languages are written quickly, sort of like students doing homework problems. I heard one neophyte pattern writer say that when writing patterns he just writes what he knows and can write four or five patterns at a sitting.

The answer is to build software piecemeal, incrementally, partially designed and reflecting - much more like a Turkish rug and that's the next chapter, "The Bead Game, Rugs, and Beauty".

Defenders of the Web

Tim Berners Lee goes postal on spam "So how are you going to stop the Semantic Web being poisoned?"

TBL, the GLB, replied:

Well, everybody who's building the semantic web pretty much that I know are building systems take data from lots of places, but take data with an awareness of where those places are. So for example, suppose you're getting Geotags and the OS runs a service, lots of people in this country might trust the OS to say this point has a church with a spire - other people might say it's a great church to go to, other people might say it's a heathen church to go to... those are the other sources of data...

There was no let up from the press:

"But that was the basis for Google, and Google got poisoned... "

Shadbolt and Hendler stepped in to shield Sir Tim, but he was seething at the impertinence:

I remember a conference, we were discussing the Semantic Web, and someone asked what do you think is the worst thing that can happen and all the pencils come out. I know you two have been asking about "Woargh - I know the one about... what about the bad guys? Won't we be phished" There's a temptation to give readers about all the terrible things out there OK, and all the ways the web can become less usable.

At this point, your reporter wanted to remind Sir Tim that of all the problems the web has, a hostile press is not one of them. In fact, you can't pick up a newspaper or magazine without reading about how it's ushering in a New Age of Enlightenment. Time magazine gave "Person Of The Year" to every web user in America - or at least every one who looked at the mirror Time placed on its front cover.

He continued, cryptically:

Yes you'll find a bank that's less usable - ... I've never been phished.

So the Greatest Living Briton has never been phished, which is a relief. His answer to the Semantic Web didn't inspire much confidence for the rest of us: it would be used within the firewall, amongst trusted groups, "areas where one is much less worrying about the bad guys"."

Testing and AJAX

Two completely unrelated areas of interest.

The first is Unitils which automates a number of testing tasks. It includes:
* Equality assertion through reflection, with options like ignoring Java default/null values and ignoring order of collections.
* Support for database testing involving test data management with DbUnit, automatic maintenance of unit test databases and automatic constraints disabling
* Hibernate integration features such as session management and testing the mapping with the database
* Integration with Spring, involving ApplicationContext management and injection of spring managed beans
* Integration with EasyMock and injection of mocks into other objects

The cookbook offer concrete examples to do things like mock object injection and the like.

The second is Slingshot: Desktop Apps via Rails, "Slingshot looks like it wraps up a Rails app and packages it in such a way that the user is opening up a Site Specific Browser that accesses your world". It's seen as an alternative to the Firefox 3 or Apollo.

Just One More Layer of Abstraction

" Open Source, Open data, Process Models" has Sean McGrath linking to "Open Data matters more than Open Source" which is a comment on "Open Source is Dead".

Sean writes:

Traditionally, reference implementations (i.e. traditional source code) has been the way to do this. "Running code" is the final arbiter.

Maybe this is as good as it gets? Unfortunately, a fully blown word processor runs to many, many thousands of lines of code and the semantic devil is buried way down in the details...

Dave writes:

Until we can convince (or force) web sites to embrace and standardize on Open Data formats — XML, JSON, or even CSV, as appropriate — we will be in some ways even more locked in than we were in the bad old desktop days.

Dare writes:

Similarly, how much value do you think there is to be had from a snapshot of the source code for eBay or Facebook being made available? This is one area where Open Source offers no solution to the problem of vendor lock-in. In addition, the fact that we are increasingly moving to a Web-based world means that Open Source will be less and less effective as a mechanism for preventing vendor-lockin in the software industry. This is why Open Source is dead, as it will cease to be relevant in a world where most consumers of software actually use services as opposed to installing and maintaining software that is "distributed" to them.

The point the Sean is making is that even if we achieve what Dave is suggesting we still haven't solved the semantic problem. Making it explicit and non-proprietary is not found in XML, JSON or CSV - these just aren't descriptive enough. And having running code is all fine but it's not generic enough - it will be tied to Java or C# or whatever.

The answer is of course both, but both a data format that is descriptive enough (like RDF/OWL) and open source stores that have the ability to process large quantities of it (because you will have vaste quantities of your own data in the future and you won't want one company to own it).

MPTStore

Presentation Summary on MPTStore. A summary of an interesting approach by the Fedora guys to storing lots of triples, fast.

The real motivation behind experimenting with a new triplestore, however, was the NSDL use case. The National Science Digital Library5 (NSDL) is a moderately large repository (4.7 million objects, 250 million triples) with a lot of write activity (driven by periodic OAI harvests; primarily mixed ingests and datastream modifications). The NSDL data model also includes existential/referential integrity constraints that must be enforced. Querying the RI to determine correct repository state proved to be difficult: Kowari is aggressively buffering triple, sometimes on the order of seconds, before writing them to disk. Flushing the buffer after every write is also computationally expensive (hence the drive to use buffers in the first place).

Based on this observation, their solution, called “Mapped Predicate Tables,” creates a table for every predicate in the triplestore. This has several advantages: a low computational cost for triple adds and deletes, queries for known predicates are fast, complex queries benefit from the relatively mature RDBMS planner having finer-granularity statistics and query plans, and flexible data partitioning to help address scalability. This solution comes with several disadvantages, however: one needs to manage predicate to table mapping, complex queries crossing many predicates require more effort to formulate, and with a naive approach simple unbound queries scale linearly with the number of predicates.

They achieved basically the same performance with either asynchronous or synchronous modification.

The project is available on Sourceforge, including slides and javadoc (which has a similar design to JRDF except no blank nodes).

Reliving Old Times

SketchFighter 4000. If you're anything like me, and I assume you are, you will have done something like this. Or played one of the games inspired by thrust such as Oids.

Via, An open letter to Ambrosia Software: Please bring Sketch Fighter to the DS.

Scaling to Hundreds of Billions of Triples

Nova Spivack Talks with Talis (in MP3 format) talks about the recent work by Radar Networks and how they require a triple store to scale to hundreds of billions (from about 45 minutes in). Mentions an interesting business opportunity (48 minutes in) to build a scalable triple store (mentions Tucana and Kowari and how it disappeared) that handles lots of writes and federated queries.

News from Oracle

Oracle 11g Gains Native OWL Support and Oracle 11g to support some OWL inferencing. Following the links to it leads to, "Semantic Technologies Center" and an announcement about TopQuadrant TopBraid Composer supporting Oracle. Apart from the native OWL inferencing, faster querying and faster bulk loading the PDF referenced talks about the various scalability achieved (600 million triples), clustering, failover, concurrent read/write and query features such as a semantic relationship operators: sem_related and sem_distance.

On the Web No One Can Agree on the Schema

• "Why the Semantic Web will Fail" responses include "Business Doesn’t Cooperate? Are You Kidding?" and "Why the Semantic Web Marketing Message Has Failed". Which businesses or organisations cooperate internally, let alone externally? Couldn't this have been the same argument as to why TCP/IP, HTML, RSS, XML, relational databases, or any technology would've failed?
• Ajax Anti-Patterns and more includes: reduce polling, not inspecting callbacks, using XML, and reducing work on the server.
• "What's wrong with Ruby?" Suggests that Ruby won't make programming easier and doesn't include a substantially better way of handling abstractions, has non-intuitive operator overloading, and that dynamic modification of code is evil. The real answer is freedom from coding for CPUs.
• "Microcultures and the WII" "When I visit a friend, we listen to a radio station only airing electro remixes of C64 songs, which you can receive on the net. See, its the dude from Kaiserslautern who enjoys electro remixes of C64 songs who is the market of tomorrow." Maybe microcultures are bigger than you think. For all those that dig things like a capella versions of IK+ (the video is interesting).

A Better NetInfo

Sick of searching for unused GID and manually adding properties using NetInfo? A recent Peachpit artcle, "Master Mac OS X Users and Groups by Making Your Mac Think It's a Server" suggests downloading the Server Admin tool. It can run on any OS X machine by just hitting cancel initially and then connecting to the local directory (Apple+D). It suggests using GID from 600 so it doesn't clash with others.

Wondering About Transactions

Transactionless

A couple of years ago I was talking to a couple of friends of mine who were doing some work at eBay. It's always interesting to hear about the techniques people use on high volume sites, but perhaps one of the most interesting tidbits was that eBay does not use database transactions.

My immediate follow-up to the news of transactionless was to ask what the consequences were for the application programmer, in particular the overall feeling about transactionlessness. The reply was that it was odd at first, but ended up not being a big deal - much less of a problem than you might think. You have to pay attention to the order of your commits, getting the more important ones in first. At each commit you have to check that it succeeded and decide what to do if it fails.

I've been wondering about the usefulness of transactions for sometime and ACID in particular. One of the easiest to dismiss, suprisingly perhaps, is isolation. For example, skip list implementations ruin it as the data may change during iteration and it can easily be avoided by writing the information of one transaction to a file to be read by something else in another (either by accident or on purpose). The alternative might be something like CAS and RDF triples. Expecting application programmers to do it sounds awful however.

Also, noticed by, "Living Without Database Transactions".

Self Selection

I'm a bit late with this one but was interesting none the less, "The Power of the Marginal":

If it's corrupt enough, a test becomes an anti-test, filtering out the people it should select by making them to do things only the wrong people would do. Popularity in high school seems to be such a test. There are plenty of similar ones in the grownup world. For example, rising up through the hierarchy of the average big company demands an attention to politics few thoughtful people could spare. [3] Someone like Bill Gates can grow a company under him, but it's hard to imagine him having the patience to climb the corporate ladder at General Electric—or Microsoft, actually.

It's kind of strange when you think about it, because lord-of-the-flies schools and bureaucratic companies are both the default. There are probably a lot of people who go from one to the other and never realize the whole world doesn't work this way.

I think that's one reason big companies are so often blindsided by startups. People at big companies don't realize the extent to which they live in an environment that is one large, ongoing test for the wrong qualities.

If you're an outsider, your best chances for beating insiders are obviously in fields where corrupt tests select a lame elite. But there's a catch: if the tests are corrupt, your victory won't be recognized, at least in your lifetime.

He also contradicts the Bible:

Almost everyone makes the mistake of treating ideas as if they were indications of character rather than talent—as if having a stupid idea made you stupid. There's a huge weight of tradition advising us to play it safe. "Even a fool is thought wise if he keeps silent," says the Old Testament (Proverbs 17:28).

And Yoda:

The word "try" is an especially valuable component. I disagree here with Yoda, who said there is no try. There is try. It implies there's no punishment if you fail. You're driven by curiosity instead of duty. That means the wind of procrastination will be in your favor: instead of avoiding this work, this will be what you do as a way of avoiding other work. And when you do it, you'll be in a better mood.

Also the entire movie file is available as are the others from that conference (Martin Fowler deconstructing Rails is good too) in mp4.

Radiant RDF

A new and hopefully readable article on XML.com, "A Relational View of the Semantic Web". The main reason is to illicit some feedback and to make a more understanble version of things like this. It also discusses whether or not to use blank nodes when mapping RDF to the relational model (which is something I first published in my thesis). It includes my take, which is no doubt influenced by Simon and working on Kowari, on the DISTINCT issue amongst others.

Update: Well if any feedback is good feedback, having positive feedback is awesome: "But it begs the question: given the relative proximity of RDF/SPARQL to the relational model, why are the Semantic Web standards not closer both syntactically and semantically to it? This is, for me, the major reason which prevents its adoption...Nice to see such a straightforward and well-referenced article outlining the Semantic Web and relational worlds in any case. Thank you."

The question raised in the comment, why not keep pushing SQL, is one I wish I had a short answer for. Often SPARQL syntax has been chosen to look like SQL. But obviously it's still different enough because it is a different domain. I still favour a clean break from SQL for that reason.

I wrote about SQL not being relational enough quite a while ago now (nearly 3 years). At the time, I think I was unaware of some of the operations in SQL like INTERSECT and EXCEPT (set difference).

Update 2: Also noted by Nova Spivacks, "Excellent Overview of Benefits of RDF and SPARQL".

Guice Boost

Guice "Guice injects constructors, fields and methods (any methods with any number of arguments, not just setters). Guice includes advanced features such as custom scopes, circular dependencies, static member injection, Spring integration, and AOP Alliance method interception, most of which you can ignore until you need it."

The moment I saw IoC wiring in code I thought of one man's wish for better IoC wiring in code (we were using Spring at the time).

Behold:

 public class MyModule extends AbstractModule {
  protected void configure() {i
    bind(Service.class)
      .to(ServiceImpl.class)
      .in(Scopes.SINGLETON);
  }
}

And how to inject the Service implementation:

public class Client {

  private final Service service;

  @Inject
  public Client(Service service) {
    this.service = service;
  }

  public void go() {
    service.go();
  }
}

Now you can test drive IoC wiring without feeling fruity (test driving XML always felt weird).

Which reminded me of a quote by Victoria Livschitz:

We now have a generation of young programmers who think of software in terms of angle brackets. An enormous mess of XML documents that are now being created by enterprises at an alarming rate will be haunting our industry for decades. With all that excitement, no one seems to have the slightest interest in basic computer science.

She was talking about web services but it seems applicable to IoC wiring too.

Via, Guice 1.0 Released.

Update: Similarly, Configuration in Java - It sure beats XML!. Mentions, using Java 6's hotpatch to reload the configuration too.

Navigating Java Collections

As a followup to a posting a while back two articles about SortedSet and SortedMap made easier and the interfaces they implement, NavigableSet and NavigableMap. They are implemented by skip lists, Concurrent Skipt List Set and Concurrent Skip List Map.

The first article describes in more detail the poll, iteration and LHCF (lower (L), higher (H), ceiling (C) and floor (F)) methods. The later is useful returning elements in a sorted map less than, greather than, less than or equal to and greater than or equal to.

The second article, notes the lack of returning subsets based on values rather than keys: "The disadvantage with NavigableMap is that the subMap method has no provision for returning a map based on a range of values rather than on keys. Therefore, implementing something similar to database views is laborious."

The thing to remember with these skip list data structures is: "Iterators are weakly consistent, returning elements reflecting the state of the map at some point at or since the creation of the iterator."

Also, there's a recent article about being able to hotpatch in Java 6 (which Symantec Visual Cafe was doing back in 1997). Visual Cafe was an excellent IDE in its day, where many IDEs are still catching up although it was behind in things like refactoring.

Offensive Erlang

erlErr

Handling errors in Erlang is very different from handling errors in all other programming languages [...]. One of the most important rules we'll learn is not to program defensively. The idea of defensive programming—checking all arguments to a function—is alien to Erlang. So much so that we say that if the arguments to a function are incorrect then you should just let your program crash."

"This approach will seem very strange at first, but don't worry. The benefits are worth the effort. To start with, your programs will be a lot shorter. Sometimes as much as 30% of a conventional program can be devoted to defensive testing of function arguments—there is no such code in an Erlang program.

Incremental Adobe

Adobe edits the development cycle

The change we made was going from a traditional waterfall method to an incremental development model. Before, we would specify features up front, work on features until a "feature complete" date, and then (supposedly) revise the features based on alpha and beta testing and fix bugs. But we were scrambling so hard to get all the committed features in by the feature complete date - working nights and weekends up to the deadline - that the program was always very buggy at that point. We'd be desperately finding and fixing bugs, with little time to revise features based on tester feedback.

At the end of every cycle, we faced a huge "bugalanch" that required us to work many nights and weekends again. Of the three variables: features, schedule, and quality, the company sets the schedule and it's only slightly negotiable. Until feature complete, we could adjust the feature knob. But when we hit that milestone, quality sucked and we had only a fixed amount of time until the end. From there to the end, cutting features was not an option and all we could do was trade off our quality of life to get the quality of the product to the level we wanted by the ship date. We've never sacrificed product quality to get the product out the door, but we've sacrificed our home lives.

The quality of the program was higher throughout the development cycle, and there have been fewer total bugs. Instead of the bug count climbing towards a (frighteningly high) peak at "feature complete", it stayed at a much lower plateau throughout development. And we were able to incorporate more feedback from outside testers because we didn't switch into "frantic bug fix mode" so early.

Alphabetti Spaghetti!

Triple Soup "Apache TripleSoup is an effort started within the Apache Incubator to create a SPARQL endpoint that is easy to set up, fast maintainable, and such and so forth. We're only just getting started..."

The Triple Soup Wiki has more information: "TripleSoup is the simplest thing that you can do to turn your apache web server into a SPARQL endpoint. TripleSoup will be an RDF [2] store [3], tooling to work with that database, and a REST [4] web interface to talk to that database using SPARQL [5], implemented as an apache webserver module."

It's to be implemented in C and the authors of Redland (Dave Beckett) and B or lib B, a storage backend for RDF developed at Joost (Andrea Marchesini) are involved. Via Simon.

Also mentioned, "SPARQL Via HTTP Methods".

RDF is DOOMed and other links

• The Future of the World Wide Web, testimony of Sir Timothy Berners-Lee, via Seth Ladd. Some commentry by Nate Anderson.
• Peak Code "At some point, half of the total lines of code for that platform will be written. The further a platform gets past its peak, the less attractive adding new features to it becomes since there is less and less potential return on investment due to the dwindling number of programs that could benefit from the improvements."
• REST Issues, Real and Imagined False Choice: “High” vs. “Low” REST, Red Herring: The Evils of POST, Red Herring: URI Design, Red Herring: Cookies Are Bad and False Choice: Machines vs. People. Sure to be answered in, "REST Books Galore (Sam Ruby also mentions the recent flood of REST books).
• RDFRoom DOOM rendering of FOAF via Semantic Desktop Workshop wrapup
• What engineering has in common with manufacturing and why it matters "Engineering – software engineering in our case – is remarkably like manufacturing, once we shift to view the unvalidated decision as the unit of internal inventory."
• It’s Not RDF versus Microformats "If your world is limited to business cards, social events and reviews then I can see where you might favour expression in HTML. Humans want to read those things too. But the world is bigger than social networking and there are other types of first-class data that people want to exchange with one another."
• Ruby on Rails IDE comparison. Favours NetBeans.
• The State of Swing Mentions one of the small Java 6 enhancements, SwingWorker.

JRDF 0.4.1 and GUI 0.3.1 Released

JRDF 0.4.1 and the GUI are now available.

The main things changes from the last release:

• Initial SPARQL support that's been available in the JRDF GUI.
• RDF/XML writer.
• Push and pull RDF NTriples parser. In the future an RDF pull parser will also implement the RDFEventReader interface (which is similar to the StAX XMLEventReader).
• Allows sorted results from a graph find() to be returned.

SVG Support

Recently, Adobe announced the End Of Life (EOL) for their SVG plugin (the way that most people view SVG). And more importantly, the plugin is not certified to be installed on Vista.

From what I can tell that's because of their acquisition of Macromedia and Flash:

The bomb, though expected by the people in the know, was Adobe's discontinuation of its SVG viewer plug-in, the dominant SVG viewer. It was the way to show SVG in any browser (which for most users means the Internet Explorer browser), and though as Adobe had paid good money to buy Flash, the terms struck everyone as stunningly harsh, if not directly hostile to SVG: By the first announcement the plug-in would be gone and unavailable by the end of this year. Later Adobe relented giving the plug-in, a longer lease of life, you can download the unsupported plus-in next year too. This was obviously taken as bad news, but personally I think this is advantageous in the long run, much like when AOL Netscape ceased ownership of Mozilla. This was commonly seen as the end of Mozilla, but I thought it was more likely to be the beginning and was proven right. Sugar daddies are only advantageous to a point. Worse than the loss of ASV is that authoring products like Illustrator are unlikely to provide useful SVG support.

For cross browser rendering of SVG (or something SVG like) some alternatives include:

• Embedding Mozilla into IE to render SVG.
• Use a Javascript library that renders graphics either using VML (IE), SVG (Firefox, Opera) or Canvas (Safari, Firefox, Opera) these include: Dojo, ExplorerCanvas and Plotkit (which also has a good list of browsers supporting various SVG like technology).
• Use Batik as a Java applet.
• Use another plugin such as Rensis (0.5 released this month).

You Had Me at Hello

iPhone advert

Patterns In Software - Part 3

Patterns of Software: Tales from the Software Community (freely available in PDF). I was going to talk about, "The Failure of Pattern Languages" but I thought that a little discussion on the preceding chapter, "Pattern Languages", is required. This is somewhat of a generalisation of the preceding chapters and a discussion on Alexandar's "A Pattern Language", which offered the template for the software pattern books.

A comparison is made between art and science - art is creation, science is description - and that there are few activities that combine both. The exceptions given are computer science and architecture (I'd suggest maybe biological science is another). Trying to avoid the separation of construction and design is a mistake in computer science. Much or almost all of computer science is design and that it is expensive because it does combine both aspects for example, "What is Software Design?".

So this is specifically what all the discussion was a few years ago about TDD - being Test Driven Design (not Development). And the focus on behaviour rather than testing, for example Tom's recent Instinct project.

I want to set the stage for the accepted view of how architecture is done, at least in the pre-Alexander world. Architects are hired to solve the problem of how to construct a building or buildings that meet certain constraints as specified by the future inhabitants or people in a position to specify what those inhabitants will need. The architect generally interviews those future inhabitants, studies the site, refers to the local building code, observes neighboring buildings, considers the building materials and construction protocols in the area, and then is inspired, in the manner of all artists, to create a set of drawings which a general contractor, in occasional conference with the architect, reduces to a physical object.

Following from the suggestion given in the book, that the inhabitants are the programmers, it's obvious that "software architects" should ask the inhabitants of the software the best way to solve their problems. But that's not enough, more often than not they dictate solutions without knowledge of any of the complexity that their "solutions" might bring. This aligns with Alexander's suggestion for architects.

The first of Alexander’s contributions to architecture was to reject the separate architect and builder model and to posit user-centered design—in which users (inhabitants) design their own buildings—and the architect-builder who would blend the activities of design and construction. This was viewed by architects as high treason enough, let alone the curious quest for an elusive quality that Alexander cannot name.

The mechanism he proposed to accomplish his new model was the pattern language. A pattern language is a set of patterns used by a process to generate artifacts. These artifacts can be considered complexes of patterns. Each pattern is a kind of rule that states a problem to be solved and a solution to that problem.

I recently started to read a copy of "A Pattern Language" and I was struck by the same thing that Gabriel was, especially in comparison with the GoF book:

One thing that strikes many readers of the pattern language is the degree to which Alexander talks about people and their activities in the patterns — the patterns are a response to his arguments about how life is best lived. Because of this, many regard "A Pattern Language" as a manual of how to live and how physical surroundings can support and enhance living.

The patterns discussed in Alexander's book are all about what makes an area good for people - side walks (foot paths), water features, stairs, the height of crossings, making a city accessible to bicycle and foot traffic, etc - all these things make the environment livable and most ideas are supported by empirical evidence. The world of software patterns, by comparison, seems much more barren and arbitrary.

So there are some obviously good things about patterns:

Patterns seem to be a partial solution to the over abstraction problem I talked about in “Abstraction Descant.” They are a way to take advantage of common patterns without building costly, confusing, and unnecessary abstractions when the goal is merely to write something understandable. That is, when there are more idioms to use, using them is far better than inventing a new vocabulary.

Patterns in software are a good idea, but I have rarely seen the attention to the human side—to living within software—that I see in Alexander’s patterns. One architect commented on Alexander that he regarded the patterns as providing information, sort of heuristics, about architecture, and I have to admit the patterns I’ve seen about software do that—they provide information that the software practitioner just plain ought to know.

But this also highlights one of the problems with patterns. They discuss the context of the solution to a problem without first defining and more closely examining the problem itself, especially in a more general way. While it's often discussed in this chapter that patterns interact with other patterns and that they must be free of "systematic variations" the problems themselves don't seem to require this property. When people are critical of patterns, this is what I think they are objecting to, rather than the patterns themselves. He quotes Alexander saying: "You see that the patterns are very much alive and evolving."

And in a foreshadowing of what's to come:

The worst thing is this: Alexander had the chance in the 1970s to try out his pattern language. He observed the results of others trying it out and he even tried it out himself several times. And guess what: It didn’t work.

The next chapter, "The Failure of Pattern Languages", discusses the failure to apply these pattern but also a proposed solution to the problem: that the process should also have "the quality without a name".

Ruby Interfaces and Design By Contract

Interfaces in Ruby (attempted summary)

Technically speaking Ruby does not need interfaces since it is dynamically typed. Any argument list that at least satisfies the arity condition can be handed into a method.

Since people appreciate the usage of interfaces and even more Design by Contract (DbC) some mechanics were developed that mimic interfaces and certain aspects of DbC. I list them in order of increasing complexity.

(1) Document which methods a method argument must implement or its expected type.

(2) Define a module containing the methods that all raise an exception like this:

module FooInterface
def bar(a,b) raise "bar(a,b) must be overridden"; end
end

class FooClass
include FooInterface

def bar(a,b) a+b; end
end

(3) Create a framework (like the one Paul Brannan suggested here [1]) for defining interface methods and for checking that a particular instance implements these methods.

(4) Try to support even more of DbC by dealing with preconditions and postconditions like I tried in [2].

Recently announced 0.1 version of Ruby Design by Contract and Java style interfaces for ruby.

Stone Soup

I was looking at "Unnatural postscript hacks" (which points to the original 1988 version of the postscript ray tracer) where various crazy hacks exist including one of the Mandelbrot Set with a familiar quote about the difficulty of simplicity.

A co-worker said that the image rendered reminded him of the Buddha (turn the output to portrait and see what he means). So I explained what it was, a bit about complex systems, fractals, the length of a coastline, Lorenz attractors, etc and that reminded me of Fractint which I played with ages ago and I was surprise to see that it is still maintained. It was written by the "Stone Soup Group" which I vaguely remembered was a fable.

Sure enough, Stone Soup is on Wikipedia and C2 Wiki and it's often used as a metaphor for some aspects of open source software. Which raises the question of deception:

The villagers deceive the soldiers (OK - they lie to them), and that's bad. The soldiers deceive the villagers (tricking them to bring out the hidden food) and everyone benefits - the soldiers get to eat, but so do the villagers (in the version I know, the villagers hide their food not just from the soldiers but also from themselves). So, at a deeper level, the story raises questions of morality. When is deception acceptable? What gives anyone the authority to claim to have 'the big picture?'

I doubt that the villagers are hiding a shared collection of food. If they were, when each of them were getting items for the soup you'd think they'd see other villagers getting other items and discover what the visitors were up to. If the villagers are individually hoarding the food then it makes more sense that they are unaware that by working together they could produce a meal. In this version, the villages aren't being deceptive and the visitors are allowing them to work together to produce something they couldn't otherwise.

To Coin a Word

Tony makes Reddit again with a familiar term, Refunctoring.

Why ‘Refunctoring’? First, I refuse to reuse euphemistic terms that have served only to denigrate the standard of the software development industry, therefore, I cannot use ‘Refactoring’ (oooh! did I just say that!!?). To show this, consider the fact that the entire notion of ‘Refactoring’ is done away with one simple phrase ‘Functional Programming’ in the case shown and hundreds more. Of course, you might decide to redefine Refactoring to a different context, but then, redefining terms often leads to confusion (the term ‘function’ is itself a perfect example). To further demonstrate this point, open the book or catalog on Refactoring and turn to the section titled, ‘Replace Parameter with Method’. Then say out loud, ‘partial application’ — if you don’t know why you’re saying that, learn what partial application is, then perhaps it might dawn on you.

Anyway, Refunctoring is what most people really are doing. They are converting their code that is written in not-too-powerful programming languages to mimic the power that is already available in functional programming languages. They are representing higher-order functions with (importantly co-variant) polymorphic type parameters, refactoring out what partial application already provides and much more. This is being done, despite having been invented many, many years ago, but the fact that it is being done mandates a name, albeit how absurd it might be. I use the term Refunctoring, for no other reason than it was suggested to me by a bemused colleague (you know who you are :) ) one day and upon reflection, I have decided that it is entirely appropriate.

You don’t think the map function is a special case do you?

It's good to be known as bemused.

All Annotations

JSR-311: a Java API for RESTful Web Services?

I am still not so sure what they want to do exactly, but it seems to be based on JRA which has annotations of the form:

@Get
@HttpResource(location="/customers")
public List getCustomers();

For me a more interesting JSR would be one that would standardise the @rdf annotations the way so(m)mer is doing it.

The RESTlet framework was created because:

"I noted the lack of a REST framework in Java. The only project that came close to it was 1060 NetKernel developed by 1060 Research but it had too many features for my needs and I found that the mapping of REST concepts was not as direct as I was expecting."

It seems that annotations are an exceedingly useful tool, especially having used Elmo's annotations of interfaces to generate beans from RDF (link to PDF documentation). Annotating interfaces rather than classes, seems to be the way to go, given that interfaces can represent relationships like multiple inheritance. Elmo was inspired by subject oriented programming that provides a way "to group common behaviours and separate these different roles into unique interfaces and reusable classes." It's especially useful for unplanned extensions (through composition).

Googling for the various projects brought up, Tripresso looking at mapping RDF to OO (including Java, Ruby, Python and PHP projects). If object to relational mapping is the Vietnam of computer science, I wonder what object to RDF mapping is.

See the previous discussion, RDF on Rails ( mentioning ActiveRDF, RdfReactor, etc) and Henry Story's original, Java Annotations & the Semantic Web.

Behind Blue Links

• The other DL, defeasible logic.
• SableVM: A Research Framework for the Efficient Execution of Java Bytecode "...using our bidirectional layout, a gc only accesses the following memory locations while tracing: reference fields and lock word, for all instances (objects and arrays), and at most three additional accesses for objects with many fields..."
• Speaking of things Sable, Martin Fowler on SableCC "So far I'm not convinced by the approach of removing parser actions and automatically generating a parse tree. Since it's a parse tree, you have to walk it to do anything useful...One plus in tree walking is that if I'm making changes to the tree walker I don't need to re-generate - which keeps me in IntelliJ."
• Evolution rapidly evolving, Horizontal Gene Transfer Adds To Complexity, Speed Of Evolution, "...the speed of evolution has increased over time because bacteria and viruses constantly exchange transposable chunks of DNA between species...", via "The Coming Revolution in Biology".
• And the song to which or rather who the title alludes to.

The Death of Applets and AJAX

Hybridizing Java

So here’s my question. Allow for a moment the possibility that, after 10 years, Java is not going to take over the world of RIAs. Further allow that Ajax is just “how JavaScript was supposed to work in the first place,” but that the limitations imposed by browsers, HTML and CSS committees seem unlikely to let it expand beyond its current bounds. What are we going to use to build RIAs?

And the answer is?

The only obvious solution is Flash. Flash has always been all about cross-platform multimedia experiences and user interfaces. People are very familiar and comfortable with Flash, and it is installed on almost all machines in the world. It’s trusted, stable and reliable.

Do we just forget every 10 years what's going on, does anyone remember Flash Generator from 1998? And I still remember the horrible, inconsistent Flash UIs from that era. Has anything changed with Flex?

Via, Bruce Eckel: Use Flex as the UI for Java Applications

Why I May Never be Good at Scripting

In an effort to highlight my extreme ignorance I'll point out a recent bug I raised in the excellent Mochikit framework. In the end I came to the conclusion that Javascript libraries, while they can handle a great deal of incompatibilities between browsers, can't do it all.

I wrote the following Javascript:listMin([10,9,8,7,]);

This finds the minimum value in the list. In IE the result is undefined, in Firefox it is 7 and in Safari it's a syntax error. As I've found out, an array in Javascript as written above does a different thing in Firefox (an array containing 10, 9, 8, 7), IE (an array containing 10, 9, 8, 7, null) and Safari (syntax error). Checking this code against the specification (using something like JSLint) shows that the above code is really a syntax error (the extra comma after 7).

Part of the problem is that when comparing values in Mochikit a null or undefined is less than any other value. My expectation is either an exception or trying to make all browsers work the same way (not possible in Safari of course).

Now the result of these arrays were being passed around, in a functional way (which is part of why I liked Mochikit), and a more typical example is the second one given in the bug report:
map(function(x) {
if (!isUndefined(x)) return x[0];
}, [[10,10],[9,9],[8,8],[7,7],]).length;

Again, IE is creating an extra null object and returns 5 and it returns 4 in Firefox. The actual code using this was getting an exception in IE but not in Firefox because it was trying to treat a null object like an Array.

This is perhaps an example too, where functional passing style was more difficult to debug than if it was in a loop. But then you'd expect arrays to work the same (or at least I did) and that types are maintained (null not being an Array).

Now I'm generating code and allowing commas after each element in a list is a pretty familiar concept in heaps of languages. I can see why Firefox allowed this behaviour. I can also understand Safari following the specification. IE's behaviour though, doesn't make sense to me at all. It seems to have been purposefully created to cause problems.

All of this conspired to take many hours trying to debug code from a very lower level in several libraries, all the way back up to the highest where the arrays were being created.

A good API prevents these kinds of things from being processed - fail fast. I like Safari's behaviour the most - it would've saved me the most time (of course I was on a Windows machine so no help there).

Compliance Pays

Washington's $8 Billion Shadow "Another failed effort involves the F.B.I., which paid SAIC $124 million to bring the bureau, whose computer systems are among the most primitive in American law enforcement, into at least the late 20th century. The lack of information-sharing is one reason why the F.B.I. failed to realize that in the year leading up to 9/11 two of the future hijackers—including one with known "jihadist connections"—were actually living in the San Diego home of an F.B.I. informant. SAIC set to work on a system called the Virtual Case File. V.C.F. was supposed to become a central repository of data (wiretap transcripts, criminal records, financial transactions) from which all F.B.I. agents could draw. Three years and a million lines of garbled computer code later, V.C.F. has been written off by a global publication for technology professionals as "the most highly publicized software failure in history." The failure was due in part to the bureau's ever shifting directives, which points up the perverse nature of government-by-contract. When the government makes unrealistic demands, the contractors go along anyway: they are being paid not to resist but to comply."

Relational SPARQL?

ARQ : what next? "What next? This release matches the published DAWG working draft but the working group is considering a SPARQL algebra the formalization of the semantic of SPARQL to be based on some like the relational algebra."

Within Specification

The definition of quality is that it meets a certain specification. What a bunch of bollocks! I have a Holden Vectra (a rebadged Opel which people say is a very bad brand in its own country, Germany) - it requires more oil (about 1 litre) every 3,000-4,000 kilometer ever since I've bought it. It has only done 30,000 kilometers in 3 years but in that time it's run out of oil 4 times (between services). Now I ring up Holden support and say that this is an unacceptable specification. They say the car is fine, it is within the oil consumption specification.

Quality is excellence and that's not Holden.

Update: So here's a bit more, based on what Matthew wrote in the comments.

So the quality of the specification comes out of a process. And if that process doesn't have "the quality of excellence" in it, at all levels, then you're not achieving quality yet.

One of these aspects to this quality is a sort of self similarity - like well designed code - each bit of code does a small bit and does it in an understandable way and is roughly the same as the other bits - independent of which level of the system you're at.

If you can't raise an "IT Strategy Defect" then your organizational processes (which your lower processes are a part of) lack that aspect of quality. Now maybe that has to come out of high costs, failed projects or whatever.

From the perspective of the software industry, it seems that we've got a much higher expectation on what can be done. There aren't that many industries where the customer, project manager, support, maintenance, engineer, tester, etc. are even able to say (let alone expected or encouraged) "there's something wrong, fix it".

The guy at Holden support says, "I wouldn't buy a car with those specs" and so on - they know it sucks but they can't change it.

Now that isn't to say software should produce the equivalent of bubble domed cars that play "La Cucaracha". I'd like a flying car but at the moment I'd settle for something that doesn't need to be checked every 3,000 km and isn't going to destroy the planet.

Need for Speed

Bosworth On Why AJAX Failed, Then Succeeded links to which isn't the first time he has examined AJAX's success and failures or why other technologies fail such as RDF. Some commentary was interesting in the comments and a few blogs, including this summary:

All in all the speaker (intentionally) kept his talk simple and stupid but here are some points I think are worth noting:

• Products fail because of [lack of] human psychology not poor engineering
• Top 3 keypoints of UI:
1. Speed
2. Speed
3. Speed
• Your app should be a tool, it shouldn't leave the user feeling like one
• Search isn't perfect but it is far better than any human can do so it is tolerable.
• Analyze the frequency of use of feature and implement those first
• If your app takes more than 2 seconds to load - forget about it
• The Motorola RAZR was cool enough so the poor UI (narrow keypad) didnt matter

Semantic Spreadsheet

What happens when you combine a Google spreadsheet of Semantic Web tools and Simile Exhibit you get Sweet Tools (Sem Web).

The only problem seems to be that it only shows the first 10 or all results - it'd be good to have a previous/next button.

SPARQLing AJAX

Dojo Data "As of January 2007, we have five simple datastores, which are included in dojo as example datastore implementations...dojo.data.RdfStore a read-write store that uses SPARQL to talk to RDF data servers including, for example, the Rhizome RDF application server"

From the source of RdfStore: "RdfStore provides a dojo.data Store for querying and updating a server that supports the SPARQL Query Result JSON format. (see http://www.w3.org/TR/rdf-sparql-json-res/). It also maps RDF datatypes to Javascript objects. RdfStore makes following assumptions about the Result JSON: (1) The result always contains 3 bound variables named "s","p", and "o", and each result binding is treated as an RDF statement. (2) When saving changes to the store, the JSON "results" object will also contain a "deleted" key whose value is a list of deleted RDF resources."

Update: OAT: OpenAjax Alliance Compliant Toolkit "Ondrej Zara and his team at Openlink Software have created a Openlink Software JS Toolkit, known as OAT. It is a full-blown JS framework, suitable for developing rich applications with special focus to data access."

"OAT also provides Data Aware controls for the above that include...SVG based RDF Graph Control".

Only works in Opera and Firefow - IE doesn't support SVG.

"OAT is Open Source and GPL’ed over at sourceforge and the team has recently managed to incorporate our OAT data access layer as a module to dojo datastore."

Via Planet RDF.

Patterns in Software - Part 2

Patterns of Software: Tales from the Software Community (freely available in PDF). "Quality Without a Name" spends a lot of time taking Alexander's objective "meaning for beauty, for the aliveness that certain buildings, places, and human activities have" and applying it to software. This chapter also has a fairly good definition of patterns:

Patterns certainly have an appeal to people who wish to design and construct systems because they are a means to capture common sense and are a way to capture abstractions that are not easily captured otherwise.

The second thing (and second paragraph) is a re-definition, for most people in the software industry, on who the users of the software are:

...when you read Alexander, it is clear that a “user” is an inhabitant—someone who lives in the thing constructed. The thing constructed is under constant repair by its inhabitants, and end users of software do not constantly repair the software, though some might want to.

So these are not the regular definitions of quality or user. It's certainly not the TQM definition of quality: the fitness to a standard or set of requirements. It's much closer to the principle of quality in XP and other agile methodologies. Meeting requirements is rejected as the definition of quality because they are often contradictory, he quotes Alexander on his experience with Bay Area Rapid Transit (BART) system:

So it became clear that the free functioning of the system did not purely depend on meeting a set of requirements. It had to do, rather, with the system coming to terms with itself and being in balance with the forces that were generated internal to the system, not in accordance with some arbitrary set of requirements we stated...What bothered me was that the correct analysis of the ticket booth could not be based purely on one’s goals, that there were realities emerging from the center of the system itself and that whether you succeeded or not had to do with whether you created a configuration that was stable with respect to these realities.

He also takes some time using other (somewhat confusing) words such as alive, whole, comfortable, free, exact, egoless, and eternal to further refine what "the quality without a name" is. The purpose for this is clear though, it's to rediscover objective quality or the combination of fact and value.

We in software are not so lucky—all of our artifacts were conceived and constructed firmly in the system of fact separated from value. But there are programs we can look at and about which we say, “no way I’m maintaining that kluge.” And there are other programs about which we can say, “Wow, who wrote this!” So the quality without a name for software must exist.

Some of them I'd consider fairly vague but others have some parallel with XP principles and similar ideas. In order to describe how a system is alive Gabriel uses the metaphor of a fire and a fireplace. The structure of the logs, chimneys and so on is a well thought out system which supports a self-sustaining process that once set in motion reaches a predefined end - a small pile of ashes. And "whole" is a property where something is self consistent. This is what occurs when code is reflected upon and refactored. And while Gabriel doesn't spend much time talking about a system being egoless, this concept should be familiar to most agile practitioners. XP and other methodologies encourage the idea of egoless programming and collective ownership.

All of these concepts and "the quality without a name" all seem to me to be descriptions of development methodologies, practices, principles and values and the particular kind of software that is developed as a result of applying these processes. That is, software with this quality is a result of a software process with the same quality.

He ends this chapter with "some things" (I'd say requirements) for software which possesses the "quality without a name". I'd summarize it as:

• Not written under unrealistic deadlines.
• Modules and abstractions not too large and small enough to understand and remember what they do.
• Code constantly repaired (he'd probably say refactored today).
• A fractal nature of code - looking at the large or the small the code is coherent.

He ends it with this lament:

I wish we had a common body of programs with the quality, because then we could talk about them and understand. As it is, programs are secret and protected, so we rarely see any but those we write ourselves. Imagine a world in which houses were hidden from view. How would Alexander have found the quality with no name?

Since this was written there's been a lot of code made public but I still can't think of any that I've really looked at and appreciated (and in fact I tend to avoid looking at Java code written under the Apache licence for example). Any suggestions?

Good Singletons

Singletons - we’re better off without them

I was first introduced to the singleton pattern as an alternative to global variables (”global variables are bad, use a singleton instead”). But this is one instance where singletons emphatically should not be used. Replacing a global variable with a singleton is just a lazy way of avoiding global variables without avoiding any of the problems inherent in global variables.

He goes onto list the reasons not to use singletons this way which include (in my order of importance): difficult to test, inhibits code reuse, breaking modularity, inadvertent changes occur between accesses and multiple references.

A singleton is responsible for both its behaviour and for ensuring that only one instance of it exists. In other words, it is responsible for two unconnected activities. This should usually be avoided and the unconnected activities should be implemented in different classes.

One View

MacFUSE Release Opens Up File Systems on Mac OS X "Some examples of applications using virtual file systems through FUSE include GmailFS, which allows users to set up a Gmail account as a local disk, and SSHFS, which allows users to interact with files on a remote computer via SSH (Secure Shell).

Singh said Mac OS X users could also use MacFUSE to mount Windows hard drives."

The demo video shows sing RSSFS (view of RSS feeds), ProcFS (view of OS processes), DocsFS (view of Google Docs and Spreadsheet), SpotlightFS (view of Spotlight results) and PicassaFS. The project web site list others.

Know Fear

Irene Khan -- 2006 Sydney Peace Prize (mp3)

It’s December 2001. I am in a hot and dusty Afghan refugee camp in Pakistan. The Taleban have been defeated and there is jubilation among the Afghan refugees. Refugee women are climbing into the buses that will drive them home...These women understand only too well the real horror of war but they also know that peace is much more than merely an end to fighting. Sitting on the bus next to Zubaida I ask, “What will you do when you return home?” She does not hesitate for a second. Clutching her baby close, she looks me straight in the eye, and says “I want to go to school. Some day I will be a scientist.”

What an amazing answer! Here is this woman discreetly covered from head to foot in a blue burqah, but there is nothing hidden about her message. She is telling me that peace is not a matter of military victories; it is about equality, justice and freedom for women as well as men. It is about creating the possibility for every human being to reach their full potential. And it is about hope.

Fast forward two years to July 2003. I am in Kabul now but I can’t find Zubaida or, for that matter, any woman studying science. Instead I find a fortress town guarded by American troops: a country caught in the grip of warlords and drug barons, torn by insecurity, afflicted by extreme poverty. I sense the fear in women activists as they tell me of the abduction of young girls from homes and schools, and of rampant sexual violence.

Later I am taken to a prison in Kabul, crowded with women and girls accused of adultery, or of wanting to marry the man of their choice or of running away from brutal husbands.

I am telling you this story about Afghanistan because what I saw in Kabul is, in a microcosm, what I see happening across our world today; a world in which peace is being redefined, in the interests of the powerful and the privileged, at the expense of the poor and the marginalized.

A new agenda is in the making in which the rules are being rewritten for the greater security of a few, while the actual sources of insecurity that affect the lives of many more are ignored. The “war on terror” dominates while sexual terror is ignored, even though it affects millions of women and girls around the world, in bedrooms, on battlefields, and in workplaces.

Persistent JRDF

After thinking about it many times I've started to integrate a persistent store for JRDF. Now I know Kowari/Mulgara would be the obvious choice but I decided to try something different - Apache Derby.

A recent post on the Sesame developers list about persistent blank node maps got me into action as well as all this talk about XA2 of course (JRDF's modified RIO parser in Kowari/Mulgara uses a persistent StringToLongMap).

One of the reasons to use Derby is because it has an XAResourceManager. Though I don't expect it to scale or be as fast as most stores (even if the table size is supposed to be unlimited).

I couldn't find an easy way to create a DiskHashtable (although the TestDiskHashtable gave some clues). Here's how I managed to get a persistent DiskHashtable going called derbyDB (it may not be quite right of course):
String driverStr = "org.apache.derby.jdbc.EmbeddedDriver";
EmbeddedDriver driver = (EmbeddedDriver) Class.forName(driverStr).newInstance();
final EmbedConnection30 connection = (EmbedConnection30) DriverManager.getConnection(
"jdbc:derby:derbyDB;create=true");
final LanguageConnectionContext languageConnectionContext = connection.getLanguageConnection();
languageConnectionContext.setRunTimeStatisticsMode(true);
TransactionController controller = languageConnectionContext.getTransactionExecute();
ContextService service = ContextService.getFactory();
service.setCurrentContextManager(languageConnectionContext.getContextManager());
DiskHashtable diskHashtable = new DiskHashtable(controller, TEMPLATE, INDEXES, true, true);


I have some ideas that it might be possible to take the relational RDF operations and put them into Derby (or vice-versa). The datatype support would be nice to leverage (especially the XML datatype for example). It's very preliminary at the moment and I may ditch it in the future.

Patterns in Software - Part 1

Patterns of Software: Tales from the Software Community (freely available in PDF format) is something I came across last time I was talking about design patterns.

First off, I think this book is very well written - style, language, structure - it's all good. I wish I could write as well as this guy. Christopher Alexander is in the preface (the guy who is attributed to inventing patterns).

Secondly, I'm going to cover the chapters that I've liked as I get to them. The first is "Abstraction Descant".

In this chapter he introduces the concept of compression which is: "...the characteristic of a piece of text that the meaning of any part of it is “larger” than that particular piece has by itself."

Examples are macros, function or class names. There is a danger though that compression and abstraction can be overused.

"The problem is that people are taught to value abstraction above all else, and object-oriented languages and their philosophy of use emphasizes reuse (compression), which is generally good. However, sometimes the passion for abstraction is so strong that it is used inappropriately — it is forced in the same way as it is with larger, more complex, and typically ad hoc abstractions."

"Another problem with complex abstraction arises from the observation that abstractions are about ignorance...Some complex abstractions, however, contain information about the implementation that is legitimately required, such as its performance, the algorithm, coding tricks, and resource usage—keep in mind that almost all interaction issues are about resource conflicts."

The next point is that software evolves through time, in little pieces by programmers not designers. The danger is the designers use abstractions (usually in ignorance) whereas coders should write code and design it in order to make it more habitable.

"...creating habitable software that can be effectively maintained, recognizing that the reality of software development is piecemeal growth and to plan accordingly, and to understand that the power of object-oriented programming is compression, which carries a terrific requirement for careful use of inheritance—relate to how we use abstraction and how much we use it."

"In programming, if a set of large abstractions does nearly the right thing, it is tempting to use them and to bend the structure of the surrounding program to fit them. This can lead to uninhabitable programs."

"Worse: You can fight this temptation and choose not to use them. This choice also can lead to uninhabitable programs because you will be using parts similar but subtly different from possibly familiar ones. The only way to avoid this is to use small blocks rather than large ones, or to use blocks well-designed and tested by experts."

He also has a comment on teaching programming where we need to "learn from the classics".

"How much time do we spend reading in our ordinary education? And from our reading we gain a foundation for writing...But in programming we just learn the language and solve a bunch of short puzzles. Sort of like writing 50 limericks and then off to write books."

The next point took me a while to digest but I think it's probably the most useful part of the chapter. It is that we rarely spend time creating new control abstractions (like loops) and most of the time creating abstractions of data. He argues that the two should go hand in hand.

"Let’s look at another problem with abstractions: Data and control abstractions are generally best when they are codesigned and this is rarely done anymore. Consider, for example, the Fortran abstractions of arrays and iteration. Arrays are abstractions designed to represent vectors and matrices. Iteration is a control abstraction useful for traversing vectors and arrays. Think, for example, of how easy it is to implement summation over the elements of a vector. This is because arrays and DO loops were codesigned."

"But an interesting thing happened to Lisp in the early 1980s: the use of macros to define control structures became forbidden style. Not only did some organizations outlaw such use of macros, but the cognoscenti began sneering at programmers who used them that way. Procedural abstractions are acceptable, but not control abstractions. The only acceptable control abstractions in Lisp today are function invocation, do loops, while loops, go statements (sort of), non-local exits, and a few mapping operations (such as mapcar in Lisp)."

"Regardless of what you make of this view of data versus control abstraction, it is certainly true that because almost every programming language does not allow any sort of meaningful user-defined control abstractions, there is always a mismatch in abstraction levels between control and data. If there is a good reason for allowing data abstractions, why isn’t that a good reason for allowing control abstractions; and if there is a good reason to disallow control abstractions, why isn’t that a good reason to disallow data abstractions?"

Finally a word on the general use of patterns.

"Common patterns are similar in nature though not detail to the patterns that Christopher Alexander uses in his so-called pattern languages. A pattern language is a language for generating buildings and towns with organic order. Patterns generally specify the components of a portion of a building or a place and how those components are related to other patterns."

More of this is covered in "The Failure of Pattern Languages" but the next part will be "The Quality Without a Name".

Chatting and Cheating Chooks

So I've known for a while after seeing "Cheating Chooks" that: "Chickens are underrated. In fact, they have a complex communication system which includes over 20 different signals, and the males at least are practiced in the art of deceit."

Late last year Chris Evan's reported that chooks have representational language, "This shows that the call triggers other chickens to look for specific information – in this case, whether or not they already know there is food about – and to respond appropriately, researchers claim. This is similar to how human language works, they say.

Such “representational” communication has been demonstrated in some primates before, but never in a bird."

Creating your own Peep Show

Breaking copy protection for entertainment "I bought my wife Peep Show series 3 on DVD for Christmas (for the non-Brits, this is not what you think). It doesn’t play in our desktop computer or her laptop, and most certainly doesn’t play in our DVD player (which is a first-gen PlayStation 2)."

"Dear Channel4 and Macrovision, you’ve just forced me to rip my own DVD in order to watch it. What’s wrong with this picture?"

Spring's ObjectFactory

More fun with Spring scopes "This time I would like to show another interesting application for the custom bean scopes. In this case conversation is bound to the page in web application (or even to each unique set of request parameters for that page). Practical examples include per-page caching of the static data (i.e. for Ajax use), allow page visitors to interact or edit page content together and many others..."

The Springframework reference has another example of using the ObjectFactory, "Knowing who you are". As noted, it doesn't move away from requiring Spring but it is slightly better than being BeanFactoryAware. The limiting factor on this is that if you need to create two types of objects you need two object factories. A generic object factory could do the trick and generics could be used to remove the casting too.

A recent InfoQ article, "Spring 2.0: What's New and Why it Matters", also has a section about the new bean scopes.

Announcing Andrae

lca: Andrae Muys on RDF "On the first day of linux.conf.au, I ran into Andrae Muys. He hacks Java and RDF for clients who want semantic web hackery done. I have to admit that early Semantic Web hype put me off: it sounded too much like 1970s AI hype. Andrae was interesting, though, and completely free of the wide-eyed uncritical enthusiasm that characterized a lot of my early RDF engagement."

"Andrae runs the Mulgara project, a Java RDF store. His goal is to be able to deal with 1E13 statements (aka tuples, facts, assertions) in three years. It'll do 1E9 right now, next stop is 1E11. He refers to this goal as "3 Ts: three trillion triples". A consortium is forming around Mulgara to make this happen: if it coalesces, Andrae will be the coder to make it happen."

"The next version of Mulgara, 1.3, will ship in February and have this relational mapping in it. A quick Google search shows a lot of RDF-relational mappings going on, but the list of other mappings he had impressed me: Lucene, RSS, mbox, ID3."

"I think it's time I looked again at the world of RDF. They may yet be doing interesting things. I said as much to Andrae and he replied, "I am an engineer. In the early days it was scientists and logicians in RDF. Now the engineers have arrived, and we just want it to work and to scale." Bold claim! If you have a favourite RDF package or practice, let me know in the comments."

Andrae also announced the paper presented at linux.conf.au. Among other things it references David Wood's paper presented in 2004 "Scaling the Kowari Metastore" ("Makepeace" is a good search term).

Quality is Free

The business value of software quality "Organizations that develop low-quality software, whether for internal use or for sale, are always looking backward, spending time and money on fixing defects in "finished" products."

"One common misconception about quality is that it can be traded for improved development speed, reduced development budgets, or added functionality. In practice, however, most organizations find the opposite is true. In the long run, improved quality enables teams to deliver more projects on time, at lower cost, with more features. We should realize that Meskimen's law -- "There's never time to do it right, but always time to do it over" -- is a tongue-in-cheek adage for a reason. A development team that continuously ensures quality does it right the first time. By not introducing defects into the system throughout the entire development process, a team eliminates the time and cost required to find and fix those defects later on."

And why don't customers care about quality: "The simple answer is: “Because defective software works.” The reason it works, however, is because software doesn’t wear out, rot, or otherwise deteriorate. Once it is fixed, it will continue to work as long as it is used in precisely the same way."

The Software of Star Wars

Code Reads #7: Parnas's "Star Wars" paper links to the well known paper, "Software Aspects of Strategic Defense Systems", where he quotes a description of what a software developer actually does, "How, then, do we end up with any big programs that work at all? "The answer is simple: Programming is a trial and error craft. People write programs without any expectation that they will be right the first time. They spend at least as much time testing and correcting errors as they spent writing the initial program.""

"With these observations in mind, Parnas casts a cold eye on the SDI project, which aimed to produce working systems that could identify and target incoming enemy missiles in a matter of minutes. The system couldn't be tested under real-world conditions; it would be expected to function effectively even when some of its pieces had been disabled by enemy attack; and it was intended to be foolproof (since, with incoming H-bomb-armed ICBMs, 90 percent wasn't good enough). No such system had ever been built; Parnas maintained that no such system could be built within the next 20 years, using either existing methods or those (like AI or "automatic programming") on the horizon."

Logic is highlighted as the mathematics of programming much like continuous mathematics in electrical and mechanical engineering. A lot of the reasons why SDI was unfeasible was that the requirements gathering - it may seem that the customer is a ruthless, dictatorship, with the finger on total annihalation - but it's rarely true.

There are a couple of important questions that probably have better answers now. Can we provide proofs for code larger than 500 lines (in whatever language)? Is hardware failure considered in these proofs? Is rules based programming more efficient?

I'm glad I found this article, it's good because I vaguely remember hearing that the SDI project was not technically feasible (games like this demonstrate the error of linear scaling) but I had not idea that one of the problems was the development of the software (obvious in retrospect). This is slightly different to the management lessons learnt about the space shuttle but similarly illuminating.

The author of the initial blog entry also interviewed Joel Spolsky in which he mentions the idea of programmer fatigue. His book is available of the 16th of this month, called "Dreaming in Code", it talks about software development and the Chandler project. Maybe setting out to be the OS/360 project of the next generation?

Allchin's Buying a Mac in Context

Allchin's 'Buy a Mac' E-Mail Exposed "Jim Allchin's "I would buy a Mac" statement now has ontext. The e-mail is publicly available."

Allchin says: "I am not sure how the company lost sight of what matters to our customers (both business and home) the most, but in my view we lost our way. I think our teams lost sight of what bug-free means, what resilience means, what full scenarios mean, what security means, what performance means, how important current applications are, and really understanding what the most important problems are customers face are. I see lots of random features and some great vision, but that doesn't translate into great products.

I would buy a Mac today if I was not working at Microsoft."

Ob. iPhone Comment

Apple's Son of Newton "Gibson learned that, like iPod, iPhone will have a non-removal battery. Let me repeat: The battery is fixed. Non-removable battery is a shortcoming If only Ffor a device with only five hours talk time and functions like Web browsing and music listening that sap power. Heck, Cingular sells the BlackJack with a spare battery in the box."

"With Zune, Microsoft adopted a more end-to-end approach of providing all the pieces (albeit, hardware from Toshiba). The Windows Mobile business model is Microsoft software and partner hardware. If Microsoft had considered releasing a Zune branded phone, greater debate inside the company is sure to follow."

So Apple's move seems quite smart compared to Microsoft's Zune and Smartphone combination. A lack of removable battery isn't so bad when you have cars and airlines all providing iPod integration. And reminds me of the lament of Douglas Adams' for a standard power adapter. The iPod dock connector provides a variety of 3.3, 5 and 12 volt pins depending if it's plugged into Firewire or USB. If only firewire had been that standard.

Microsoft isn't all bad, it announced to much fan fair the next version of Office for the Mac.

A Quick Overview of Relational SPARQL

I've put a quick overview of the relational SPARQL operations that I developed in JRDF onto the Google Code Wiki, called Relational SPARQL Operations. This is just a reworking of what I've done in my thesis in hopefully a more digestable way. I hope I'll have time to expand it and the JRDF Wiki generally to include more documentation on things like the other features of JRDF.

Upgrading the Road

When Product Cycles Collide "he auto makers have looked over the fence at the short product cycles of industries like the cellular phone (typical replacement interval 18 months), and are coveting the opportunity to sell someone a new car every two to four years instead of every five to 15. This would attack a trend that's bound to be concerning the auto makers: In 2005, the median age of cars on the road in the United States was almost 9 years, up from 6.5 years in 1990 and 5.1 years in 1969."

I wonder if taking the entire mobile phone model would be good for car makers. They could provide an electric car on a contract and sell access to the required infrastructure. The product cycle probably wouldn't be 18 months but might be less than 9 years (however long those batteries last).

Three Links Plus One

• Major revision (1.01) of the Music Ontology Scooping the goodness inside as I'm currently designing an ontology (online instruments that are net ready). I'm after any well designed ontologies at the moment (and their success in production).


• Unrolling nested queries Ahh the quest for answer closure continues.


• AllegroGraph and TopBraid 1.5 and Calendar Mashup in TopBraid Composer Currently using SWOOP and Protege but TopBraid is impressive (and expensive). "More recently they have started to use their Lisp platform to develop Semantic Web technology solutions. AllegroGraph is one of their Semantic technology flagship products, and they have done some great progress with it in recent months. From what I have seen, AllegroGraph has really good performance and is now (as far as I know) the best professional RDF triple store on the market. They even have a free entry-level version of AllegroGraph, that scales to up to 50 million triples."