Thursday, January 13, 2005

Google Still Doesn't Trust Metadata

Semantic Web Ontologies: What Works and What Doesn't "A friend of mine just asked can I send him all the URLs on the web that have dot-RDF, dot-OWL, and a couple other extensions on them; he couldn't find them all. I looked, and it turns out there's only around 200,000 of them. That's about 0.005% of the web. We've got a ways to go."

A lot of RDF is XML or N3 and ends with dot-XML, dot-N3, dot-RSS, dot-ZIP, dot-GZ, etc. - a lot of this data is going to be fairly invisible.

"The best place where ontologies will work is when you have an oligarchy of consumers who can force the providers to play the game. Something like the auto parts industry, where the auto manufacturers can get together and say, "Everybody who wants to sell to us do this." They can do that because there's only a couple of them. In other industries, if there's one major player, then they don't want to play the game because they don't want everybody else to catch up. And if there's too many minor players, then it's hard for them to get together."

P2P searching shows that you don't need a rich, top level ontology to be able to find songs by Britney Spears, Birtney Speares, or however you spell it. To find a good quality song you need a bit of metadata. To find a specific performance of a song you need even better metadata. This is how it can grow in a bottom up manner. RDF and OWL allow you to grow out. If you don't need to change the code to support new metadata, as your ontology grows, then that is a positive thing for users and developers.

"So there's a problem of spelling correction; there's a problem of transliteration from another alphabet such as Arabic into a Roman alphabet; there's a problem of abbreviations, HP versus Hewlett Packard versus Hewlett-Packard, and so on. And there's a problem with identical names: Michael Jordan the basketball player, the CEO, and the Berkeley professor."

HP/Hewlett Packard/Hewlett-Packard all cluster statistically together. This kind of technology is sufficiently sophisticated - it's not very different from deciding between what's spam and what's not. Or a human can do it or most likely a combination. Being able to tell which Michael Jordan you're talking about is a problem that is solved by metadata.

"What this indicates is, one, we've got a lot of work to do to deal with this kind of thing, but also you can't trust the metadata. You can't trust what people are going to say. In general, search engines have turned away from metadata, and they try to hone in more on what's exactly perceivable to the user."

I can trust my metadata and I might trust yours. People may try and cheat Google's ranking algorithms but won't cheat themselves. Where people care about their own metadata and have to rely on it, it will improve over time.

Wednesday, January 12, 2005

5 for Java 5

Five Reasons to Move to the J2SE 5 Platform "The JVM is now self-configuring and self-tuning on server-class machines. A server-class machine is a machine with two or more CPUs and at least 2 GB of memory. The server-based performance ergonomics kicks in by rightsizing both the memory required and the class of optimizations needed for longer lived applications. This has resulted in an 80 percent improvement on one application server benchmark without changing a line of code or supplying any runtime options!...The Java Community stands behind the improvements in J2SE 5.0. The J2SE 5.0 expert group comprised the following who's who of the Java industry: Apache, Apple, BEA Systems, Borland, Cisco Systems, Fujitsu, Hewlett-Packard, IBM, Macromedia, Nokia, Oracle, SAP, SAS Institute, SavaJe Technologies, Sun Microsystems, John Zukowski, Osvaldo Doederlein, and Juergen Kreileder."

TestNG Article

An updated from Good-bye JUnit, an article from IBM's developerWorks TestNG makes Java unit testing a breeze: "Some peculiar features of JUnit come in for particular criticism from the latter group:

* The need to extend a TestCase class, because the Java language has single inheritance, is very limiting.
* It is impossible to pass parameters to JUnit's test method as well as to setUp() and tearDown() methods.
* The execution model is a bit strange: The test class is reinstantiated every time a test method is executed.
* The management of different suites of tests in complex projects can be very tricky."

"All these features, together with the adoption of Java annotations to define tests, make the whole testing process much more simple and flexible. There are only a few rules that you must obey to write tests; beyond these, you are absolutely free to choose the testing strategy you prefer."

More FTrain

Installing updates "Ftrain is on hiatus for many reasons, all of them good. First, it is on hiatus because I am writing a book...I am also starting a full-time job in February, rather than sitting around the house in my socks. & I'm getting well underway with coding a real, grown-up version of the framework under this site, which I must do in order for the new job. Thus, I'll give away more code for building weird semantic webbish sites soon, under an open-source license, if anyone wants it. For real this time. I swear. I really want to. I won't screw it up. It will be a combination of Apache2 mod_rewrite rules, PHP5, XSLT, MySQL, and the Sesame RDF engine, and you'll need to understand all five technologies to use it. It won't have a content management interface aside from weird XML files, and it won't work at all when you start it up. You'll love that.

AndroMDA 3.0M3

"AndroMDA is a code generation framework that follows the Model Driven Architecture (MDA) paradigm. It takes a UML model from a CASE tool and generates classes and deployable components (J2EE or other), specific for your application architecture. Because of its built-in code generation support for real world target platforms, it has become the mainstream open source MDA tool for generating enterprise applications."

Metafacades " AndroMDA Metafacades are facades that are used to provide access to models loaded by a repository. These "metafacades" shield us from the underlying meta model implementation. Meta models are MOF modules such as UML 1.4, UML 2.0, etc. Metafacades are generated by the andromda-meta-cartridge."

SPARQL4J

Profium, Asemantics and HP Laboratories initiate SPARQL4J open source effort "Profium, Asemantics and HP Laboratories have initiated an open source effort to provide Java programmers with a JDBC driver to access SPARQL-enabled metadata repositories."

SPARQL4J SF project.

A standard driver would be a great thing. Modelling RDF queries in SPARQL doesn't seem accurate using JDBC.

A JDBC driver seems doesn't really seem like the appropriate interface to support - there's a lot of data types in a JDBC driver that just aren't supported in SPARQL and probably won't be. RDF just has a different way of handling datatypes. Also, you have to use things java.sql.ResultSet, java.sql.Connection, java.sql.SQLException etc. Does it make sense to send a SPARQL query and receive an SQLException? Even other APIs built on SQL databases like Hibernate's Query Language and JDO's Query Language (or Using JDO 2.0: JDOQL) use their own APIs. XML databases have XMLDB.

Maybe having a look at some of the JDBC 4.0 which has things like XML datatype support but it's still "working with relational SQL data stores from the Java platform".

Tuesday, January 11, 2005

MusiK documentation

MusiK is Kowari's demo application. It does seem to have problems with some Mp3 metadata in iTunes but the supplied Mp3 in the data directory works. The UI is still a little rough (you have to scroll up the panel divider in OS X, for example). The point though is to show how to write an application using Kowari and the speed at querying, loading, etc.

MusiK (Music Player for Kowari).

Kowari now uses JID3 - A Java ID3 Class Library Implementation (in CVS) and it seems to do a better job of not dying when parsing the ID3 tags.

2 for J2EE

Improve the quality of your J2EE-based projects "Enter the Eclipse IDE, which provides built-in capabilities that, when used with several plug-ins, can aid in increasing the quality of both the codebase and the system. Eclipse is an open, extensible IDE built for anything and nothing in particular. Eclipse’s Java development environment is open source, free, and fully customizable. Eclipse both enables and promotes the addition of new capabilities via open source and commercially available custom-built plug-ins. By utilizing Eclipse...it is possible for a developer, and a team, to measure the quality of any J2EE- or Java-based system."

Includes: Checkstyle, Jupiter, Metrics, XDoclet, WSVT, JUnit, GroboCodeCoverage, Eclipse Profiler.

A non-free tool LISA smiles on J2EE app testers.

Monday, January 10, 2005

Both!

Jan 06, 2005: Folksonomies? How about Metadata Ecologies? "Neither works especially well on its own: controlled vocabularies often miss out on input from content authors and become rigid, stale, and distant from the vernacular of users; folksonomies will begin to break down...Treating them as major parts of a single metadata ecology might expose a useful symbiosis: encourage authors and users to generate folksonomies, and use those terms as candidates for inclusion in richer, more current controlled vocabularies that can evolve to best support findability."

Sunday, January 09, 2005

Better than Nothing

folksonomies + controlled vocabularies "The advantage of folksonomies isn’t that they’re better than controlled vocabularies, it’s that they’re better than nothing, because controlled vocabularies are not extensible to the majority of cases where tagging is needed. Building, maintaining, and enforcing a controlled vocabulary is, relative to folksonomies, enormously expensive, both in the development time, and in the cost to the user, especailly the amateur user, in using the system."

"The cost of finding your way through 60K photos tagged ‘summer’, when you can use other latent characteristics like ‘who posted it?’ and ‘when did they post it?’, is nothing compared to the cost of trying to design a controlled vocabulary and then force users to apply it evenly and universally."

Folksonomies succeed where the Semantic Web fails "I hazard a guess that most of the Semantic Web crowd is, like me, firmly in the ‘well-designed metadata’ camp. Our vocabularies and ontologies are designed by experts, then handed down to the users. We are not at ease with the idea of users creating their own categorization schemes. If we’ve learned anything from experience, then that the average user is unable to get a subclass realtionship right. A bunch of sloppily assigned tags will not be useful for inferencing."

"Betting on the Semantic Web is betting against ease of use, conceptual simplicity, and maximal user participation. And I don’t see how ontologies and the RDF data model stand even the slightest chance in this particular area."

Via Pro metadata will lose to folksonomy.

Saturday, January 08, 2005

How to Make Code Rule

Why Your Code Sucks "If you don't have tests for your code, it sucks. And I mean comprehensive, fine grained, programmer tests (aka something like unit tests), as well as higher level functional and integration tests. Tests that are automated. Tests that are run routinely. With the programmer tests run after any change to the code...Writing tests before you write the code means that the code is testable by definition."

"Code should be easy to read. Steve McConnell made the statement at his SD West '04 keynote that code should be convienent to read, not convienent to write...Choose style and formating conventions early in the project, and conform to them."

"When making framework decisions, consider if a lighter framework will do the required job. Using something like Hibernate, Prevayler, Spring, PicoContainer, NakedObjects, etc. can be a real win in many situations. Never blindly adopt a heavy framework just because it's the current bandwagon. Likewise, don't blindly adopt a lightweight framework in defiance. Always give due consideration to your choices."

Librarian Action Figure

Capitalising on richer Web data "The project’s second demonstrator involves semantic blogging in the bibliography management area. It was chosen because people working in this field need to share small items of information with a peer group in an easy and timely fashion. “The portal includes peoples’ Web diaries [blogs] with three semantic behaviours – view, navigation and query – built over the base,” says Miller. “If you are in a research reading group, you can call on a number of scripts to generate forms which tell others what the research paper discusses, and so on.” The enriched information generated by blogging and Web technologies makes it easier for computers to retrieve archived data."

The LIBRARIAN ACTION FIGURE "Weapon of Choice:
The Dewey Decimal System...The role of a librarian is to make sense of the world of information. If that's not a qualification for superhero-dom, what is?"

How to Manage a Triple Store

Views in Triple Stores "The first is to apply a "window" on the graph, and only extract the data that's within a certain distance of my origin. The MusicBrainz API has a similar notion of query depth; see the ASCII art under "Select & Get Documentation" section in in the docs, and the subsequent section for more information."

The related-to operations in TKS did some of this but it was unaware of a schema - it used statistics to find the most relevant relationships between resources.

When you start extending from a resource to other related resources you have to know about the schema - more importantly what to exclude or how to rank these relationships. With something like RDFS entailments everything is a resource so following that to other resources means everything is related by one level.

Maybe a walk with multiple predicates and a depth.

"The second subset may be created by filtering out the classes and properties extracted from the database based on their namespaces. For example I might have a triple store containing a mixture of public/private data, with the latter in a separate namespace and I want to pull out just the public aspects for returning from a web service."

"I see the combination of these subsetting techniques, along with a declarative mechanism to state how and when they should be applied as broadly analagous to a relational view.

Has anyone implemented anything like this, or aware of triple stores (or more likely APIs) that offer this facility?"

To get out URIs based on their namespace LIKE must be enabled (which is implemented just not available in iTQL). After that, though, these groups could be put into different models and alternatively viewed as one.

The functionality comes from being able to use set operations in the FROM clause:
"Because models are sets of statements, it is logical to compose them using set operations. The from clause permits set union using the or operator and set intersection using the and operator, with parentheses used to control association.

The following example queries only the statements appearing in all three models.

... from <rmi://mysite.com/server1#model1> and <rmi://mysite.com/server1#model2>
and <rmi://mysite.com/server1#model3> ..."

Semantic Web Experiences

Why You Should Be Looking at Semantic Technologies Now "From publishers to VCs: Look at what—and how—they're enabling solutions and innovation."

"There are at least 20 new vendors. One way to judge the arrival of something is the increase in the number of vendors. Now I have 10 suppliers of triple stores to consider. We have existing companies to look to. What is going on in IBM now? What is going on with HP? Why are they doing all this stuff with RDF? What is happening at Sun with SwoRDFish?"

"At NASA we're working on ontologies for the space shuttle and for new risk management, and now for wire management on the aging wire."

Hmm, the wire management ontology, I didn't see that one. Who said the Semantic Web was going to be merely academic.

RDF Beans

Re: Object triple Mapping "I have implemented a little rdfreactor like library[0][1] over the vacations for use in BlogEd. This allows me to create very simple interfaces using a java beans like pattern that contain all the information about the Ontology."

"This makes development of RDF aware programs in Java incredibly easy. And it also makes it much easier to explain RDF I think to java programmers. With a little more work (or waiting for java 5.0 annotations) it will be very easy to specify the owl ontology completely from an java file, and use those files to generate the
Ontology."

Thursday, January 06, 2005

AudioMan meets Kowari

AudioMan and Kowari "The only problem is the ID3 parser implementation, I guess it chokes on my iTunes-tagged files. I’m not aware of any Java ID3 lib that gets this right, let’s hope Ryan’s jid3rL can solve this."

"About AudioMan(ager), I’m considering if I should use Kowari. I always liked the idea of storing the AudioManager data as triples, but Kowari looks big and complex to me, at least for the moment. However, Kowari would nicely fit into Durham. I’d imagine the existing MP3 content handler could be modified to use the jid3rL parser, and additional handlers be created for XPSF, M3U, Ogg, etc.

For now AudioManager will stick with the Jena API (as I was already using it and Kowari provides some Jena interfaces anyway), and I’ll get working on MusicBrainz integration."

Quick Kowari Update

The most up-to-date (post Pre-Release 2) of the Kowari code is now available from CVS.

A second Kowari mailing list has been added for Kowari developers (I use that term generally not just code but high-level design, future direction and such).

Still looking for work.

Tuesday, January 04, 2005

Extracting Structure

Semantic Web vision is missing filters for unstructured data "I'd agree that the TBL's Semantic Web vision is missing filters for unstructured data, but I don't have the solution and I also don't have the answer to Adam's Where Have all the databases gone?"

Semantic Search Technology "As Artificial Intelligence (AI) technologies become more powerful, it is reasonable to ask for better search capabilities which can truly respond to detailed requests. This is the intent of semantic-based search engines and semantic-based search agents. A semantic search engine seeks to find documents that have similar ?concepts? not just similar ?words.? In order for the Web to become a semantic network, it must provide more meaningful meta-data about its content, through the use of Resource Description Framework (RDF - http://www.w3.org/RDF/) and Web Ontology Language (called OWL - http://www.w3.org/2004/OWL/ ) tags which will help to form the Web into a semantic network. In a semantic network, the meaning of content is better represented and logical connections are formed between related information."

"One short term approach is to garner semantic information from existing Web pages using LSI."

Semiotic Development

Application Semiotics Engineering Process: Towards ontology-based modelling of application semantics "The ASEP [application semiotics engineering process] is intended for modeling complex business rules, application logic and domain knowledge which need either encapsulated for change or separated from conventional software modelling functional dimensions of IT systems for different development or asset management. It targets systems with rich application semantics, such as knowledge systems, system integration with divergent and rapidly changing business logic, semantic interface specification of software components or web services, protocols for semantic interopation of collaborative processes or systems. The ASEP is aimed at the development of corporate or organizational intelligent systems and open services such as knowledge manage systems, semantic web services [18]."

"Lexons represent binary relationship between two entities. They are the vocabulary (not terminology) of the application semiotics. Similar to the vocabulary of the natural language, they have ideational purport without reference to specific application or task contexts...Thus underspecified, they serve as basis for consensus, agreement, reusability and versatility."

"While the lexons underpins the flexibility and reusability of the application semiotics with under-specification, the commitment is essentially dedicated to the semantically well-formed, fully specified, consistent actualizations of the underlying patterns with respect to a particular task or application."

"The layered model of application semiotics is important for encapsulating changes and dynamics of models. For example, the continued business process improvement or integration can be catered to by optimisation (different commitment constraints), resutructuring (changed in commitment networks), or innovation (new business concepts with additions to lexons).

From STAR Lab publications.

Aduna

Aduna Metadata Server 2005.1 RC1 Release "The Aduna Metadata Server automatically extracts text and metadata from information sources, like file servers, intranets or public web sites. The extracted information is available for tools such as Aduna AutoFocus and Aduna Spectacle. These tools enable the user to find and explore information by using the extracted metadata. Other tools can also make use of the extracted information by use of the Sesame library (see www.openrdf.org)."

I see a lot of similarities with what we did at Tucana. The metadata server is basically TMex - which then became the content handlers and resolvers in Kowari 1.1 (without the document workflow). They also have the commercial visualization add-ons.

Sunday, January 02, 2005

Looking forwards, looking back

2004: Retrospective "2004 is coming to an end, so I decided to jot down some of my more memorable moments, both good and bad...Served my first full year as CTO of webMethods...Thought a lot about the Semantic Web and Model Driven Design."

2005: Resolutions "I take New Year's resolutions pretty seriously, as it's depressing to make promises to yourself that you don't keep. That being said, here are some of my resolutions for 2005...Evangelize the Semantic Web."

Update: Also has Predications which include the Semantic Web overtaking Web Services in mindshare.

A couple of other 2005 predications:
* My predictions for IT in year 2005 "The Semantic Web will still be mostly at the same point it is now, at the end of 2005. That is, some nice ideas, including RDF and XML will stick around and find some uses, but OWL won’t take off."
* Yet Another 2005 Prediction List "XML Query, the Semantic Web, and WS-* will continue to hold promise. This is the polite way of saying that none of the above will have an explosive burst of adoption in 2005."
* Search 2005 "Metadata-enhanced search. Will be ad hoc and pragmatic, pulling useful bits from private sources and people following officious Semantic Web and lowercase semantic web practices."

Lock Free Programming

Java theory and practice: Going atomic "Until JDK 5.0, it was not possible to write wait-free, lock-free algorithms in the Java language without using native code. With the addition of the atomic variables classes in the java.util.concurrent.atomic package, that has changed. The atomic variable classes all expose a compare-and-set primitive (similar to compare-and-swap), which is implemented using the fastest native construct available on the platform (compare-and-swap, load linked/store conditional, or, in the worst case, spin locks). Nine flavors of atomic variables are provided in the java.util.concurrent.atomic package (AtomicInteger; AtomicLong; AtomicReference; AtomicBoolean; array forms of atomic integer; long; reference; and atomic marked reference and stamped reference classes, which atomically update a pair of values)."

See also, More flexible, scalable locking in JDK 5.0, Atomic Javadoc and an interesting article The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software.

And Java 1.5 Update 1 is out too.

Memory Bank

Break "Another frustration is that Tucana was the best job I'd had in years, which is why I stayed so long (5 years, including 4 months when I took unpaid leave to go contracting when Tucana ran out of money). We all know quite a lot about RDF and OWL now, and I think we've made something pretty good in Kowari and TKS, but that expertise is all about to go to waste. While some people in Australia are looking at RDF, there are no commercial organisations in Brisbane who have moved that far ahead yet. As a result, everyone will end up working in completely unrelated areas.

From my perspective, I was enjoying learning how to infer for OWL in Kowari. Since my Masters relies upon this I'll be continuing to work on Kowari in my own time. I'll just have to be disciplined as I won't get the opportunity to work on Kowari in my work time."

I haven't yet sat down and listed all the ideas we've had over the last 3 or so years working on this stuff. From what I can tell we've got heaps of ideas (good/ordinary/bad) that need to be written out and explained. Wonder GUI, handling blank nodes in a multi-user distributed database, naming graphs and so on. The recent talk about agile databases is something that one of our very first customers started using TKS for and something that Ben and others talked about ages ago. At the very least, we shouldn't let the ideas disappear with the company.

Meme with a View

Word on the street is that there's a hot new consultancy business called Software Memetics. From their resource page looks like they're pushing something called the Semantic Web and Kowari. It also includes links to Urchin-Kowari write-up (the demo shows Lucene integration) and Kowari: A Platform for Semantic Web Storage and Analysis (paper submitted for WWW2005).

The Winning Web

Why we will win "But do you think, being OS independent is enough? Are you content with having your programs run everywhere? If so, fine. But you shouldn't be. You should ask for more. You also want to be independent of applications! Take back your data. Data wants to be free, not locked inside an application. After you have written your text in Word, you want to be able to work with it in your Latex typesetter. After getting contact information via a Bluetooth connection to your mobile phone, you want to be able to send an eMail to the contact from your web mail account."

"Using a common data model with well defined semantics and solving tons of interoperability questions (Charset, syntax, file transfer) and being able to declare semantic mappings with ontologies - just try to imagine that! Applications being aware of each other, speaking a common language - but without standard bodies discussing it for years, defining it statically, unmoving.

There is a common theme in the IT history towards more freedom. I don't mean free like in free speech, I mean free like in free will.

That's why we will win."

Stuff

* Predictions for 2005 "Spring, Hibernate and lightweight framework backlashes occur...java.util.concurrent will be a talking point and will produce a rash of articles and guidance as developers weep like children in the face of Doug Lea's 3rd edition of CPIJ....Paul Graham declares victory and ships Arc, which end up being Common Lisp without the libraries."
* Predictions for 2005, Yet Another 2005 Prediction List and Microsoft's Top 10 Milestones for 2005.
* What's Out and In for 2005.
* Stapless Stapler.
* Java in MySQL
* tis the season: II "Code and Other Laws of Cyberspace" goes Wiki for version two.
* Translating Semantic Web languages into SCL "The three SWeb languages so far given W3C 'recommendation' status (RDF [RDF], RDFS [RDFS] and OWL[Webont]) can all be translated straightforwardly into SCL. This document gives full translations, showing how to express all of the content of these languages directly in SCL."
* Optional match with iTQL.

Agile Databases Again

Let them eat layer cake: flexibility versus clarity in data "Recently I've had some experience at the non-planetary scale of trading off between the extreme flexibility of technology like RDF versus a domain model that a person coming after could reasonably be expected to understand...It turns out that RDF is surprisingly cheap stuff to generate. The downside was that for purposes of communicating intent, ongoing maintenance and adding functionality against the collected data, RDF is not very pleasant to work with, at least not compared to SQL, Objects or XML. This is especially so at the presentation layer. It's also a different paradigm, and by using it you're technologically committed to yet another data model, directed graphs, along the usual suspects - objects, markup and relations. The cost of introducing a new model should not be underestimated. As a result RDF has been useful but not as cheap to manipulate as one would like."

Also, "This is another area where RDF falls down. Yes, there is Sparql and before that other SQL like languages, but again you're left iterating over raw RDF graph result sets, which is not always ideal."

As linked to previously, SPARQL results can be either a sub-graph or variable bindings.

"Arguably we could have used an RDF store such as Kowari, the in built persistence mappings of Jena, or even XQuery, along with the RDF interchange. The reality is there's only so much new technology you can apply in one go without taking on too much risk, especially in a short time frame, whereas we had a good idea of what we were getting into with a relational store."

Are triplestores good databases? " The Sparql language may not quite be finished but it’s certainly comparable to SQL...Overall this would suggest that RDF stores are potentially good DBs, on many points potentially much better than regular RDBMSs (or XML DBs) because of the more flexible model. But for this to be practicable it assumes the performance can be brought to a comparable level as RDBMSs, which if it hasn’t already been done would I think would only be a small matter of programming."

I've been fairly negative about SPARQL in the past because it's lack of counting and sorting and it also doesn't make sense to me to have DISTINCT - everything should be distinct, it's a graph.

There's also some comments about Kowari using Lucene for the database and not having transactions which is just wrong. Kowari uses NIO and AVL Trees for its store and has since the beginning. It's much faster than JDBM (which we looked at to store our blank node map and discarded because of speed).

Saturday, January 01, 2005

Java and Schemas

A recent posting to the Kowari, Sesame and other RDF lists about a Hibernate/JDO like tool for RDF. This lead to a link to RDFReactor.

As mentioned by Re: Object triple Mapping there are some tricks to modelling objects in RDF like multiple inheritance. My preference has been ontology based programming where the developer programs to a general ontology rather than trying to concretely to tie either RDF or Java to each other.

Crisis

In the same week Tucana goes under we get probably the busiest week on the Kowari mailing list and 45 downloads for Pre-release 2. Considering three quarters of these is the 50MB version, there's either a lot of good bandwidth out there or great patience.

Before the holidays there was great progress being made. We have been attempting to make the "10,000 triple/second challenge" and had succeeded (at least the first 1 million triples, it degraded to a few thousand a second after 200 million). This is on the same 1.6 Mhz Opteron system that we use for all our tests.

Andrae was working on a large refactoring of the transaction operations so that any JRDF, Jena, or iTQL (or anything new) could be put within a transaction or use an existing one. They were already in a transaction but they couldn't tell whether they already had the write phase for example. From what I can remember Simon was working on the problems associated with using file and HTTP protocols in the FROM (external resolvers). David M was working on speeding up deleting triples as well as loading speed. Paul had been working on SOFA and OWL. Robert had started looking at JDO, EJB and mapping relational databases to RDF.

Our commercial focus had been on speed and scalablilty. With Tucana gone some of this focus is probably going to change. It takes a lot of time, effort and resources to keep everything changing and to continue to perform at a commercial level. Between 1.0 and 1.1 there practically isn't a portion of code that wasn't modified - more often than not it was completely changed. This kind of development, on multiple fronts, is something that probably can't occur in the future.

So the future is probably smaller, simpler, with an eye on standards compliance. I hope some things like multiple writes, phase holding, pluggable datatypes, inferencing Hotspot, SPARQL support etc. will be developed but I doubt many of these things will see the light of day now. I'm not discounting them entirely, but there are many things that had to occur in parallel that need a team of developers. A lot of the new features depended on the multiple writers feature. Multiple writers is not an easy feature to describe but suffice to say it's more like Lucene than a normal relational database.

So I think that means improving RDF, RDFS and OWL support. We made some good improvements in Pre-Release 2 with better datatype support. David M added the functionality to allow literal and URI prefix matching - someone just has to add it into the query layer. Combining this with matching nodes based on type (literal, URI or bnode) and trans/walk queries and they're a very powerful combination of features. In the background there's also been a focus on languages (I know we were talking about 3066bis support). Paul and I were looking at inference models (seems to have similarities with pseudo models and datalog vs tableau). Paul's Masters will probably push some of this into reality.

So all in all there does seem to be some good opportunities in the future. I know that Tucana had customers who were paying lots of money for TKS and I know that Kowari did help in the risk assessment. So my intentions, at least in the short term, is to see that good support occurs, bugs get fixed and the like. Although there isn't anyone left to do TKS releases.

I don't really understand why this happened (or what's going on) but to quote Marge Simpson: "One person can make a difference. But most of the time they probably shouldn't."

Friday, December 24, 2004

Happy Festivus

This should really annoy the people trying to keep the true meaning of Christmas.

'Seinfeld' Festivus display vies with Nativity

"When a Florida church group put a Nativity scene on public property, officials warned it might open the door to other religious -- and not-so-religious -- displays. They were right.

Since the Nativity was erected in Polk County, displays have gone up honoring Zoroastrianism and the fake holiday Festivus, featured on the TV sitcom "Seinfeld.""

It's real if it has a Wikipedia entry: http://en.wikipedia.org/wiki/Festivus oh and a NY Times article "Fooey to the World: Festivus Is Come". Danny has a very Britney Christmas (still the number one search term).

Thursday, December 23, 2004

Visual Browser

Visual Browser is a Java application that can visualise the data in RDF scheme. The main principle of the visualisation is that:

* the triple (resource, resource, resource) is represented by two nodes connected by an edge
* the triple (resource, resource, literal) is represented by a hint (small window appearing on mouse over the subject node)

Visual Browser uses the Jena framework to obtain the data, since the RDF scheme can be saved in different forms (a single XML file or a relational database).

The visualisation engine is derived from TouchGraph LLC."

Only for Java 1.5.

Wednesday, December 22, 2004

Pre-release 2 Available

See the Kowari project page. Includes a sample application which extracts metadata using resolvers (mp3s), an example of an AudioManager.

Gridbag

IBM Reflexive User Interface Builder "The IBM Reflexive User Interface Builder is an application that constructs and renders graphical user interfaces (GUIs) for Java Swing and Eclipse Standard Widget Toolkit (SWT) based upon a descriptive XML document. (Java Swing is a rich GUI toolkit included with Java that provides operating system-independent GUI components. Eclipse SWT is an add-on GUI toolkit that takes advantage of host operating system GUI components for maximum host integration.) IBM Reflexive User Interface Builder is both a specification for a mark-up language in which to describe GUIs and an engine for creating (and, if desired, rendering) them. This application can be used as a stand-alone application for testing and evaluating basic GUI layout and functionality, or it can be used as a library within the context of a Java application for creating and rendering GUIs for that application."

So you never have to go Totally Gridbag (via Tom).

Flink

"Flink is a visualization of the social networks of the Semantic Web
community. Information about researchers and their relationships is
extracted from the Web, FOAF profiles, emails and publications. Flink
itself uses Semantic Web technology to represent, store and reason with
metadata. For more information, please see the About section of the
website."

Results of a Query

SPARQL Variable Binding Results XML Format "This document describes an XML format for the variable binding results format provided by the SPARQL query language for RDF, developed by the W3C RDF Data Access Working Group (DAWG), part of the Semantic Web Activity as described in the activity statement"

Tuesday, December 21, 2004

Just Ask for Help

Came across Yet Another RDF Store: Perfect Index Structures for Storing Semantic Web Data With Contexts "We tried to install Kowari, but failed to get a running version. In one of our installations, inserting a 1 MB N-triples file via the Jena interface resulted in 30 minutes processing time before the process threw an exception because of a full disk (there were 200MB disk space available before starting the Kowari server). On another installation, we got a core dump of the JVM when running Kowari. Therefore we concentrated in our efforts on Sesame 1.1RC2 and Redland 0.9.18."

This is the first record of anyone having a problem with Kowari in this respect and without any idea of what version of Kowari or Java or the OS or anything else I went and downloaded their example code. Using Kowari 1.0.5 (they mentioned Sesame 1.1RC2 which was released after 1.0.5) with the given code you get an exception as NTriples is not supported as a parameter. NTriples is a subset of N3. So this code doesn't work. However, if the code is modified so that the last lines become:
  model.read(new FileReader(file), file.toURI().toString(), "N3");
model.close();
database.close();

And run the code you get:
...
INFO [main] (AbstractDatabaseSession.java:699) - Loading
file:/.../University1_0.nt into rmi://.../server1#camera
INFO [main] (AbstractDatabaseSession.java:770) - Loaded 7304
statements
...

If people have any problems running Kowari, before they write it up as a paper maybe they should ask someone how to get it running first. Although, this isn't the first time.

Type Safe Enums

Overloading int considered harmful: part "On the other hand, enums do not handle multi-classloader issues automatically. It is absolutely possible to have FontStyle.PLAIN != FontStyle.PLAIN when the FontStyle class is loaded twice by two different class loaders. If this is a real possibility in your code (and if you're writing a library, it's always a real possibility), the obvious solution is to override equals with a method that does work. Unfortunately, you can't do that. The equals method in java.lang.Enum is declared final, and all it does is compare objects for object identity with ==. So you really have no choice but to buckle down, write your own type-safe enum class, and warn client programmers to always use equals instead of == when comparing objects."

Monday, December 20, 2004

Eye on Search

Conquering the Digital Haystack "But clearly it's time to revisit the search engine. Google's IPO didn't end the search wars, it fanned the flames. Few fields are as rife with activity, and a slew of start-ups are angling for position. Some claim new and better technology than the PageRank algorithm made famous by Google. Others seek merely to be different -- filling voids left by the big players. And though the technologies, in most cases, are brand-new and untested, they promise to change the way consumers search the Web -- and the way advertisers reach those consumers. A look at three of the hottest search start-ups -- all planning services for small businesses by early 2005 -- shows how."

"San Francisco-based Blinkx launched last July and already claims more than a million users. What does Blinkx do differently? Its technology not only matches keywords but also locates related concepts...What's more, Blinkx searches everything -- not just the Web but also the contents of your computer, including e-mail messages and attachments and files on your hard drive, as well as weblogs and digital television content, which are currently ignored by most other search engines."

"Hence Dipsie, which searches based on semantic rules rather than keywords or even concepts. Wiener claims his semantic algorithm can sift through Web information and get you in one click what might take several with a conventional engine -- if it got you there at all. He also says the ability to map concepts will enable him to index some 10 billion webpages, more than double the four billion claimed by Google."

Proximity and MonetDB

"PROXIMITY incorporates major research findings from the Knowledge Discovery Laboratory, including model corrections for statistical biases inherent in relational data such as autocorrelation and degree disparity, as well as our graphical query language. PROXIMITY provides an open-source platform that can be used for both research into relational knowledge discovery and practical applications to real-world data."

"MonetDB achieves this goal using innovations at all layers of a DBMS: a storage model based on vertical fragmentation, a modern CPU-tuned vectorized query execution architecture that often gives MonetDB a more than 10-fold raw speed advantage on the same algorithm over a typical interpreter-based RDBMS. MonetDB is one of the first database systems to focus its query optimization effort on exploiting CPU caches. MonetDB also features automatic and self-tuning indexes, run-time query optimization, a modular software architecture, etc.. In-depth information on the technical innovations in the design and implementation of MonetDB can be found in our digital library."

Proximity is Apache license and MonetDB is MPL.

Another Agile Database User

Re: Non SemWeb uses of RDF "Of course this could also be done using a big relational database. A big benefit of RDF we've found is that you can dump the data together first, and then join it up with heuristics (e.g. switch port has same MAC entry as server NIC etc..). I suspect that this is an order-of-magnitude time saving compared with doing static schema design up front."

StAX

StAX utility classes "The purpose of this project is to help facilitate the adoption of JSR-173: Streaming API for XML (StAX) by providing a set of utility classes that make it easy for developers to integrate StAX into their existing XML processing applications."

Jena 2.2 beta 1

"This release is primarily a maintenance release. The "beta" designation
means that not all work on Jena 2.2 is completed but also it is a
recognition that a pre-release to the growing Jena user community is
needed before a full release in order to smooth the transition between
versions."

"This release was built with Java 1.5.0 with source and target set to
level 1.4. It has been tested on WindowsXP, Linux and MacOS with a
mixture of Java 1.4.1, 1.4.2, 1.5.0."

Some good stuff from the release notes:
* "Reifier::getHiddenTriples() and getReificationTriples() REMOVED and replaced by iterator..."
* "added new API operation Model::removeAll() which removes "all" statements from a model"
* "a new memory-based Graph, SmallGraphMem, has been introduced."

Available from Sourceforge to download.

Also released was NG4J 0.3.

Sunday, December 19, 2004

The Future of Everything

Future of Online Marketing: Semantic Ontology & Social Marketing "Despite these struggles, the Web, for all its flaws and bottlenecks and viruses and spam, is one of the crowning achievements of the last 50 years."

"As for the impact, the Semantic Web will make what we have more useful, and that has been Berners-Lee's mandate all along: "It's all about the people, not the technology," he said at W3C10.

"We made the Web to use the Web, to get other people to use the Web and to get things done," Berners-Lee said. "We are going into a new area ... that could change the rules of Web and enable us to do all that we on this planet can accomplish." "

Intelligent Enterprise also recently published an article, Beyond the World Wide Web.

Great Southern Land

A land of wasted web opportunity "Of the World Wide Web Consortium's (W3C) 368 members, only five are from Australia, says the head of W3C's international offices Ivan Herman, who was in Brisbane last week for a conference at the Distributed Systems Technology Centre (DSTC)."

""I often hear people in Australia say they are too small to do anything, they are happy to let the big guys in the US fight out the standards and they will just use them. I find this attitude strange because there is no reason for (it); the W3C tries to have a structure where any organisation can have their wishes heard," Herman says.

This attitude limits Australia's involvement in the development of the "semantic web", the next big e-commerce wave that promises to unleash a slew of new technologies, where computers on the internet can communicate data effortlessly with each other and therefore provide more useful applications. Whereas HTML was developed for humans to read websites, the semantic web puts computers in touch with each other so they can automatically exchange data, such as stock prices, weather forecasts, bus timetables, plane routes, and sports scores and GPS co-ordinates. It is envisaged that once these services are widely available that new applications and business opportunities will emerge."

It's basically correct, Australia is generally very conservative sprinkled with a bunch of very active people. Brisbane is doing okay, my GP is working on ontologies, libferris is made here, and of course Kowari/TKS. DSTC, UQ and QUT are all doing semantic web work, of course.

Principles

W3C Delivers Web Architecture Overview ""In the architecture document, [TAG participants] emphasize what characteristics of the Web must be preserved when inventing new technology," said Tim Berners-Lee, the Web's creator who now serves as W3C director and co-chair of the TAG." They notice where the current systems don't work well, and, as a result, show weakness. This document is a pithy summary of the wisdom of the community.""

Some of the highlights:
* 2. Identification "Global naming leads to global network effects."
* 2.5. URI Opacity "Agents making use of URIs SHOULD NOT attempt to infer properties of the referenced resource."
* 4.6. Future Directions for Data Formats "Data formats enable the creation of new applications to make use of the information space infrastructure. The Semantic Web is one such application, built on top of RDF [RDFXML]. This document does not discuss the Semantic Web in detail; the TAG expects that future volumes of this document will. See the related TAG issue httpRange-14."
* 5.4. Protocol-based Interoperability "Protocols designed to be resilient in the face of widely varying environments have helped the Web scale and have facilitated communication across multiple trust boundaries. Traditional application programming interfaces (APIs) do not always take these constraints into account, nor should they be required to. One effect of protocol-based design is that the technology shared among agents often lasts longer than the agents themselves."

Wednesday, December 15, 2004

Two Papers

* Web ontology reasoning with logic databases "Put together, the results developed in this thesis allow scalable and practical solutions to ABox-related reasoning problems for OWL-based applications."
* Reverse Engineering Ontology to Conceptual Data Models "One of the main problems facing genomics data integration is related to multiple representations of the data semantics within a set of sources. For example, the same gene may be represented as a genetic map locus in one data source, an aggregation of multiple individual exon entries in another data source or a set of EST sequences in yet another one. In this case, ontologies become an integral part of bioinformatics since they encourage a common vocabulary for describing complex and evolving biological knowledge [17, 18] and can be used as a common access to diverse information repositories."

Testing

* Unit Test Patterns "...two things are needed--a formalization of unit testing by establishing unit test patterns, and the early adoption of object oriented design patterns in the developing application to specifically target the needs of unit testing."
* FIT "Fit allows customers, testers, and programmers to learn what their software should do and what it does do. It automatically compares customers' expectations to actual results."
* SelfEsteem "SelfEsteem is a graphical presentation of Acceptance Test results."
* Marathon Similar to Abbot, JFCUnit, and GUITAR.
* XMLUnit.
* MockEJB "MockEJB is a lightweight framework for running EJBs. MockEJB implements javax.ejb APIs and creates Home and EJBObject implementation classes for your EJBs. Internally, MockEJB relies on dynamic proxies and interceptors...Additionally, MockEJB comes with the "mock" implementation of non-EJB APIs. Currently it provides in-memory JNDI and JMS implementations which can be used independently from MockEJB's EJB support."
* ThoughtWorks OSS.

Tuesday, December 14, 2004

I am the Amazon and the Zip Code

Suggested Google Alphabet All someone has to do now is design the letters like they did at primary school.

Mock Objects

Recently, I was link to by The case for unit testing; Mock databases?; Single-day software development... which points to things such as I changed my mind - Mock objects are wrong for database unit testing and the very good How To Write Unmaintainable Code.

Zoom, Zoom, Zoom

A Structured 2D Graphics Framework " Why use Piccolo? It will allow you to build structured graphical applications without worrying so much about the low level details. The infrastructure provides efficient repainting of the screen, bounds management, event handling and dispatch, picking (determining which visual object the mouse is over), animation, layout, and more. Normally, you would have to write all of this code from scratch. Additionally, if you want to build an application with zooming, that’s built right into the framework too."

Used by Jambalaya.

Waiting for the Schema Fairy

The Semantic WinFS "Instead, people will simply release metadata. Bad metdata. Good metdata. Incompatible metadata. Heck, they're already doing it without Microsoft's filesystem as a catalyst. Rather than a Schema Fairy, we need a standard method for generalized structuring of metadata coupled with a language for semantic translation. The good people of the world can release all the metadata they want, and when critical mass has been reached, somebody will create a translation between the two. If only we had some knight in shining armor to rescue us!

Have you ever heard those Trojan commericals where the Trojan Man rides up on his horse and saves the, um, date? Well, imagine that same commercial, except that the horse is the W3C, the Trojan Man is Sir Tim Berners-Lee, and instead of free prophylactics he's handing out RDF and OWL."


Via On the WinFS delay "If WinFS was only about figuring out how to plug in an object store to the file system, I think it would have a definite ship date by now. The technical issues involved in doing that are tractable, and the PDC build showed that MS is making decent progress on that front. The problem of associating blobs of metadata with blobs of information in the filestore is most likely done (or getting there). However, even with that in place there are larger issues: what does this metadata look like, and where does it come from?"

"Even if the Schema Fairy came down today and blessed us with canonical schemas for everything, there would still be the problem of metadata extraction and annotation. WinFS will only work if it has metadata to process, and right now there aren’t very many good ways of marking up a document with semantic metadata. Manual annotation just doesn’t work – the time cost to the user is too high."

Monday, December 13, 2004

REST of the world

How I explained REST to my wife... "Ryan: Machines don't have a universal noun - that's why they suck. Every programming language, database, or other kind of system has a different way of talking about nouns. That's why the URL is so important. It let's all of these systems tell each other about each other's nouns.

Wife: But when I'm looking at a web page, I don't think of it like that.

Ryan: Nobody does. Except Fielding and handful of other people. That's why machines still suck.

Wife: What about verbs and pronouns and adjectives?

Ryan: Funny you asked because that's another big aspect of REST. Well, verbs are anyway."

Updated Kowari Javadoc

For all your API goodness Kowari Metadata Store 1.1.0 API.

Sunday, December 12, 2004

Another XML Accelerator

I was recently talking about Intel's old NetStructure™ 7210 XML Accelerator a more recent piece of hardware (which is different) DataPower XML Hardware Accelerator "As Rich mentioned in his comment hardware acceleration is a pretty hot ticket at the moment, and for good reason. While some might argue all hardware acceleration does is bring XML applications to the same performance level of binary data-based applications my counter question/remark would be "You mean I can get all the power and benefit that a XML/XSLT-based solution brings AND get the same performance as an application that has been painstakingly handcoded in C or at best C++? WOOHOO!!!""

To create an RDF accelerator maybe you could use a programmable graphics card.

How to Create a Good Web UI

Or should that be Google Web UI: "Looking at the web applications Google is producing, I can comfortably say (and I’m almost never comfortable making technological predictions), that if you develop web applications and you aren’t looking today for ways to include dynamic interface techniques like those made practical by XmlHttpRequest, you’re going to end up losing to someone who is."

Friday, December 10, 2004

Kowari 1.1.0 Pre-Release 1

What's new:
* Resolvers allows developers to create components that expose data sources as RDF.
* Content handlers allow Kowari to extract metadata from different types of files.
* Improved datatype handling for most XSD datatypes, RDF's inbuilt XML Literal datatype. Allows for the storage and querying of unsupported datatypes.
* Improved performance, specifically on small queries and subqueries.
* AbstractDatabaseSession replaced to allow pluggable Session implementations. Jena, JRDF or iTQL are able to be used separately.

See http://www.kowari.org/.

Provenance to Unify Browsing

A Framework for Unified Information Browsing via RDFX "This Unified Information Service Browser (UISB) project is investigating the means of joining heterogeneous information from multiple sources into a combined and navigable structure...Attempting to unify disparate data sources illustrates that hierarchical and relational data models can place undesirable constraints on the stored data; however, the same data can be stored without constraint in a dynamic graph, which also provides significant potential for enhanced traversal and discovery capabilities."

"The solution that was adopted was to include provenance information for every statement that is added to the graph. Thereafter if a graph, or a statement or a resource is removed from the store, it can be done without the risk of losing information. In addition to identifying where a statement originated, the provenance objects can store metadata describing (among other things) when it was discovered, and when it should expire, therefore enabling the automated management of expired information, or potentially providing an event mechanism to instigate the refresh of the data. The benefit of this solution, is that it does not rely on the graph implementation to ensure that graph merging and un-merging happens as expected, however the added overhead of four extra statements for tracking provenance may negatively affect performance and increase the size of the data store. This will depend on the implementation of the graph store since some stores (e.g. Jena 6.4) can reify statements more efficiently."

Bottom Up Tagging

Playing with Taxonomies "I spoke with Stewart Butterfield of Ludicorp, developer of Flickr, about this effort. (Latin scholars note: The Ludi in "Ludicorp" suggests the Latin words for play and game.) Stewart had many insights about this new approach to building taxonomies. "If you can hire enough excellent librarians, you will get better keyword results than with social approaches. However, as the content grows, tagging (and retagging) becomes an order of magnitude more difficult. In other words, social approaches are 80% as good as and 10 times easier than top-down approaches." As to whether Flickr's approach would work in the button-down corporate world, Butterfield had this to say: "Anticipate resistance in the CIO crowd who don't want to risk losing control in a social self-correcting process and do not want anything to get lost." Butterfield says that at least 55% of photos uploaded to Flickr have one or more tags, and 66% have both a tag and user-supplied metadata. As of early September, Flickr had 500,000 photos on its site and was growing at the rate of 15,000 to 20,000 more each day."

Thursday, December 09, 2004

Sometimes You Can't Make It On Your Own

On Folly "The unthinkable rapprochement between topic maps and RDF has occurred, signified by the formation of the W3C RDF/Topic Maps Interoperability Task Force. The task force is part of the Semantic Web Best Practices and Deployment Working Group. The last time I was with the majority of the people listed as members of the task force, it was in a very pleasant drinking establishment in Amsterdam. It's nice to think that the bonhomie of that evening has persisted into forming the basis of the task force."

"...and I think he mischaracterizes Berners-Lee's approach as calling for globally agreed ontologies. Nevertheless, he does note an increase in independent software activity concerning ontologies and the Semantic Web."

"Both Shirky and Udell seem to be pretty much convinced the Semantic Web requires, from the outset, globally agreed ontologies. It seems more that they've set up a straw man. I had always envisaged that in the same way user interface and other conventions have emerged from the messy web, so would ontological conventions. Messy, but good enough."

When different isn't better

Why Coding Standards? "Coding standards are being adopted by more development organizations as a means of ensuring the delivery of reliable and sound applications. Estimates suggest that 80 percent of the total cost of a piece of software go to maintenance, and not enough effort is being directed at ensuring that the quality goes in during the development process.

With the advent of coding standards, companies are saving money and time on the front end, reducing potential safety risks and safeguarding their reputation both internally and externally. Equally, development teams can reduce their time in code reviews, correcting coding standards violations that could be detected—and in some cases corrected—automatically."

IBM CICS J2EE

IBM Extends CICS for Web Services, J2EE "Announced last week, the CICS Transaction Server for z/OS 3.1 provides Web services capabilities, while the CICS Transaction Gateway 6.0 provides J2EE (Java 2 Platform, Enterprise Edition) connectivity. The software enables customers to more easily integrate business processes and extend existing mainframe-based applications with better Web services and integration capabilities."

This was a long time in coming: "IBM Corp. will bring its San Francisco project into line with Sun Microsystems's Enterprise JavaBeans object specification and announce intentions to add Enterprise Java application server functionality to a range of its middleware products, all at the Java strategy day at JavaOne this March in San Francisco...IBM may achieve this by building a separate application server, forming a partnership with EJB server vendors such as Gemstone or WebLogic, or developing an application container that will run inside or on top of current server offerings such as CICS, according to sources."

Metafilter on Metadata

Recently we've all been thinking about flat (or better, faceted) hierarchy web apps that organize email, photos, bookmarks, and general knowledge. The common threads are metadata (tags, categories, labels) that enrich relationships within and hence searchability of large collections. But besides marketroid hype (buzzwords, snark) and a computer that plays Twenty Questions what else can we do and study using faceted data structures: searchable culture references in The Simpsons, library science, computer filesystems, A.I. development, models for human memory and cognition?

FacetedClassification.
From http://www.metafilter.com/mefi/37515.

RDF, the ultimate agile database

Perspective on XML: Be humble, not imperial "The revolt against imperial modeling of code has already taken shape in the form of languages and agile methods. Agile programming emphasizes highly iterative development in close collaboration with the eventual users of the product. Even more important, it stresses the inevitability of change and evolution. In effect, agile developers pride themselves on being able to rapidly accommodate change."

"The same revolution is in the offing for data modeling. There have been some developments in agile databases which, literally, adapt the ideas of agile programming to the design of (usually relational) databases, but there is also progress occurring in semi-structured databases and, in particular, XML."

Is XML Zen in opposition to "strong" data modeling? "One thing I seem to share with so many of my colleagues in the XML world is a wary attitude towards traditional data modeling practices. It's an attitude that has also informed my thinking in related articles pondering data supermodels, coupling of distributed systems, OO encapsulation, and the like.

Some of us see XML as a bit of a refuge from established schools of data modeling. OO and Unified Process in my case, E-R and other relational based modeling in others'. Some just came from document-centric backgrounds where such extremely rationalized data modeling was not the mainstay. In my case, interest in XML was part of a general interest in data modeling as a vehicle for human expression rather than for robotic simulation of the real world."

As I've said before, XML is not relational enough and RDF is relational or rather "RDF provides a relational data model of the Web".

2-4 Players, Ages 10 and above

The Epistomat: Generating Consensus About RDF Ontologies and Rules "What happens when you take an ontology-driven interface generator, and feed it the ontology for OWL itself? A simple recursive technique allows for bootstrapping a world model and programmatically generating interfaces to it. Such a model can then serve as the platform for the evolution of collective consensus about ontologies and statements, by means of a curious game of Nomic."

From the Nomic site:
"Nomic is a game in which changing the rules is a move. In that respect it differs from almost every other game. The primary activity of Nomic is proposing changes in the rules, debating the wisdom of changing them in that way, voting on the changes, deciding what can and cannot be done afterwards, and doing it. Even this core of the game, of course, can be changed."

Protein Database

Constructing ontology-driven protein family databases. "The protein phosphatase resource, PhosphaBase, is freely available over the internet (http://www.bioinf.man.ac.uk/phosphabase). The DAML+OIL ontology for the protein phosphatases and the ABC transporters is available on request from the authors."

Ganesha

I previously mentioned Ganesha but before it was available for download. Well, download it.

Wednesday, December 08, 2004

OO DB goes OS

Object database goes open source "Startup company db4objects this week is releasing its object database, db4o, under an open source format, with the product now available either under the GPL via open source or commercially as embeddable software.

Built for Java and .Net development, db4o enables storage of objects, according to the company. An example of an object could be a vitamin in a biotech application or a brake configuration in an automotive application, according to Christof Wittig, CEO of db4objects."

Their homepage prominently displays the fact too.

Tuesday, December 07, 2004

Good-bye JUnit

TestNG "TestNG is a testing framework inspired from JUnit and NUnit but introducing some new functionalities that make it more powerful and easier to use, such as:

* JSR 175 Annotations (JDK 1.4 is also supported with JavaDoc annotations).
* Flexible test configuration.
* Default JDK functions for runtime and logging (no dependencies).
* Powerful execution model (no more TestSuite).
* Supports dependent methods.

I started TestNG out of frustration for some JUnit deficiencies which I have documented on my weblog here, here and in particular, here. Reading these entries might give you a better idea of the goal I am trying to achieve with TestNG. You can also check out a quick overview of the main features and an article describing a very concrete example where the combined use of several TestNG's features provides for a very intuitive and maintainable testing design."

A simple example via this blog entry.

Monday, December 06, 2004

More SemWeb Articles

The anniversary of the Web is bringing out a lot of Semantic Web press. Another two:
* Beyond the World Wide Web and
* W3C focuses on future at 10th anniversary meeting

Metadata as Self Defense

Bootstrapping the semantic Web "It's tempting to draw parallels between the careers of Albert Einstein and Tim Berners-Lee. Both men made world-transforming breakthroughs and then pursued even grander visions. Einstein, of course, never found the unified theory he sought for three decades. A lot of people think Berners-Lee's vision of a semantic Web will prove equally elusive."

"Semantic-Web naysayers think people and organizations can't be bothered to assert machine-readable facts about themselves. And, today, that is undoubtedly true. But when others assert facts about you -- as they increasingly will -- the tide could begin to turn. Individual acts of self-defense may ultimately combine to bootstrap the semantic Web."

Rice of the future

Berners-Lee Maps Vision of a Web Without Walls "To envision the Internet of the future, W3C director and WWW founding father Tim Berners-Lee suggested during the W3C's 10-year birthday bash here Wednesday, first envision groceries—say a box of rice.

On the box's side, in small, rice grain-sized type, you will find nutrition information. On its back, you will find directions on how to cook it. Somewhere else you may find a URL that you can use to research any number of rice-related things: recipes, country of agricultural origin, Uncle Ben company data or relevant information pertaining to the allergenic nature of rice, perhaps."

"Haystack knocks down the partitions that separate e-mail clients, file systems, calendars, address books, the Web and other repositories so that information can be worked with regardless of its origin.

Such applications will have a big impact on personal information management, Berners-Lee said, as users will be able to do things such as drop their bank statements onto their calendars and have items automatically populate given dates.

Such descriptions sound familiar to anybody who's been following IBM's work with its Information Integrator technology or Oracle Corp.'s upcoming Tsunami content management offering, which it plans to roll out at Oracle OpenWorld in San Francisco next week."

"The Semantic Web is going to be like a huge data bus, Berners-Lee said—a back-end bus that spans the planet. Comparing it to Tsunami or Information Integrator is like saying there used to be Hypercards before the Web. "Yes, there were innumerable Hypercard applications before the World Wide Web," he said. "They just didn't talk the same language.""

Books, Music and Humour

* The nation's 10 favourite books and there's actually 11. 1984 is an interesting inclusion, panelist noting that it's relevant because of the similarities between Osama and Goldstein (with recent news of him disappearing).
* Don't believe the hype "U2 are probably the most over-rated band in history. Their debut, Boy, was a classic and still sounds fresh and impassioned. Fatally though, they became a band that believed their own (fawning) press and whose egotism has devoured their talent. The Joshua Tree showed what a good guitar group/stadium rock band U2 could be. Sadly, they had the sort of pretensions that usually afflict mediocre American outfits like the Chili Peppers. On Achtung Baby and Zooropa they started plundering other bands' innovations and moving into "dance music" - though only the whitest, geekiest student could dance to them..." Also The Beatles, both Evlii (Elvis Costello and Elvis), Prince, Nirvana, etc.
* The 10 Least Successful Holiday Specials of All Time Includes: Ayn Rand's A Selfish Christmas (1951) and The Lost Star Trek Christmas Episode: "A Most Illogical Holiday" (1968).

Processing Java Annotations

Gram "Gram is a simple xdoclet-like tool for processing doclet tags or Java 5 annotations in source code or bytecode and auto-generating files, data or resources.

Gram = Groovy + JAM. JAM does all the hard work of abstracting away the details between annotations and doclet tags and handling Java 1.4 and 5 compliance. Groovy takes care of the scripting, code generation & templating. Gram is the little tidy bit of code in between."

To Southampton and beyond

Berners-Lee takes professorial chair at Southampton University "Berners-Lee will take up a chair in computer science at the University of Southampton's School of Electronics and Computer Science, holding this position as well as being senior research scientist at MIT, and director of W3C."

Friday, December 03, 2004

OWLchestra

OWLchestra "OWLchestra is a Web vocabulary management system (WVMS) that enables the collaborative development of small-scale OWL ontologies and RDF schemas." Includes screenshots and a brief paper. I saw this a while back, via BNode.

My Other Computer is Google

The magic that makes Google tick "The numbers alone are enough to make your eyes water.

* Over four billion Web pages, each an average of 10KB, all fully indexed.
* Up to 2,000 PCs in a cluster.
* Over 30 clusters.
* 104 interface languages including Klingon and Tagalog.
* One petabyte of data in a cluster -- so much that hard disk error rates of 10-15 begin to be a real issue.
* Sustained transfer rates of 2Gbps in a cluster.
* An expectation that two machines will fail every day in each of the larger clusters.
* No complete system failure since February 2000."

PISTA

Semantic Association Identification and Knowledge Discovery for National Security Applications "Our goal is to research new techniques and improving effectiveness of techniques to identify semantic associations and knowledge discovery by exploiting a large knowledge base. Specific objectives include (a) ontology driven lazy semantic metadata extraction (i.e., annotation) to complement traditional active metadata extraction techniques, and (c) formal modeling and high-performance computation of semantic association discovery including ontology-based contextual processing and relevancy ranking of interesting relationships."

From the project report: "The nai?ve algorithm to find all paths between 2 nodes in a directed graph [1] shown below is a recursive implementation of a depth-first search. Our first implementation of the ?-path operator is based on this algorithm."

"Our initial implementation of the ?-Intersect operator is based on the ?-path operator. It searches for nodes where two ?-paths intersect (see Figure 5). We recognize the fact that there could be multiple intersection points for the ?-paths. Hence our implementation returns the sequence of nodes that are common between 2 ?-paths."

"The goal of ?-Iso is to take two resources as input and discover all paths that are “isomorphic” in both resources..."

Also, Semantic Association Identification and Knowledge Discovery for National Security Applications and Context-Aware Semantic Association Ranking.

Keeping Secrets

When Secrets Make Sense "Recently I wrote a short piece making a strong and general claim that the same forces that are pushing data towards XML are pushing software towards Open Source." From that article "These days, interoperation and integration are everything. You’d better have open interfaces, open networks, open services; that is, open data."

"’I'll put my finger on two pieces of Apple technology that benefit from being (for now) closed-source: Aqua and the video part of iChat. Both of them contain some magic that nobody else has figured out how to do yet, and if they can figure out how to make a few bucks in the gap before the world catches up, more power to ’em."

"Closed source isn’t over or anything like that. It’s just headed for a niche role, business-wise."

Not sure I believe that XML is the force to free you from vendor lock-in. RDF is of course - remember to put all your data in Kowari/TKS though, you know it makes sense.

A couple of quickies

* Via Matters EPOS "The objective of EPOS is to leverage a user's efforts for his personal knowledge management for his own benefit as well as to evolve this within the organization." The successor to Frodo.
* I'm sure everyone is now reading SWS IG scratchpad "Helpful paper tips: Repeat, "my god, we save X billion dollars with OWL-S! But we can't quite tell you how since that would reveal trade secrets! But trust us!" as many times as possible. Shrink margins if you must!" for the WWW2005 Tutorials and Workshops

Thursday, December 02, 2004

Space Colonies and Ontological Programs

What the Heck is an Ontology? (in PDF format) "There's no doubt about it; ontologies are on the up. The ideas are gaining wider acceptance and these days you're much less likely to be greeted by that familiar baf ed expression of those who are too afraid to ask what the heck you are talking about. Tools like Protege (http:// protege.stanford.edu/)continue to improve in robustness, and extend their functionality; and there are new tools on the scene too,such as SWOOP (http://www.mindswap.org/2004/SWOOP/) and Haystack (http://haystack.lcs.mit.edu/index.html)."

"Perhaps the hardest technical challenge is using ontolo-gies to enable the ad-hoc reuse of existing software com-ponents. In other words, it is much easier to design a suite of programs that all commit to the same ontology than to coerce existing programs to communicate effectively."

Wednesday, December 01, 2004

Coding for Failure

Automating Software Failure Reporting "Developing a failure reporting system requires an understanding of a product’s customer base, as well as usage profile. It is also important that you under-stand the failure profile of your product so that you can focus on events most annoying to your customers. Although product attributes are unique, there is a generic set of data that, if collected, will help in diag-nosing failures. In collecting customer data, however, you must address all privacy concerns prior to rolling out your process. Along with a failure collection system, you also need a process that can distribute patches to address those failures.

At Microsoft, our experience has led us to develop a generic methodology to process, transmit, analyze, and respond to customer failure data. Differences exist in the way this process can be implemented; it is usually dependent upon the product type and its failure profile. "

Java and LGPL

The LGPL and Java "FSF's position has remained constant throughout: the LGPL works as intended with all known programming languages, including Java. Applications which link to LGPL libraries need not be released under the LGPL. Applications need only follow the requirements in section 6 of the LGPL: allow new versions of the library to be linked with the application; and allow reverse engineering to debug this."

"If you distribute a Java application that imports LGPL libraries, it's easy to comply with the LGPL. Your application's license needs to allow users to modify the library, and reverse engineer your code to debug these modifications. This doesn't mean you need to provide source code or any details about the internals of your application. Of course, some changes the users may make to the library may break the interface, rendering the library unable to work with your application. You don't need to worry about that -- people who modify the library are responsible for making it work."

Monday, November 29, 2004

IBM boosts Oncology Ontology

IBM and Massachusetts General Hospital Announce Effort to Improve Information Sharing Among Cancer Researchers ""Effective tools for information management, integrated tightly with underlying computing and data infrastructures, are key to life sciences researchers gaining new insights into complex problems," said David Grossman, Distinguished Engineer, IBM Internet Technology Group. "In addition, the use of semantic web technologies to integrate many sources and formats of data with advanced modeling algorithms is particularly helpful for this type of large-scale collaborative project.""

""There is an urgent need to develop a common, unifying infrastructure that enables the integration and sharing of knowledge about cancer -- both in terms of disparate data and distinct computational tools -- with the goal of modeling cancer as a complex dynamic system," said Dr. Deisboeck. "While advances in cancer research and new technologies have generated a wealth of new data and insight, all too often the lack of shared systems and standards makes integration of this crucial knowledge difficult or impossible.""

Python and PHP

The Next Language "The vast majority of J2EE deployments (over 80% according to Gartner) are simply Servlet/JSP to JDBC applications. Basically HTML front-ends to relational databases. It is ironic that much of what makes Java complicated today is all of its numerous band-aid extensions, such as generics and JSP templates, which were added to make these types of simple applications easier to develop."

"Apparently what is needed is a language/environment that is loosely typed in order to encapsulate XML well and that can efficiently process text. It should be very well suited for specifying control flow. And it should be a thin veneer over the operating system."

Just to make this clear, the idea that the future of application development is turning them into "a big text pump" seems rather foreign and completely opposite to where application development seems to be going.

Requirements and Architecture

Requirements guru shares 'cosmic truths' "Wiegers' list of cosmic truths also includes:

* "Customer involvement is the most critical factor in achieving software quality."
* "The customer is not always right, but the customer always has a point."
* "Change happens."
* "If it’s not in the requirements specifications, don’t expect to find it in the product."
* "Even the best requirements document cannot replace human dialog."
* "You are never going to have perfect requirements.""

Architected RAD gets an A in Gartner study "The Gartner survey of development teams, completed in the past month, found the ARAD approach reduces training time and increases productivity of coders regardless of the vendor tool used.

"We have gotten consistently positive feedback from users of Computer Associates' Advantage:Plex, Compuware's OptimalJ and IBM's Rational Rapid Developer offerings," writes Michael Blechar, one of the Gartner analysts who worked on the survey."

"Of the newest tool technology, the Gartner report says: ARAD methods and tools are just beginning to achieve recognition by mainstream Java 2 Platform, Enterprise Edition (J2EE) and .NET developers. The tools provide development teams with pre-built J2EE and .NET frameworks as well as pre-built technical components, which Gartner says can be customized by technical architects and used to generate 60 to 85 percent of the code. Then the programmers on the development team can add the business logic specific to the application."

Sunday, November 28, 2004

Learning with the Semantic Web

Reasoning and Ontologies for Personalized E-Learning in the Semantic Web "Adaptive educational hypermedia systems are able to adapt various visible aspects of the hypermedia systems to the individual requirements of the learners and are very promising tools in the area of e-Learning: Especially in the area of eLearning it is important to take the different needs of learners into account in order to propose learning goals, learning paths, help students in orienting in the e-Learning systems and support them during their learning progress...We propose a framework for such adaptive or personalized educational hypermedia systems for the semantic web. The aim of this approach is to facilitate the development of an adaptive web as envisioned e.g. in (Brusilovsky and Maybury, 2002). In particular, we show how rules can be enabled to reason over distributed information resources in order to dynamically derive hypertext relations. On the web, information can be found in various resources (e.g. documents), in annotation of these resources (like RDF-annotations on the documents themselves), in metadata files (like RDF descriptions), or in ontologies. Based on these sources of information we can think of functionality allowing us to derive new relations between information. "

Friday, November 26, 2004

Free trade that isn't free

Patently yours "We quite understand that (the title) How to Kill a Country may sound alarmist...We use the parallel experience of Canada to buttress some of these points. Canada is now being described by leading author, Mel Hurtig, as a "Vanishing country"...By the mid 1980s, about half of the major US corporations in Canada were 100-percent American-owned. Ten years later, some 85 per cent had no Canadian shareholders...As Canadian shareholders were eliminated, corporate boards were substantially reduced in size and more American directors were added, as were more U.S. CEOs and board chairmen. As external directors were eliminated, there was no longer a force to influence policy decisions which would be beneficial to Canada. Gone too was the ability to scrutinise the payment of dividends, management fees, and content costs paid to the parent company."

"But the Australian negotiators overlooked the point that Australia is a net importer of IPRs...As a whole, Australian industry has everything to gain by moving away from the Microsoft stranglehold and towards an Open Source mode - rather like governments in Germany and Taiwan are currently doing in earnest...local firms would do well to shift towards the Open Source model, and utilise open source programs such as Linux..."

"...frequently the actions are entirely justified, and entirely in the spirit of competition - as when an importer of copyright-protected CDs seeks them out in a third market and imports them, entirely legally, at a lower cost than is stipulated by the IPR-holder. The FTA makes this action much more difficult - in the name of placing severe restrictions on parallel imports. Another name for this is placing restrictions on free trade in IPR-protected goods - all within a "free trade" agreement!"

Dangling Databases

Why Relational Databases And Semantics Don't Mix "Jarg Corporation, which takes its name from "jargon", is about the next evolution of search. Actually, it is about more than that since semantics has wider applicability than search, but we will stick with search as an easily understood example of this technology."

"So, the question is: how does it do that?

The first answer is that it doesn't - at least in any general sense - only where it has already built an ontology which, in this case, is within healthcare. Indeed, its first customer is a hospital medical library. However, industry knowledge bases are becoming widely available and Jarg reckons that about 75% of the work involved in creating an ontology can be automated, so extending its product for new customers should not be a big issue."

"The second answer is that it achieves this sort of performance by refusing to use a relational database. Instead, it has patented its own approach, which involves storing semantic fragments. A segment fragment is either two elements and the relationship that joins them (for example, "Waterloo is a station") or it can store and element with a "dangling" relationship. This latter concept is especially important. The whole point about searching is that you want to be able to discover relationships that you didn't know existed."