Kowari on XML.com

A First Look at the Kowari Triplestore "Kowari is a solid tool created by an enthusiastic, knowledgeable team. That said, it's not for everyone: the architecture of the application is clearly focused on the server, and developers looking for an embeddable RDF store for desktop apps will likely want to look elsewhere, unless they are willing to add several megs to their applications. Kowari's dependence on Java is another possible sticking point for those developing tools using other frameworks. Documentation is brief and unfinished, but what's there is useful for the adventurous."

Updated documentation is going to be released to coincide with 1.0.4. Hopefully, this should make it a little more accessible to most.

"According to Tom Adams, "Our triplestore is really a quad store, the 4th tuple being the group/model that a triple belongs to.""

Apparently, I made this quote but it doesn't matter.

Paul said, "a better name might be "platypus"" - that was already taken by a previous project.

Semantic Clarity

Two related pieces dealing with integration and the Semantic Web.

Triple-based Computing "[Johanson & Fox, 2004] expect ubiquitous computing as the “killer app” for tuplespace-based computing because of the model’s portability, extensibility, flexibility, and ability to deal with heterogeneous environments...Since applications are decoupled in reference, time, and space many issues in protocol and process alignment disappear because they are provided by the underlying middleware that implements the tuplespace."

Semantic Web bringing clarity to the Universal Server concept "I would also like to conclude that what we know today, as the monolithic "point of presence" on the web called a "Web Site" (which infers browsing and page serving), is naturally going to morph into a different kind of "point of presence" that is capable of delivering the following from a single process:

1. Serve up Semantic Data from existing data sources
2. Provide execution endpoints for Web Services
3. Provide an instigation point for events that trigger Service Orchestratio

This is what Virtuoso is all about, and why it is described as a "Universal Server"; a server instance that speaks many protocols, delivering a plethora of functionality (Database, Web Services Platform, Orchestration Engine, and more)."

D2RQ

D2RQ - Treating Non-RDF Databases as Virtual RDF Graphs "Using D2RQ you can:
* access information in a non-RDF database using the Jena model API.
* query a non-RDF database using RDQL or find(spo).
* do RDFS and OWL inferencing over the content of a non-RDF database using the Jena ontology API.
* publish the content of a non-RDF database on the Semantic Web using the Joseki RDF server."

D2R is already integrated into TKS 2.0 and hopefully we'll get D2RQ in there. Good to see it using RDF for configuration.

News Services go RDF

Adobe and IPTC Collaborate on the Implementation of Metadata Standards for News Organizations " Adobe Systems Incorporated (Nasdaq:ADBE) today announced that it is collaborating with the International Press Telecommunications Council (IPTC) to extend the capabilities of IPTC metadata through use of the open-standard, Adobe XMP (Extensible Metadata Platform). Based on World Wide Web Consortium (W3C) standards, XMP is a specification expressed in XML that provides a standardized framework to capture and manage metadata, the information that describes files.

Photographers and news services around the world use IPTC metadata schemas to embed identifying characteristics such as captions, credits, location and time and dates within digital photos and scans. This information can be read by content management systems, used by virtually every newspaper publisher, to review and identify photos they want to publish to accompany a story. Adobe and IPTC also announced plans to develop future implementations that will be accessible via the Adobe(R) Creative Suite, and related point products including Photoshop(R) CS, through a customizable metadata user interface. This is intended to streamline the data capture process and allow relevant IPTC metadata to be preserved as the file is utilized across news and derivative workflows. "

Eve of Sleepycat

Apache Leader: Berkeley DB Java Edition Eases Hand-Rolling "In the case of the Apache Directory Project, Karasulu says the project members are so convinced that Sleepycat's pure-Java database will become the "de facto standard Java API for manipulating B-tree databases" that they're basing the project's pure-Java LDAP directory server, called Eve, on back-ends built using Berkeley DB Java Edition."

"Who's building applications in Java? Does it span the spectrum?

I think so. People can use a B-tree anywhere a fast, relatively constant time lookup is needed, regardless of the size of the data. This happens all over the place.

We're actually in the process of building an embeddable, pure-Java LDAP server called Eve. She's a beaut, introducing triggers and stored procedures to the world of LDAP. "

" What do you think of Berkeley DB Java Edition's performance capabilities?

It uses NIO [new input/output], which should have a considerable effect on performance due to the way memory is accessed. The NIO packages are new APIs that allow Java to have the power of C where IO is concerned. These APIs are new in the 1.4 JDK and make Java-based servers much more effective. Furthermore, a binding API makes mapping objects to records very intuitive while avoiding the overheads in serialization.

Other implementations often leave this up to users, who often use serialization and wind up paying for it with a massive performance hit. "

There was also a recent comparison between C++ and Java showing Java is faster in all but three tests.

The Next Web - Web Services

The Third Web "Web services represent the third attempt to rebuild enterprise IT on the standards-based foundation conceived by Tim Berners-Lee. To realize that vision, vendors and IT pros must resist some familiar temptations and avoid some traditional mistakes...But it would be a sad repetition of past errors if IT vendors seized upon the success of Web services as their latest excuse for locking customers into product stacks—perversely building proprietary structures from standards-based components, for example, by devising proprietary XML schemas for their applications and documents."

Scaling Taxonomies

Verity Announces New Content Classifier (with screenshot) "VCC is based on roles and rules. Roles include taxonomy experts, subject matter experts within the organization (e.g., chemists, engineers, human resources staff, etc.), editors, and publishers. The company says the workflow feature allows taxonomy and classification management to be distributed to subject matter experts who know the content, as well as to knowledge engineers who know and understand taxonomy development. Different people who serve different roles are assigned different permissions to alter categories. The company claims that VCC is the only software that enables such real-time collaboration between knowledge workers and subject experts.

VCC uses rules to define how documents should be classified. Once taxonomies have been set up, VCC automatically classifies new documents as spiders discover them. The customer can control automatic classification by defining how well VCC thinks a document matches a category. For instance, if VCC’s confidence level for a candidate category exceeds 70 percent, then automatically publish it into that category; if not, route the decision to an assigned knowledge worker. Verity calls this “automated classification with manual oversight.”

Verity worked with DuPont to develop VCC. Whitney said the company uses VCC to manage a 25,000 node taxonomy. Internal users include “everyone from a bench chemist to a knowledge engineer—whoever.” He said VCC provides DuPont with “frictionless review between highly specialized knowledge workers, many with PhDs, and the knowledge engineering staff.”"

Semantic Web for 5th Graders

This was an interesting find using the Google Catalog to search for the Semantic Web.

It says: "Twenty-two lessons show students how to build a vocabulary...(KWL, SQ3R, semantic web making, outlining)...". Of course, it doesn't look like it's *the* Semantic Web.

Searching for Java by itself reveals some interesting results. Searching for Java in the Computers category reveals no hits; the category function seems to be broken.

Global Scalability

Semantic Web: Hype-Bubble or an interesting research area? Links to a comment made by on the SUO list "In 6 years (1998 to 2004) with ENORMOUS hype and funding, the semantic web has evolved from Tim BL's book to a few prototype applications, which are less advanced than technologies of the 1970s such as SQL, Prolog, and expert systems -- and they're doing it with XML, which is far less advanced than LISP, which was developed in the 1950s."

There are many interesting responses in the thread including one from Frederick Kintanar: "...I do think the Web makes a big difference between earlier knowledge representation efforts and Semantic Web initiative. A big one is global scalability, where the key element is URI's (and their already deployed global acceptance). Hypertext was already a relatively mature technology in the research community, when Tim Berners-Lee hit upon what is needed to make it scalable: global identification..."

And no this isn't scaling a single agreed upon ontology across the Web.

Spicy Searching

IBM expands search push with Masala "The computing giant, based in Armonk, N.Y., is gearing up to release Masala, a new version of its DB2 Information Integrator software that will let corporate employees retrieve information from databases, applications and the Web at the same time. Subsequent improvements will include a data-mining component code-named Criollo."

"Microsoft, though, isn't standing still. It is working on its own distributed search plan with Longhorn and a new release of its SQL Server database, code-named Yukon, and plans to build its own Internet search service. BEA Systems and others are working on similar technology."

"Information Integrator is a software layer than can pull data from different software--Oracle databases, Microsoft Excel, IBM's own DB2 and Lotus databases--with a single query. IBM and other companies are touting this "federated" database approach, in which searches tap into spread-out data sources, as a potentially cheaper alternative to shipping and storing large amounts of information in a single database."

Ruined by Developers

How Microsoft Lost the API War Most of this is standard "how the Web has won" but there were a couple of interesting snippets.

"WinFS, advertised as a way to make searching work by making the file system be a relational database, ignores the fact that the real way to make searching work is by making searching work. Don't make me type metadata for all my files that I can search using a query language. Just do me a favor and search the damned hard drive, quickly, for the string I typed, using full-text indexes and other technologies that were boring in 1973."

"RSS became fragmented with several different versions, inaccurate specs and lots of political fighting, and the attempt to clean everything up by creating yet another format called Atom has resulted in several different versions of RSS plus one version of Atom, inaccurate specs and lots of political fighting. When you try to unify two opposing forces by creating a third alternative, you just end up with three opposing forces. You haven't unified anything and you haven't really fixed anything."

Loom and Drools

Loom is now open source (a BSD-like license). "Loom is a language and environment for constructing intelligent applications. The heart of Loom is a knowledge representation system that is used to provide deductive support for the declarative portion of the Loom language. Declarative knowledge in Loom consists of definitions, rules, facts, and default rules. A deductive engine called a classifier utilizes forward-chaining, semantic unification and object-oriented truth maintainance technologies in order to compile the declarative knowledge into a network designed to efficiently support on-line deductive query processing."

An article theServerSide.com introduces "...the JSR-94 Rules Engine API and an Open Source product called Drools, the forerunner implementation of this up-and-coming technology...it can scale to incorporate and execute hundreds of thousands of rules in a manner which is an order of magnitude more efficient then the next best algorithm." Drools Homepage.

WSDL2OWL-S 1.1 Released

WSDL2OWL-S "...provides a partial translation between WSDL and OWL-S. The results of this translation are a complete specification of the Grounding, partial specification of the Process Model and Profile and Daml Class file, when at least one of the input and output messages are of XSD Complex type." Download it here.

TKS the RDF Database

The Semantic Web in the Enterprise ""RDF provides a simple model to represent logical statements in the form of subject, predicate, object," said Daconta. "While this follows a linguistic approach, that same model can also represent resource, property, and value. This triple model is a powerful underpinning of more powerful languages layered on top of it like OWL."

He referenced one of his non-RDF projects that uses explicit database tables to make certain data associations. This approach is neither scalable nor flexible. The unique nature of RDF, however, can provide a flexible mechanism that would allow far greater associative capabilities, thereby increasing the ability to query and make inferences on topic matters not explicitly hard-wired into tables.

Daconta added that good strides have been made in commercial products supporting Semantic Web technologies, including RDF databases (such as Tucana Knowledge Server), ontology editors and inference engines (such as Network Inference's Cerebra Server), and data source integration engines (such as Unicorn System)."

The author is Ken Fromm from Loomia who are working on "a semantic technology software company building an RDF/FoaF-based identity and relationship stack".

Behind the Wall

Something Useful This Way Comes "Mike Champion offered an optimistic note, suggesting that Semantic Web technology may first flourish behind the enterprise firewall, in a way reminiscent of the earliest days of Netscape's corporate success:

The other previously missing ingredient is that real organizations have at least something approximating an implicit ontology in their database schema, standard operating procedures, official vocabularies, etc. It is at least arguable that the technologies that have emerged from the Semantic Web efforts allow all this diverse stuff to be pulled together in a useful way -- ontology editors, inferencing engines, semantic metadata repositories, etc. I'm seeing real success stories in my day job, and a coherent story is starting to be told by a number of vendors, analysts, etc.

Champion here makes a similar point to the one I argued in an article last fall ("Commercializing the Semantic Web"), namely, that there exist today several startups and fledgling ventures that are selling Semantic Web technologies to corporate clients, including Network Inference, Tucana Technologies, and others."

"Honestly, I don't know whether to laugh, because with WinFS Microsoft seems to be buying into the Semantic Web idea, or cry, because with WinFS Microsoft seems to be embracing-and-extending the Semantic Web idea. Oh well -- outside of the realm of unenforced US antitrust legislation, Microsoft is like gravity. Eventually, you just learn to work around it."

Semantic Web Tutorial

Tutorial on basic SW Technologies Nice to get mentioned in the same breath as Sesame, Redland, etc. (not quite with Jena which got two of its own slides). Links also included: graphical editors and the Semantic Web Application Survey

WinFS and RDF to Square Off?

Questions about Longhorn, part 2: WinFS and semantics "Today's personal information systems are organized hierarchically. WinFS proposes that they be organized semantically. A number of observers have noted a family resemblance between RDF (Resource Description Framework) "triples" and WinFS relationships. An RDF triple, in geek-speak, is a subject-predicate-object relation. Sets of RDF triples can be (and Semantic Web people say must be) used to represent and organize knowledge."

Where do you get Longhorn today? RDF and schemas for RDF are available now. Taking the both bottom up and top down schemas and deployed locally and globally. Last year I asked, Should the RDF model be integrated into the File System?, the answer still seems to be 'yes'.

Danny has a good summary of part 1 of this discussion.

Can't get enough of the Semantic Web?

The Special Interest Group on Semantic Web and Information Systems. The first bulletin (PDF) is 75 pages of Semantic Web goodness. Includes interviews with Jim Hendler and Amit Sheth, where the SIGSEMIS is positioned, and various papers on the Semantic Web. There's also related events, an industry column (about Semagix Freedom) and book reviews.

Cynical Development

A fresh look at the waterfall "Analysis -> Dream, Design -> Guess and Waffle...Support -> Duck and Deny"

Sesame Speed

The KIM Platform: Performance and Scale "The result: a repository of 15M explicit statements, describing 1.2M entities, is manageable with an indicative upload speed of 1300 statements/sec...The experiment was performed on a 2xOpteron 240 (1.4GHz) server with 6GB of RAM - a $3000-worth, brandless machine. 64-bit beta-versions for Amd64 of Windows 2003 Server and JDK 1.5.0 were used."

iTQL evaluated

iTQL evaluation We do support optional match through the ugly sub-query syntax but apart from that I think Kendell has it right. Aliases allow you to shorten URIs.

Datalog and Inferencing

Implementing OWL Lite in rule-based systems and recursion-enabled relational DBs "The semantics of negation in stratified Datalog is not compatible with the semantics of negation in Description Logics...Alternative, more compatible semantics for negation in Datalog exist, namely well-founded semantics, that also relies on such a three-valued logics. However, this is not semantics available in SQL:1999 compliant databases. Well-founded semantics only regards the minimal model when computing negation, whereas Description Logics regard all possible models. Hence, they are also incompatible."

Two other related papers: Bubo - Implementing OWL in rule-based systems and Description Logic Programs: Combining Logic Programs with Description Logic.

This maps well to 3VL and the recursive predicate functions trans and walk.

Not Relational Enough

Two recent articles about XML and database technology. One of the FAQs about Kowari is in respect to relational databases. Taking the pedantic view, the two databases mentioned are not relational but SQL. I've previously posted links about XML management systems too.

In, XML, the New Database Heresy "One of the major benefits of using XML in relational databases is that it is a lot easier to deal with fluid schemas or data with sparse entries with XML. When the shape of the data tends to change or is not fixed the relational model is simply not designed to deal with this. Constantly changing your database schema is simply not feasible and there is no easy way to provide the extensibility of XML where one can say "after the X element, any element from any namespace can appear". How would one describe the capacity to store “any data” in a traditional relational database without resorting to an opaque blob?

I do tend to agree that some people are going overboard and trying to model their data hierarchically instead of relationally which experience has thought us is a bad idea."

Edd Dumbill wrote in, Ron Bourret on XML and databases "My guess is that everything will pick up on this front in a year or two, with companies moving towards what I consider the holy grail of XML support in relational databases: native storage behind a first-class XML data type, XQuery support with extensions for (a) including relational data or SQL queries and (b) updates, SQL/XML support with extensions for embedded XQuery queries, and support for JSR 225 (see below)."

One of the aspects we've found in implementing TKS/Kowari was that it pays to stick closely to the relational model. Everything comes down to tuples and I think that the problems with putting RDF on top of SQL database is that they aren't relational or "relational enough" (whatever that means).

Actually, the DAWG is asking for comments, maybe ensuring that the queries are expressed in relational algebra. That should prevent things like NULL getting in there.

Why FOAF? Why XFN?

This is a fairly old blog entry (4 months) but I hadn't seen it before, it discusses another social software format, XFN.

Social software snippets "So Jonas poses a good question, why was FOAF invented? Perhaps an attempt to justify RDF? Is this an example of a solution looking for a problem? A clever acronym looking for a reason to exist?

Speaking of which, the biggest irony I see about FOAF is its name, which stands for "Friend of a Friend", and yet the technology has nothing to do with "friends". Like Jonas said, it's simply vCard recast in RDF, except for the "knows" relationship, which itself is quite meaningless (quite ironic for a Semantic Web effort), as it provides no more meaning than a plain hyperlink. Why work so hard for so little?

FOAF could be saved however, simply by adding an XFN module, thus enabling FOAF to finally fulfill its namesake, and represent friendship rather than just claiming to."

TCO of open source software

Weighing the costs of open source "While MySQL passed performance tests, the IT staff at the credit card processing company became concerned that MySQL didn't have enough formal support backing it up. What if the credit card processing company's databases failed, asked Tim Kelly, who serves as technical director of technology at TSYS.

"We were not prepared to deal with new support contracts and rely on an alternative database with our customer data." Tim Kelly director of technology, TSYS

"We have a procedure on how to roll back transactions when something goes wrong, but in the event that everything you try doesn't work and you look for support, there area million comments out there on the Web -- and newsgroup articles on MySQL -- but in production scenarios, you can't really rely on that," Kelly said.

Figuring out the total cost of ownership for a DBMS can be more complex than many companies expect at the outset, said Mike Schiff, vice president of data warehousing and business intelligence at Sterling, Va.-based Current Analysis.

"The cost of ownership isn't just the cost of acquisition or maintenance; it's also the vendor responsiveness when you've got a critical issue and downtime," Schiff said. "The cost of dollars to a company that has a database down can be staggering.""

RDF Data Access Use Cases and Requirements

No keeping your bananas unbruised but some pretty important use cases:
* 2.2 Finding Information about Motorcycle Parts (Supply Chain Management)
* 2.5 Avoiding Traffic Jams (Transportation)

More seriously, they don't mention beep but I would assume that the "web" in Semantic Web will mean the protocol will be http. Aggregate queries (4.5) might mean getting a FROM clause, or something like it, in the language.

http://www.w3.org/TR/rdf-dawg-uc/

Link Extravaganza

More news from the WWW2004 conference:
* WWW conference mulls Web as personal memory store
* Doug Cutting Interview (interesting interview about Nutch)
* Wanted: Cheap Metadata
* The semantic web and uncertainty (does the Semantic Web stack need a representation of uncertainty)
* SWRL: A Semantic Web Rule Language Combining OWL and RuleML.
* Some *really* Unknown News: Here comes the Semantic Web
* New Web Structure Promoted
* Berners-Lee Keeps WWW2004 Focused on Semantic Web
* Semantic Web to Take Center Stage at WWW2004
* Apparently, "As We May Think" and Memex are still news too.

Two other conferences

Two other conferences that maybe of interest occurred in May:
* 1st European Semantic Web Symposium (ESWS 2004) and
* Metadata Practices on the Cutting Edge

The Semantic Web is your Friend, FOAF that is

The Semantic Web is Your Friend "...given the proliferation of data about data (metadata) that underpins the Semantic Web, it is tempting to focus in on the obvious prospect of a better-than-Google search engine...As clever and useful as this may be, it is only an evolutionary enhancement of something that is already possible on the Web today. By looking in a little more detail at what is happening on the Semantic Web today, it is possible to gain a deeper insight into where revolution is starting to occur."

"Applications like FOAF are at the vanguard of the Semantic Web, enabling a glimpse of what might be achieved."

I'm sick of FOAF being held up as the vanguard of the Semantic Web...

A Few New Semantic Web Tools and Browsers

Many of these were presented at the WWW2004 conference:

* Ganesha will be available soon for download.
* Simile and Longwell (a faceted approach using Jena and HTML).
* Haystack the next version - based on SWT and Eclipse.
* SWOOP 2.0 (which changed from using Jena to the Machester OWL API).

Ontographics

"Ontographics is another visualization of graphs structures. It's a "...organization and presentation software. It is suitable for managing all kind of information and data: documents, projects, knowledge, and so on. Ontographics makes it easy to present, share and communicate informations to other people."

"It is a fully functional program, but is missing some features from Ontograpics Silver:

* no attachments (Word documents, images, etc) on nodes
* no installer
* no support"

Screenshot.

Goodbye New York

Paul Ford mentioned Kowari/TKS/Tucana Technologies in his writeup: WWW2004 Semantic Web Roundup.

"Many of the Semantic Web projects discussed during the conference used Jena as a backing store, but one contender to Jena's throne is Kowari, from Tucana Technologies."

"Rather than competing directly with Jena, Kowari includes support for the Jena API, as well as JRDF, an alternative RDF-management API, and adds to these APIs with a new SQL-like query language, iTQL, that can be used via a built-in interactive shell."

The TKS/Kowari presentation can be downloaded here. At the Semantic Web Developers Day Kowari got mentioned during the "Harpers.org: a Semantic Web Case Study", Simile and MINDSWAP presentations.

We should continue to make it clear that Kowari is not really going to take Jena's throne. In the short and medium term I can only see us improve Jena support. The slated FastPath and improvements in ontology speed are both things we're looking at putting into Kowari soon.

JRDF's goal is to be an API more approriate to use for our underlying triple store. With Kowari, we needed a better abstraction of using RDF triples and throwing away everything and doing Jena would've taken too long. The Jena support is basically a mapping of Jena to JRDF (and back again).

It was great to meet a lot of people face to face. And I got a lot of interesting news about various people using or evaluating Kowari. I think it'll lead to some interesting news in the near future.

Semantics in P2P and Grids Notes

Not having looked deeply into this in the past except at the The Semantic Grid web site I came into the workshop without much context as to what the talks were going to be about.

The first talk was about using query expansion on a peer to peer network. It extended Limewire to allow different nodes to expand the keywords using something called Keyword Relationship Databases (KRDB). Weightings are graded based on how successful the data they return is.

The second speakers presentation was about using Small World Networks (SWN) as a way to scale P2P networks. You place the data and the nodes in a k-dimensional space. The problem then is that you have to keep the number of dimensions reasonable. The speaker gave an example of flattening a 2-dimensional space into a 1-dimensional space by basically turning it into a linked list.

Bibster was the topic of the third paper. This used a combination of SeRQL, Sesame and JXTA to implement a distributed bibliographic system. It also used similarities in the metadata to route queries and remove duplicates.

The next talk focused on creating truely distributed P2P databases. It appeared that they were using something that looked liked lattices to implement schema matching between peers. This was called "correspondence rules". There was also "coordination rules" which dictacted how queries were forwarded to peers. It also used JXTA. The main feedback seemed to be that the paper gave names to these problems and but didn't really solve them.

coDB was the next topic of discussion. Which seemed to be a way of doing First Order Logic (FOL) over P2P networks. I had a problem understanding the point of this one. For example, the paper mentioned that if peers that created the result drop out then the answers from that peer should be removed. The feedback from someone else was whether it was useful in an open network to do this. The example brought up, was marriage. He was saying a peer would make the inference that if one person in a marriage was male the other would be female. And that brought up the rather topical example of gay marriages. What one peer inferred maybe different for different reasons than another peer. This was a bit beyond the scope of the paper though it seemed like a valid criticism.

The sixth talk was done by a someone from HP. This was pretty much the highlight of the morning. Because it specifically talked about RDF and OWL and they used Jena (of course). The main thing to come out of it is that they wanted to be able to calculate properties ("virtual properties") expressable in RDF/OWL. For example, server price is a funtion of software and hardware price. They created a type of TransitiveProperty called FunctionalProperty. A functional property would define a function like "foo = x * y + z". How this would be defined in OWL hasn't been defined as far as I know. It sounds similar to aggregate functions like average, sum and count but are fully configurable. I vaguely remember someone at work talking about this. Obviously, a cool idea anyway.

The last three talks were about trust, GridDB and ebXML. They were fairly short but interesting. Probably the most interesting thing I got out of it was that OASIS (who are working on ebXML) are trying to be inclusive of current technologies like Topic Maps and RDF/OWL. In fact, the speaker did say that UML was going to be expressable in RDF/OWL. I also thought that GridDB looked a lot like our current TMex product.

Anyway, apart from that I'm really enjoying the conference. Google set up a booth to try and get people to work for them - little did they know I was only after their pen.

Pure Abstraction

RDF as data abstraction "Mozilla's central interface for dealing with data is RDF. RDF datasources can have any implementation they like behind them. However, they must provide the information through the nsIRDFDataSource interface that specifies a graph API of nodes and edges. Once this is done, any client can operate with the datasource without reference to the domain objects, methods, or underlying implementation. You just need to know how to manipulate a graph, and you're set.

This is the true potential of RDF. It's not a substitute for databases or XML. RDF is a directed labelled graph. It hides information and provides a data interface which can be used to aggregate multiple datasources that can be themselves RDF datasources. This is the purest data abstraction. Theoretically, if you weren't worried about performance you could access an RDF datasource and not know or care whether you're accessing a database, a web service, or both. The RDF datasource might even optimize its data structures based on what your queries or iteration patterns have been, in much the same way as hotspot compilation optimizes algorithms in the JVM.

A good question at this point is whether RDF is as powerful an API as JDBC. The answer is yes, or not quite. RDF is only a data structure, and there are several APIs for manipulating that data structure. The most well known implementation is Jena. Jena's API for manipulating RDF graphs is very powerful, and encompasses most things you would find in JDBC, such as transactions, a query language, and prepared statements."

The Blank Node Release

Kowari 1.0.3 is now available for download. Loading RDF/XML with blank nodes is now faster, inserting statements with blank nodes now works correctly, and blank nodes in Jena are now properly mapped. There are other things. It's annoying that RDF has named things (about) and unnamed things (blank nodes) and locally named things (blank nodes with node IDs). iTQL adds variables which are differently labelled blank nodes too.

The next release should have some inferencing and resolver work done to it which will be a major change to the architecture - unless we find some critical bug.

Paul has also updated the plans for a new triple store for Kowari/JRDF. I'm favouring the second approach but they both have positives and negatives. The ring structure has some advantages over the current structure (and the first approach) as it greatly reduces the amount of disk usage. I think bloom filters have been put on the back burner.

Rewerse

Reasoning on the Web with Rules and Semantics "The objective of REWERSE is to establish Europe as a leader in reasoning languages for the Web by

1. networking and structuring a scientific community that needs it, and by
2. providing tangible technological bases that do not exist today for an industrial software development of advanced Web systems and applications."

Bloom Filter Implementation

Based on the bloom filters post a while back it looks like Paul and DM will be doing one - hopefully with a JRDF interface.

I'll have to get JRDF up to scratch. Some recent comments from both Paul and others make me acutely aware that it's definitely not even close to being the last word in Java RDF APIs.

Danny Hillis on the Knowledge Web

"ARISTOTLE" (THE KNOWLEDGE WEB) ""I am interested in the step beyond that," he says, "where what is going on is not just a passive document, but an active computation, where people are using the Net to think of new things that they couldn't think of as individuals, where the Net thinks of new things that the individuals on the Net couldn't think of."

"In the long run, the Internet will arrive at a much richer infrastructure, in which ideas can potentially evolve outside of human minds. You can imagine something happening on the Internet along evolutionary lines, as in the simulations I run on my parallel computers. It already happens in trivial ways, with viruses, but that's just the beginning. I can imagine nontrivial forms of organization evolving on the Internet. Ideas could evolve on the Internet that are much too complicated to hold in any human mind.""

"In this regard, thanks to funding from the Markle Foundation, Danny been able to assemble a group of people to begin to discuss of the implementation of a medical application based on his ideas."

Medical application, hasn't this been done before? Only Jaron Lenier mentions the Semantic Web.

Euroweb

Europe spins the semantic web "A number of exciting new semantic web projects have been recently launched as part of Europe’s latest framework research programme. These projects are being showcased in the 1st European Semantic Web Symposium (ESWS 2004) on 10-12 May in Heraklion (http://www.esws2004.org/ )."

Securing the Semantic Web

Following a question on rdf-interest there's been two interesting links with regard of a vocabulary for describing user permissions:
* This document describes the ACL storage and query mechanisms used by W3C, as well as the availability and use of this data on the semantic web.
* Semantic Web Trust and Security Resource Guide and more specifically KAoS Policy and Domain Services:Toward a Description-Logic Approach to PolicyRepresentation, Deconfliction, and Enforcement.

kSpaces.net

kSpaces "...is a metadata-driven, distributed knowledge management platform. It was designed to be lightweight, transparent and extensible...kSpaces automatically tags files through the use of plugins. The two autotagging plugins that are included analyze a file's ID3 and EXIF headers, and then generate the appropriate RDF metadata.

Metadata associated with a file can be viewed and edited through the kSpaces Node application, supported by editor plugins. Five editor plugins have been included in the proof-of-concept, four of which are read only. These plugins allow the management of a subset of Dublin Core metadata, EXIF metadata, ID3 metadata and kSpaces-specific metadata. The Raw RDF plugin shows the raw RDF metadata associated with a knowledge asset."

Cool, an extensible metadata extrator.

Broken Windows

Something that I've been pushing with Kowari recently.

Don't Live with Broken Windows "You don't want to let technical debt get out of hand. You want to stop the small problems before they grow into big problems. Mayor Guiliani used this approach very successfully in New York City. By being very tough on minor quality of life infractions like jaywalking, graffiti, pan handling—crimes you wouldn't think mattered—he cut the major crime rates of murder, burglary, and robbery by about half over four or five years.

In the realm of psychology, this actually works. If you do something to keep on top of the small problems, they don't grow and become big problems. They don't inflict collateral damage. Bad code can cause a tremendous amount of collateral damage unrelated to its own function. It will start hurting other things in the system, if you're not on top of it. So you don't want to allow broken windows on your project.

As soon as something is broken—whether it is a bug in the code, a problem with your process, a bad requirement, bad documentation—something you know is just wrong, you really have to stop and address it right then and there. Just fix it. And if you just can't fix it, put up police tape around it. Nail plywood over it. Make sure everybody knows it is broken, that they shouldn't trust it, shouldn't go near it. It is as important to show you are on top of the situation as it is to actually fix the problem. As soon as something is broken and not fixed, it starts spreading a malaise across the team. "Well, that's broken. Oh I just broke that. Oh well." "

Eh

I'm really impressed by Paul's blow by blow report on our inferencing work (he also mentions Lawrence Lessig on Australian radio). We're only doing RDFS at the moment. We'll being doing at least owl tiny, we're still deciding. Without Sesame or Jena some of these architectural decisions would have been much harder. I think we've got the right mix of new ideas based upon work that's been done before.

Edd Dumbill won't be attending WWW2004 and that's a shame. Especially because I'm going to be there - hopefully to put faces to names. I think the developer's day looks good - "Doug Cutting, the leader of the Nutch open-source search engine project, and an interactive luncheon Q&A with Tim Berners-Lee." TKS will be there too. TKS is Kowari with other features such as security, related to queries, support and GUI management.

Kowari linkers

Knowing what people are thinking is important in improving the product. With that in mind I did a quick Google for people mentioning Kowari. I'll use it here to correct anything too.

DeliverableS4Simile "Naive use of persistent store in Jena decreases performance by 100x:
* Part of the problem is limited expressivity in RDQL
* Look at performance tuning on databases
* Ryan: Look at the performance of more specialized RDF databases, cf. Kowari"

Open Source Projects That Use Java NIO "The storage engine of Kowari is a transactional triplestore known as the XA Triplestore. ll relevant fields of in-memory and on-disk data structures are 64 bits wide..."

Kowari for hundreds of millions of triples "Jim Hendler emailed me in response to my having mentioned on www-rdf-interest@w3c.org that I was surveying triple stores for use in data mining and machine learning. He mentioned a Java-based, non-relational, triple store called Kowari that is available in open source form..."

RDQLPlus "I discovered Kowari last week. The iTQL language is very similar to what I've come up with for RDQLPlus... having equally been inspired by SQL/DDL. Kowari looks like a nice database. I've downloaded it but haven't had a chance to play with it yet..."

Some Tools "Kowari is a layer on top of Jena with OWL reasoning, too"

I would say that Kowari is a layer beneath Jena - one that provides persistance. The OWL reasoning is really only offered at the moment through Jena. However, we will be getting some basic inferencing, at our own query layer, in there soon too.

[protege-discussion] Re: large data sets, bulk data acquisition "I had a really bad time with Kowari earlier this year, it wouldn't compile and then pass its own self-tests...."

This is basically problems with Windows. We develop on Linux and OS X and only do QA on Windows. Our initial release had known problems under Windows - which lead to failing unit tests but was not fatal for data storage. Anyway, it is fixed now; although Windows does have some drawbacks when it comes to using NIO.

Ontological Software Development

Semantic Mapping, Ontologies, and XML Standards ""Within an ontological framework, integration analysis naturally leads to generalization."

Considering that statement, it's also clear that application independence of ontological models makes these applications candidates for reference models. We do this by stripping the applications of the semantic divergences that were introduced to satisfy their requirements, thus creating a common application integration foundation for use as the basis for an application integration project."

"Once we define the ontologies, we must account for the semantic mismatches that occur during translations between the various terminologies. Therefore, we have the need for mapping.

Creating maps is significant work that leverages a great deal of reuse. The use of mapping requires the "ontology engineer" to modify and reuse mapping. Such mapping necessitates a mediator system that can interpret the mappings in order to translate between the different ontologies that exist in the problem domain. It is also logical to include a library of mapping and conversion functions, as there are many standards transformations employable from mapping to mapping."

Limiting Complexity

The Psychology of Ontology Harmonization "My personal experience seems to suggest the exact opposite: higher the abstraction, lower the objectivity with which people can argue and, therefore, come to an agreement.

I've spoken to people that spent several years of their lives coming up with an ontology and their perception is that the complexity over time of these models to cover a particular domain saturates, does not continue to grow.

This is a basic, but vital assumption for this entire approach to work: if the ontologies grow linearely with the amount of information they can describe, the ontology creation/maintainance process simply won't scale globally."

Intellidimension's SWS

Semantic Web Search "Semantic Web Search is a search engine for the Semantic Web. Our site can be used by both people and computers to precisely locate and gather information published on the Semantic Web."

From the FAQ:
"...using our standard search engine interface you can just type a one or more of keywords describing the information you are trying to locate. This is no more complicated than a traditional Web search engine. However like a traditional Web search engine this can lead to a large number of irrelevant results. To narrow your search you can restrict it to the specific type of resource that you are trying to locate such as a person (FOAF Person) or news article (RSS Item). If your search is still producing a large number of irrelevant results than you can refine it further by specifying one or more specific property values that the resource must have."

Browse the Semantic Web

How to Make a Semantic Web Browser "Two important architectural choices underlie the success of the Web: numerous, independently operated servers speak a common protocol, and a single type of client—the Web browser—provides point-and-click access to the content and services on these decentralized servers. However, because HTML marries content and presentation into a single representation, end users are often stuck with inappropriate choices made by the Web site designer of how to work with and view the content. RDF metadata on the Semantic Web does not have this limitation: users can access the underlying information and control how it is presented for themselves. This principle forms the basis for our Semantic Web browser—an end-user application that automatically locates metadata and assembles point-and-click interfaces from a combination of relevant information, ontological specifications, and presentation knowledge, all described in RDF and retrieved dynamically from the Semantic Web."

Some thoughts on RDF rendering had some ideas on visualizing the Semantic Web too.

The Passion of RDF

"SemanticBible is ...an emerging exploration of new applications of markup and computational linguistic technology to the study of Scripture, with an emphasis on practical tools that encourage understanding and personal transformation."

See also: The Vision of a Semantic New Testament: "Just as important as avoiding commercial barriers to sharing is the requirement that SemANT support existing and emerging standards that enable use across the Internet. To this end, SemANT will build on the Semantic Web Activity of the World Wide Web Consortium (W3C), including XML as a syntactic standard for data interchange, and RDF for ontology-based representation, and DAML/OWL for additional semantic expressiveness.".

Another example of when your technology has matured like: PCs, CDROMs, Hypertext, the Web, etc.

XUL vs XAML

Mozilla, Gnome mull united front against Longhorn "So far, XUL has failed to catch on, and Microsoft questioned whether Mozilla's technology would do much to help Gnome ward off Longhorn's promised threat.

XAML, Microsoft warned, is more potent than XUL in its ability to reflect exactly what's in the operating system.

"XUL is not the multipurpose declarative language that Gnome probably wants," said Ed Kaim, product manager for the Windows developer platform. "People say that when all you've got is a hammer, everything looks like a nail. In the same way, people are trying to figure out how to crush XUL into an OS it really wasn't designed for. The browser is great for a lot of things, but when it comes to robust client side applications, it's not the best."

Another trick will be in reconciling XUL with Gnome's existing user interface technology.

"There are ways to marry them," said Bruce Perens, an open-source consultant who serves as executive director of the Desktop Linux Consortium, a marketing organization. "But it's very difficult to get the two teams working in the same direction. They both went on a several-year tour of technical creation where they sat down and created everything they needed to do GUI [graphical user interface] applications — and they didn't create the same thing. Now to get them together it would take some number of years to resolve the technical diversions.""

Query Use Cases

Query Languages Report "A report by AIFB and Sesame and Jeen Broekstra from the Sesame crew. The Authors know what they are talking about as they are SemWeb developers themselves.

Although a little self advertisement and some missing languages, its a good thing to read. If you need info about RDF Query languages, read it.

My previous demand about "optional joins in queries" is answered by SeRQL."

The report is an excellent example of the current features required from a query language. From what I can tell iTQL implements 11 of the 14. Some of the others are fairly trival to add support for (like the data type support).

Will the Semantic Web Scale?

Apparently, there's going to be a debate at WWW2004 about whether the Semantic Web will scale: "However, with only a few exceptions we noticed that current research and development is focusing on creating new technologies for facilitating the Semantic Web. Available technologies from other disciplines such as databases are rarely reused and adapted. Hence, most Semantic Web systems do not scale to Web-size problems.

Lately, several researchers doubted whether the Semantic Web idea will ever scale for numerous reasons technological [But03], theoretical [van02] and practical [MS03,Sow]. Dedicated workshops on that topic [CKDE03,VDC03] have been organized recently to promote research to improve scalability. We will pick up these three categories of doubt by organizing the panel in three parts discussing each aspect: theory, technology/implementation, and practise."

I'm not sure I agree, there are very few Semantic Web systems that don't reuse existing SQL databases - they just suck at storing triples. With Kowari, and I'm sure with other native stores, the data structures and techniques used are taken directly from databases. They mention "Is the semantic web hype?" (which I responded to) and a few others. Although, there's no links to syllogism, metacrap or gnomes. BTW, I'm still not sure why you'd want an XML version of OWL.

"Network round-trips are often considerably less costly than the time taken for a transactional database operation due to the need to forcibly log transactional operations which is very costly in terms of disk performance. i.e. network round-trips aren't always the performance bottleneck." From Martin Fowler's First Law of Distribution.

As long as you keep the Semantic Web like the Web there's no real reason why it shouldn't scale.

Google Watching

What can't you find on Google? Vital statistics "Google is famous for being a confident, open company. Its clean, uncluttered search page is supposed to be a metaphor for the organisation behind it. But when you start asking questions about its technology, then the water rapidly becomes murky...One university presentation, for example, claimed that Google handled 150 million queries a day, and 1,000 per second at peak times...If the system is handling a peak load of 1,000 queries per second, he reasoned, that translates to a peak rate of 86.4 million queries per day - or perhaps 40 million queries per day...They also claim to have '4+ petabytes' of disk storage, and have let slip that each server is fitted with two 80 gigabyte hard drives. Now a petabyte is 10 to the power of 15 bytes, so if Google had only 10,000 servers, that would come to 400 Gb per server. So again the numbers don't add up."

Google Goes Public? The Rich Get Richer "People speculate. People dream. And if the numbers are to be believed, people will drool. The current prediction is that Google, if it decides to sell shares to investors this year, would probably end up with a market value of $20 billion to $25 billion by the end of its first day as a publicly traded company."

Google's Brin Talks on Gmail Future "It was interesting to me that you did finally hit on the word conversation. It seems to me that there's a synergy between the elements of the conversation in the RSS space and what you're doing in the e-mail space.

I think that's very true. Part of the things we've seen why blogs and RSS feeds are such a success is that you can actually read it—you don't have to stop, click back and forth, collect bits and pieces here and there—but it is all presented to you as one. "

Mozilla to Upgrade RDF

RDF module owner "With Benjamin Smedberg, Chase Tingley and Ben Goodger as peers, I took over the module ownership on RDF. We gonna push for both standards conformance (there are new specs out there since early 2004) and scriptability for remote web applications. This will include some serious whacking of the RDF API in Mozilla, as that is not ready for the web by a fair amount."

Ant is now more useful

ANT's finally a real build tool "And I can finaly call ANT a real build tool (and Maven can go play in its own cacca for all I care).

In a nutshell, the task lets you reference other build files. This means that you can create common centralized libraries of build files that other people can use on their own projects - all without copy and paste. And believe it or not, the semantics all make sense too. You can provide default tasks and properties, and the importer can override tasks and properties to customize behaviors on a case by case basis if it's required. The end result is that individual project build files are smaller and easier to understand, and common behavior can be achieved across an entire large system in a natural and non-cut-n-pasty manner (I don't know about you, but I always found pasties rather unnatural)."

Also new is macrodef: "Macrodef is a way to define a new Ant task in an Ant build itself. Macrodef allows you to define standard tasks that have attributes and elements given to them when they are called."

Paul's Blog

Paul is a guy that sits next to me at work and now is putting his development notes on a blog. So if you're interested in the inner-workings of Kowari take a look. I'll let people guess who DM, AM, and RMI are. :-)

Free Bits of Description Logic

DESCRIPTION LOGICS Includes a bunch of Postscript and PDF files (including the first couple of chapters of the DL Handbook).

Metaweb Graph Updated

New Version of My "Metaweb" Graph -- The Future of the Net "Many people have requested this graph and so I am posting my latest version of it. The Metaweb is the coming "intelligent Web" that is evolving from the convergence of the Web, Social Software and the Semantic Web. The Metaweb is starting to emerge as we shift from a Web focused on information to a Web focused on relationships between things --- what I call "The Relationship Web" or the "Relationship Revolution.""

Semaview Interview

What Is The Semantic Web? "The Semantic Web provides the foundation on which we can build more intelligent Internet applications. It will help everyone find, organize, collect, use and share information more easily."

"My company, Semaview has developed an application called eventSherpa. eventSherpa is making it simple to create and organize schedules and share them over the Internet. Our application automatically creates Semantic Web content transparently without the end user knowing it...Aside from reducing the complexity issue...I believe the largest challenge is convincing application developers to make their data available in semantic format. However it is "a chicken and egg problem" -- the more content available in a semantic format, the more applications that will be developed to take advantage of it; and vice versa."

Knobot

Reto was trying to get Kowari going. Hopefully it will get used for this project. Documentation gives download links and installation instructions.

From Danny Ayers: Knobot PlanetRDF Demo.

XML 2004

The State of XML "As a software developer I feel increasingly unhappy with the development of a monolithic mass of technology building up, only reasonably accessible behind a Java or .NET API. In contrast, the REST model of composed, simple interactions seems more controllable and containable and you can still see the angle brackets in order to check that things are working. There is still plenty of work and experimentation to be done yet with the notion of more document-oriented web services."

"Consequently, even at the low level of operating systems vendors are seeing the need and advantages of implementing metadata storage and manipulation.

This is good. We have the tools to support this, whichever way you swing on the technology issues. RDF & OWL, Topic Maps, W3C XML Schema: all have the right machinery. Unfortunately that's not the biggest issue. The main problem is which terms, schemas, and ontologies to use. That's just not clear right now for most if not all metadata applications. At best, we'll get inconsistently classified information, which defeats the promise of interoperability. More typically, we'll end up with little tagged metadata and islands of de facto proprietary information."

"As an RDF fan, the realization of this truth causes me some pain. The way out is to stop thinking of RDF as an XML application, and look to easier syntaxes such as Turtle and N3."

Unix Job Ad

Unix Specialist "Ah, Unix. Its cheapass cousin, Linux, is what all Microsoft users turn to just as their sanity reaches a crossroads. Did you know that Microsoft Word stills spellchecks ‘Unix’ as ‘UNIX’? Man, how 80’s does that look? I can imagine something like that flickering on the screen of a computer you assembled yourself from a crystal radio kit."

"But the point of all that is, Unix is basically a sort of secret society where you either know it, or you don’t. And since most people just really can’t be bothered going through the agonies of learning it, it’s why we have jobs like this: “Unix Specialist”. Of course that means nothing, or at least it means about as much as “Car Specialist” or “Bread Specialist”. Bread Specialist? What the hell is that? What kind of bread? White, multigrain, mixed grain, wholemeal, sourdough? Sliced or unsliced? If sliced, sliced for sandwiches or for toast? Crusty or soft? No matter! Just eat your bread!"

RDF Engine

RDF Engine "The program RDFEngine was developed as a part of the master thesis of Guido Naudts. It was build on the example of Euler, the program of Jos De Roo. The original version was made with Haskell. It was then rewritten in Python. Purpose of the program was to implement a logic program for the Semantic Web initiative. Concerning compatibility the program is meant to be compatible with CWM in the sense that sources that work with CWM will also work with RDFEngine but not vice versa.(I like to do some experiments of my own (-: ). For input and output Notation 3 is used. See also the Notation 3 tutorial."

80/20 REST to SOAP

XMLEurope, Monday "The keynotes were by Jeff Barr of Amazon and Steven Pemberton from W3C. Interesting to hear some detail about the Amazon Web Services, including the 80/20 split between developer use of REST and SOAP interfaces to the Amazon catalogue, and the business case for making the catalogue available in machine-processible format: essentially the value is not in their catalogue per se, and they have a business model for third party sellers and affiliates, so WS access just makes this relationship easier. The output example he gave looked to me very close to RDF; with Amazon's XSLT service it could be transformed to it very easily I think."

Pointless

Netscape Desktop Navigator The thing about a circular interface...it has no point...

Phew

Kowari 1.0.2 is now released - much better than the last one. Even available in a bite sized 14MB version. Unfortunately, we didn't get the anonymous node bug in our Jena implementation fixed in time. The next release should be within the next 2-4 weeks depending on the progress of the currently outstanding bugs.

BTW, you can now do things like: "select $s $p $o from <http://www.w3c.org/2000/08/w3c-synd/home.rss> where $s $p $o ;". You can combine local and remote (via file or http) models by using "and" and "or" in the FROM clause.

Bloom Filters in Social Networks

Building a Bloom Filter in Perl "One drawback of existing social network schemes is that they require participants to either divulge their list of contacts to a central server (Orkut, Friendster) or publish it to the public Internet (FOAF), in both cases sacrificing a great deal of privacy. By exchanging Bloom filters instead of explicit lists of contacts, users can participate in social networking experiments without having to admit to the world who their friends are. A Bloom filter encoding someone's contact information can be checked to see whether it contains a given name or email address, but it can't be coerced into revealing the full list of keys that were used to build it. It's even possible to turn the false-positive rate, which may not sound like a feature, into a powerful tool."

"If any one of the filters is intercepted, it will register the full 50% false-positive rate. So I am able to hedge my privacy risk across several interactions, and have some control over how accurately other people can see my network. My friends can be sure with a high degree of certainty whether someone is on my contact list, but someone who manages to snag just one or two of my filters will learn almost nothing about me."

"Additionally, you can combine two Bloom filters that have the same length and hash functions with the bitwise OR operator to create a composite filter."

Save Our Software

Save Our Software "During the Internet boom, the traditional companies most obviously affected were bricks-and-mortar bookstores and travel agents...But in 2004, the businesses under direct financial assault by the broadband consumer Internet are not toothless independent bookstores. Instead, they are the major music labels, movie studios, and broadcasters."

"First, the FCC is conducting a proceeding to set the guidelines for what it calls 'software defined radio...in the near future, it will be possible use spectrum more efficiently and to increase competition in a space that's now the sole domain of incumbent operators. But only if we keep the FCC from regulating these technologies."

"Second...the FCC generally approved a mandate called the 'broadcast flag,' which would require that digital TV broadcasts include an anti-theft code to prevent consumers from recording programs off the air."

WebFountain posting

WebFountain, the Long Version "So how does WebFountain make answers to such complex and specific queries possible? Short answer: A lot of hardware and a shitload of metatags. Longer answer: WebFountain does more than index the web, then serve up results based on keyword matches and some clever algorithms. Sure, it indexes the web, but once the pages are crawled, WebFountain goes several steps beyond consumer search engines, classifying those pages across any number of crucial semantic categories. (Yes, IBM is active in the semantic web conversation, and has published several specs on this in the public domain). Using natural language and machine learning technology, along with a host of structured data cross-references (such as public company databases or, perhaps, a client’s proprietary database of industry terminology), WebFountain basically re-structures the web, making it accessible to a client’s queries."

"As I mentioned earlier, IBM’s model for WebFountain is platform-based. Assuming they can pay the freight, most anyone can develop for it, using a standard API that leverages simple web services. IBM won’t disclose most of its customers, but two it will mention are Semagix, which has a (pretty damn frightening) money laundering application, and Factiva, which has developed a “reputation manager” - think of it as Technorati on steroids for the serious corporate marketing or legal department. (Imagine being able to find any mention of your product or service anywhere on the web and create custom filters for the context, location, date, author, and relationships attached to those mentions, in near real time.)"

History of Treemaps

Treemaps for space-constrained visualization of hierarchies "During 1990, in response to the common problem of a filled hard disk, I became obsessed with the idea of producing a compact visualization of directory tree structures. Since the 80 Megabyte hard disk in the HCIL was shared by 14 users it was difficult to determine how and where space was used. Finding large files that could be deleted, or even determining which users consumed the largest shares of disk space were difficult tasks.

Tree structured node-link diagrams grew too large to be useful, so I explored ways to show a tree in a space-constrained layout. I rejected strategies that left blank spaces or those that dealt with only fixed levels or fixed branching factors. Showing file size by area coding seemed appealing, but various rectangular, triangular, and circular strategies all had problems. Then while puzzling about this in the faculty lounge, I had the Aha! experience of splitting the screen into rectangles in alternating horizontal and vertical directions as you traverse down the levels. This recursive algorithm seemed attractive, but it took me a few days to convince myself that it would always work and to write a six line algorithm."

I especially liked PhotoMesa.

RSS on the TV

NewsGator Media Center Edition Provides Access to Syndicated Content on TV Sets; 'Living Room' Interface Allows Users to Read Selected Content, or Watch On-Demand Video Content "NewsGator Technologies launched NewsGator Media Center Edition today, which allows users to read syndicated content feeds on their TV with Windows XP Media Center Edition. Both text and multimedia content is supported, with an interface designed to be used with a remote control from across the room. NewsGator Media Center Edition shows information that has not already been viewed on another device by synchronizing user subscriptions with NewsGator Online Services."

You know your technology has reached maturity when someone tries to put it on the television. A good followup posting with links to screenshots.

DSpace presentations

DSpace User Group Meeting 2004 Presentations A bunch of Powerpoint presentations on DSpace.

42...What was the question?

Abracadabra, 42, Curator "The Hitchhiker's Guide to the Galaxy,1 those afflicted with 42 fever argue that "UML" is actually the correct answer. The classical symptom of those afflicted with 42 fever in the sphere of software engineering is to have an a priori delusion that UML is the solution for all software-engineering problems."

Part of a larger set of articles on UML Fever.

"The entertaining tenor of "Death by UML Fever" should not be inferred to suggest that UML fever is an imaginary ailment. It is genuinely real, it is thriving, and its presence is causing cost and schedule trauma on many software programs right now. Furthermore, the root causes of this fever, in general, have nothing to do with the UML itself: Rather, this fever and its various manifestations are largely symptoms of deeper ills in an organization's software development practices. Software organizations should consider launching self-diagnosis campaigns to assess the presence or extent of UML fever on their programs and plan rehabilitation strategies as necessary. Developing good software is a difficult enough task without having to endure the preventable and often painful complications of the dreaded UML fever."

We all Agree

As We May Hack "I can walk into any meeting anywhere in the world with a piece of paper in hand, and I can be sure that people will be able to read it, mark it up, pass it around, and file it away. I can't say the same for electronic documents. I can't annotate a Web page or use the same filing system for both my email and my Word documents, at least not in a way that is guaranteed to be interoperable with applications on my own machine and on others. Why not? - A Manifesto for Collaborative Tools

Yawp after me - "RDF! RDF!". But seriously, the answer to "why not?""

From Yawping!. The collaborative tools manifesto has been mentioned several times recently it's worth a look.

Another RSS library for Java

RSSLib4J is a set of Java API to parse and retrieve information from a RSS Feed. It supports RSS version 0.9x ,1.0 and 2.0 specification with Doublin Core and Syndication namespace.

Querying Use Cases and Features

Data Access Working Group User Cases: WORKING DRAFT "Because there are no formal standards in these areas, developers in industry and in open source projects have created a wide variety of query languages for RDF data...These languages lack both a common syntax and a common semantics; there is, in fact, a wide variety of semantics: from declartive, SQL-like languages, to path languages, to rule or production-like systems. The existing languages also exhibit a range of extensibility features and builtin capabilities, including inferencing, distributed query, and domain-specific semantics."

Some of the features being considered include: optional triples, disjunction, queries with paths of length two or more edges, able to indicate whether the query response includes entailment from the graph or treats the graph as a fixed object, expressing arbitrary RDF datatypes, queries expressible as a URL, user specifiable query result formats and query results in RDF (closure).

Librarians and the Semantic Web

People at work think that I have a fetish for girls in glasses or something I think. But actually I think that librarians are going to be an important part of the Semantic Web. Oh and chicks in glasses are dead sexy...err...anyway a recent posting "thinking about the semantic web" links to a series of postings along these lines (librarians not glasses):
* Extreme! "I want people to leave understanding that the Semantic Web didn’t come out of nowhere, that there’s a good century (at least!) of work coming to terms with infoglut that Semantic Web technologies build on. I want them to start thinking critically about website information architecture, about information management in general, about metadata, about problems with (and hopes for!) search engines.",
* More on metadata "Well, I have news. Everything Cory says is true. But it doesn’t matter. Hear me? It doesn’t matter. Because all those arguments aren’t arguments against a Semantic Web—they’re arguments against a populist Semantic Web.", and
* Metadata and authority "This means, quite simply, that a lot of current Semantic Web pipe dreaming will founder on authority and representation-convention problems. Heck, it’s already foundering."

One day, when I categorize this blog there will definitely be a "librarians" section.

SASDO

Semantically Adapted Service Data Objects "...is a prototype extension to the base Service Data Objects implementation provided in the ETTK. Specifically, it adds the ability to use RDF and the W3C Web Ontology Language (OWL) as a means of attaching a semantic metamodel to an SDO DataGraph and use that model to navigate and query the DataGraph independent of it's structural model, without requiring any changes to the SDO API or running any transformations on the data."

From GRDDL.

Gnowsis

Gnowsis is a project lead by Leo Sauermann. I've read about it before but apparently never blogged it. Many of the ideas like the Semantic Desktop are worth noting. This type of application is something that seems to be coming up again and again.

He gave a list of features to add: optional triples, update language, protocol on the web and a protocol on the desktop (ODBC like). The first two are handled by iTQL although the second is more elegant than first. Kowari/TKS has always had a driver. The Kowari Lite client, currently the executable iTQL jar, is only about 1MB and doesn't include things like Jena.

why I love Patrick Sticklers URIQA approach "Building RDF aggregators is not an easy task. I have tried it in many different ways, and had varying forms of success. If you want an easy approach that does it, think about concise bounded descriptions."

I do think that the resolver idea, using the FROM clause, views and defining caching and updates, is going to be an easy way to do RDF aggregation.

Regime Change

U.S. Terrorism Policy Spawns Steady Staff Exodus "Since the Sept. 11 attacks, the Bush administration has faced a steady exodus of counterterrorism officials, many disappointed by a preoccupation with Iraq they said undermined the U.S. fight against terrorism."

""I'm kind of hoping for regime change," one official who quit told Reuters."

""Iraq has been a distraction from the whole counterterrorism effort," said the former official, adding the policy had frustrated many in the White House anti-terrorism office, about two-thirds of whom have left and been replaced since Sept. 11."

"Roger Cressey, who served under Clarke in the White House counterterrorism office, said: "Dick accurately reflects the frustration of many in the counterterrorism community in getting the new administration to take the al Qaeda issue seriously.""

Oh and for something cool with photo-mosaics:
War President "Below is a small version of an image I made. It's a mosaic composed of the photos of the American service men and women who have died in Iraq. No photograph is used more than three times." From here.

If it's broken:
http://randomfoo.net/junk/200404/warpresident/warpres.jpg

An OS tool to do mosaics (includes Tux mosaic - I thought it was something else when I first looked at it):
http://www.stud.uni-hannover.de/~michaelt/juggle/

Is WiX a trick?

MS Open-Source Move is Straight from Playbook "Looking more closely, WiX enables developers to translate programs from Windows Installer Databases (.msi/.msm) formats to a text-based, XML-file format. XML is an open standard, but to work with MSI/MSM, those XML files have a very specific format. Now, what company has already sought patent protection for specific expressions of XML code? The answer is, of course, Microsoft, with its Office XML formats.

Has Microsoft done this with WiX's XML formats yet? I don't know. But if the pros from Redmond haven't yet, they will. They did it for Office XML document formats; they'll do it for this. Thus, Microsoft's open-source code will work only on Microsoft-proprietary XML to produce Microsoft-proprietary installation programs. With open source like this, who needs proprietary programs?"

"Now, while it may look like Microsoft is doing something new, or perhaps even something helpful to the open-source community, it's not. What Microsoft is really doing is putting more of the Halloween memo's plans into action. Why shouldn't it? The Halloween plans are just an elaboration of Microsoft's time-tested embrace-and-extend technique The only embrace Microsoft is really giving the open-source community is a stranglehold. "

The Power of Free

The Profitability of Free Code "Some thought that open source meant non-commercial or even anti-commercial.' At the time, I wondered how far open source could go and in that posting I identified three criteria for success with open source:
• the user community must be vast
• the product scope must be well defined
• a viable business model must exist.

Today, I think that people are starting to understand that open source is more than just a development approach—it's a business model. What evidence is there to support this? Just follow the money: the OSBC conference was packed with venture capitalists and lawyers; techies were outnumbered ten-to-one. Even long time veterans of the closed source software world, like Ray Lane (previously of Oracle) and Chris Stone of Novell are focused on leveraging the power of open source in their businesses.

And why not? That's the nice thing about capitalism. It has a built-in Darwinian efficiency. If open source allows companies to deliver better products at a cheaper price, then it will be used to do just that. We see many of the most competitive companies in the world, like Amazon, Charles Schwab, Cisco, Corporate Express, FedEx, GE, Merrill Lynch, Motorola, Nokia, Sabre, SAP, and UPS using open source software as a platform for building new applications. Many companies that have been successful with Linux are now looking at deploying an entire open source stack, known as LAMP (Linux, Apache, MySQL, PHP/Perl/Python).

These days, it's easier than ever before for an ambitious developer to download all the software he or she needs to build a product—or even to build a business—on top of commodity hardware and open source software. Some of today's hottest pre-IPO companies, including billion-dollar-babies Salesforce.com and Google, as well as emerging social networking companies like AlwaysOn's Zaibatsu, Friendster, and Tribe.net, are all using open source software."

Kowari 1.0.2 Soon

We've fixed all the known bugs; that just leaves the unknown ones. We've got JRDF, Jena, and our own RDQL implementation. The loading and query speed has also improved considerably - there's none of the problems associated previously with 32 bit systems. Kowari lite is down to 11MB; which considering that Jena and related jars are over 2MB that's pretty good. Querying is now totally disk backed - you can do a completely unconstrained query over arbitrarily large data sets.

It's now at the stage of what to do next. Some of them are fairly obvious like full data type support, scripting (possibly Groovy) and inferencing. But we've also got zeroconf and MAYBE. There's also a large refactoring underway to make Lucene, the triple store, ontology models and others as plugins exposed via a resolver interface. This should allow just about anything to be queried in the FROM clause of iTQL. In the short term, this means extending the querying of file and http RDF sources to include other metadata formats such as MP3 tags, iCal, vCard and XML based RSS. It will allow models to be typed and configured. Using views you could easily create an RSS aggregator. Other, more difficult resolver implementations will be available in TKS.

JRDF has been a little neglected. The next things to be worked on is transactions and a ResultSet like interface.

Your Life in a Wal-Mart Database

What to do with all that information "[Wal-Mart] the largest in the world, now also maintains the world's largest database, 280 Terabytes, as it seeks to better understand how to efficiently serve its customers. For instance, it increasingly uses Radio Frequency ID tags -- tiny 25-cent data-holders affixed to a box of razors and other goods. Wal-Mart uses RFID to track its inventory through much of the supply chain. Soon the tags will only cost pennies and start going on the razors themselves. Talk about increasing the amount of data to store and analyze! (I listened in the cool new $10 shirt I bought Sunday at a local Wal-Mart store.)"

"And in the not too distant future, data may be able to interact with other data without human intervention. That was the message of Web inventor Tim Berners-Lee. He talked about the glorious future of something called the "semantic web," a concept that most people aside from programmers will probably never understand. His talk here didn't help. One hopeful journalist from the Economist asked Berners-Lee to give an example of how companies could make or save money using it, but he didn't have an answer. He doesn't have to. He's an academic. Businesspeople here assured me this is a big deal we'll hear more about. "

I thought I'd read this before but couldn't find it in a previous entry. Although I did find a previous claim that the biggest database was France Telecom's at 30 TB.

Two Quick Projects

* ruby-rdf Very timely. "A project to advance ruby in the semantic web realm. Specifically, this will be an attempt to port the Redland RDF framework to ruby in an object oriented manner."
* ROWL "The system enables users to frame rules in RDF/XML syntax using an ontology in OWL. Using XSLT stylesheets, the rules in RDF/XML are transformed into forward-chaining defrules in JESS. We make use of two more stylesheets to transform ontology and instance files into Jess undordered facts..."

Chimney Sweeps

The Next Job for the W3C "People will start to feel that when they access some data that they didn’t really think was relevant, they hadn’t thought was relevant to the problem at hand, and suddenly, their problem gets connected to some other things."

"To answer questions within a company, you’ve got to reach across boundaries. You’ve got stovepipes of information. You have HR information. You have totally separate information for bug reports and customer satisfaction.

You might have another, for example, for documentation. We find that we were tracking a customer’s problem, but most of those documents left the company, and when we left the company, we should have kept them. So let’s find out who else has written documents which are being used by people who are tracking bug reports for processing customers who have this problem.

At the moment, what we’re looking at for this is huge monster applications like the pre-Web, which are document databases. That didn’t work because everyone wanted to operate it a certain way. It’s much better to let each part of a company have a Web site, and they’re linked to each other. Let the financial folks organize things how they want, and let manufacturing control people organize their things how they want, and then merge them. Where they have things in common like people or times or places—they have to be able to connect them."

Lisp

Better is better: improving productivity through programming languages "This is one reason why Lisp and functional programming folks are a bit cool on stuff like Object and Aspect orientation. They can extend to Aspects or Objects using the language constructs directly. To most of us working in popular languages these are paradigm shifting ways of thinking about programming. But we can't use Java directly to support something like aspects or generics. There's whole communities working on language and compiler extensions, open source libraries, as well as that all important vendor support - J2EE app servers are being renegineered just to support aspects. To a Lisp person these are just constructs expressed as macros (which are written in Lisp itself), something you do on a rainy weekend. What's the fuss about?"

"It's something a running joke in the Lisp community that all language efforts are doomed to reinvent Lisp."

"It's corny to say the worst thing about Lisp is a Lisp programmer. But there's some truth in it. Work long enough in this industry and you'll run into a Lisp bigot...I think Lisp never traditionally emphasized libraries because the language is so flexible you can build one on your own, but then you have the twin nightmares of integration and reuse lying in wait for you."

I have been thinking about Lisp and aspect programming recently. The other is ontological programming or programming against an ontology.

Blast from the past

Another catch up post:
* Ercim News posting in 2002 mentioned both Corese and Notio. Their API uses strings to define concepts, relations and attributes. Apparently, it's now available for download (via PlanetRDF). It uses Jena's ARP but has its own CG interface.
* Sleepycat for Java Released under the same licence as the C version.
* Platypus Wiki Again, using Jena and looks good.
* Not in RDF The difficulties of NOT in RDF. I think Kowari/TKS lacks NOT because it was initially hard to do in our implementation; there's now no reason not to do NOT.
* Fixing the Java Memory Model, Part 2 Includes double checked locking, changes to volatile and initialization safety.
* Blog-Bleary? Try (What Else?) a Blog on kinja. Another review.
* RSS Versions Syndic8 is reporting that RSS 1.0 is now 46.4% of all RSS feeds with RSS 2.0 and 0.91 (combined) taking up 49.9%. Will RSS 1.0 take the lead soon?