Why the Cloud isn't the Internet

I think people are just starting to realize what some of the cloud vendors are providing and their drawbacks.  Steve Jobs is quoted in his biography describing the intent behind iCloud:

We need to be the company that manages your relationship with the cloud - streams your videos and music from the cloud, stores your pictures and information, and maybe even your medical data...over the next few years, the hub is going to move from the computer into the cloud...So we wrote all these apps - iPhoto, iMove, iTunes - and tied in our devices, like the iPod and iPhone and iPad...We can provide all the syncing you need, and that way we can lock in the customer.

The Mac has always been different to Windows.  One of those differences Windows users' notice is that you switch between applications in OS X compared to documents (or windows) in Windows. The Apple cloud maintains that pattern by syncing between applications rather than documents (or individual files).  This approach confuses a lot of people.

This is different to how most Mac users currently sync their files with Dropbox.  iCloud has ended up following its Mac heritage whereas Dropbox sticks to file syncing.  Matthew writes:

The difference between Dropbox and iCloud synchronization is that Dropbox is theoretically just a file system...If you have a document that you edit on your iPad and sync with Dropbox you can edit that same file, using a different application, on your PC...The iCloud experience is completely different. The only way to edit a document across platforms or devices is to use a version of the application for each device. Not a compatible application...it may actually make me change the desktop application that I use purely based on iCloud support.

If you want to read more about Dropbox and Apple there's a really good article in Forbes which details how Steve Jobs personally made an offer to buy Dropbox.

The edges of iCloud - the integration points to applications and the operating system - it's incomplete even if you buy into the idea of applications over documents.

For example, on iOS devices there is a Notes application but on OS X these notes are in a tab in the Mail application.  This seems like a weird and non-standard place to put it - if you are going to sync by application you'd think it should be the same application across platforms.

In iCloud for Windows, Windows users get more choice than OS X users.  Mail, Contacts and Calendar integrate with Outlook but you can choose your application for Bookmarks (IE or Safari) and Photo Stream.

Even within applications Apple haven't quite gotten syncing right with iCloud yet either including the new rules around where files are stored and what is automatically removed or backed up.

The cloud is about vendor lock as much as any other platform, like application servers or databases, but with the extra problem that your data is tied to the vendor's application, cloud and user base.  A stickier solution.

Some, like Google and Facebook, offer export services, but these almost don't matter, because you get an almost useless hunk of data, lose the ability to run the applications and you can't access users on their network (who may well have been collaborators).

With the Internet, the Web and open source you still have the possibility to use your data with applications shared by many people across different networks.

Global Code Retreat 2012

I went to the local Global Code Retreat held on the 3rd of December.  Overall, it was an amazing event - very well hosted and attended.  The basic structure of the day was 6 or so 45 minute sessions trying to implement something with a different person each time.  At the end of the 45 minutes, no matter how far you had got, you deleted your solution.

The problem was "The Game of Life".  I'm pretty familiar with this problem having come across "Conway's Game of Life" early on in a magazine like Compute! or Byte.

However, if you walked away with a really awesome solution to "The Game of Life" you probably missed the point - most of the things that were being taught were hidden.

The solution was really beside the point.  One of the main reasons is to repeat solving the problem from scratch based on an idea called kata (movements practiced by yourself or in pairs).  This is something that I had come across in "The Pragmatic Programmer" which at the time reminded me of the time I had spent with projects at home - reimplementing the same thing over and over again.

Steve Yegge mentions the same thing in his article "Practicing Programming".  He mentions that even as you program in your day job you may not actually be practicing programming.  Repetition in solving the same problems seems to be about keeping the problem fixed and then changing how you approach it and freeing you from any time constraints.  Most programming jobs involve solving the solution once (or if you're lucky doing a proof of concept and then implementing it again).

The first time around it was awful.  I didn't know what I was doing, my environment was a little bit shaky, we couldn't agree on a language and I spent a lot of the time just setting it up.

It made me become aware that for the first time, practically ever, my personal computer had diverged from my work computer.  Not in the "normal" Windows at work, Linux and OS X at home - but what I do at home and at work have diverged to the point where I'm learning stuff in many directions and there's almost no overlap between the two.

The second time was much better.  There was less discussion on languages to use, how to approach the problem, how do you test drive it, whose computer to use and so on.  There was still discussion but we both shared a bit more context this time which made the discussion flow.  A big difference to the first time.

The third time around changed the format a little to where you couldn't talk to the person but you could only express requirements through tests.  So this sorted out the people who were testing from those who weren't.  But it also seemed to reduce the clutter around what needed to be done.  Tests are much less ambiguous compared to talking through requirements and so once you setup a rhythm of tests it became much easier.  Also, the whole room was very quiet.  You could imagine that a team doing silent TDD and pair programming wouldn't be the noisiest group in the room (for once).

Each round thereafter changed the programming requirements: no loops, methods no more than 3 lines, and no if statements.

What did I learn?  Heaps.

I ended up doing Ruby quite a bit and mostly the solution came out at about 30 lines of production code and 30 lines of tests and you could pretty much do it in the time allocated.  I also did solutions C# and Haskell.  The Haskell solution came out at about 30 lines total - both tests and production code - and met every constraint (no loops, small functions, no if statements).

Doing the same problem over and over again is surprisingly effective and nothing replaces sitting with a person to learn a new language or to be exposed to a variety of solutions.  One of the tricks - and you find this frequently with pair programming - you have to be very good at communication - both by saying what you're doing but also getting the other person to explain themselves.

I also learnt:

• Even with something as well defined and familiar as "The Game of Life", the solutions were varied and some of the requirements (based on the rules on Wikipedia) were redundant.
• My brain is very weak compared to how well Google search works.
• Between each new attempt you tend to reflect on each previous solution and see the negatives and positives.
• By continually starting a new project setup time was greatly reduced - dependencies that get in the way were slowly reduced - editors, libraries, searching the web, etc.
• That it's good to throw code away.  It frees you up by allowing you to try different approaches or learn something new (like a different language).
• Think before you hack.
• A functional approach seemed to be where the answers were converging - meeting all the programming constraints that were given.

A Review of QI Live

When Douglas Adams visited Brisbane in 2000 (possibly 1999) I had a friend sign a copy of "Starship Titanic" - I was too busy at work to see him myself.  I have always been a little disappointed that I didn't take the hour to see him.  When Stephen Fry announced "QI Live" on Twitter I made sure I wasn't going to miss out.  It was only in Melbourne and Perth (at the time) and bugger it if I was going to Perth, so even though I didn't live in Melbourne, I got some.

As I walked into the theatre the place names of France that sound funny appeared on the screen.  I remembered them from a previous episode of the TV series.  It didn't matter, they'd just played the theme for "Pinky and the Brain" and were playing Tom Lehrer's "The Elements".  I was in a good mood and I was happy to be there.

My wife thought she saw the producer, "What John Lloyd?".  Surely not, but soon after, an usher told us to stop taking photos.  It was a little annoying because I was mainly taking photos of the theatre - I'd never been to the Queen Victoria Theatre before.  Then a prissy voice said that Mr Fry's man servant would like you all to turn off your phones - so I did.

The only bit I now remember of Stephen's opening was him telling a joke about a chicken going to a library.  It was told well, I guess, but too long for me because I'd heard it before and spent most of the joke remembering my grandmother telling me it about 20 years before.  Maybe that was what he was going for.  What was worse, it lead to the fact that a frog in California is the only species in the world to go ribbit.  Another recycled fact and I was getting a little bit annoyed.

Colin Lane was the first panelist and pretty forgettable.  He and Andrew played tennis with the "Nobody Knows" paddles which was okay the first time.  Denise Scott didn't really seem to get the format but she had some good anecdotes - such as being recognised as looking like that person on TV but that she couldn't possibly be that person.  Andrew Denton was by far and away the star - he seemingly, confused Stephen by saying that, "If nobody knows, why isn't he on the panel?"  It made the whole thing almost worth it.  Except that Stephen then went on to spend most of the evening calling him stupid.

The first question was about koalas having fingerprints that are indistinguishable from human fingerprints and that maybe they were doing all the robbing in Australia (which they said has the highest burglary rate in the world).  This fact doesn't seem to be true now although it was true in 2001 which is probably closer to when it was written.

Some of the other content that was from previous episodes included: kangaroos not farting and having 3 vaginas, when does the sun set (a video), Beatles' HELP album cover, the most popular song being the default Nokia ring tone and slavery not being illegal in the UK until recently.  This is just me guessing but Andrew Denton knew every answer.  The members of the audience shouting out certainly did.  But then so did Alan and so did I.  Well, I might not have known every question.  Alan answered over the top of the question for 100 points so I didn't hear what it was.

I might be wrong, but one of the ways the show works is that Alan doesn't know the answer to every question.  Sometimes some of the guests do (John Sessions and Rory McGrath) but the point is: Alan is the kind of guy the show is supposed to be educating - he's the audience - he's coming along with us for the ride and a laugh.  He probably did reflect some of the audience that night too though, he looked a bit bored and a bit uncomfortable.

Given the amount of recycled content I don't think they would've done this show in the UK but apparently it's okay to do this in Australia.

At the end of the show, when Stephen was awarding the points, he said either the people in the audience were smart or that they had downloaded episodes not shown in Australia from BitTorrent.  This is what made me mad.  All episodes had been shown on the ABC from series A (you can Google the series number and ABC to find them on iView like Series 4).  And some of the facts were also available in the QI books (like Animal Ignorance).  Stephen seemed to be saying that we (whoever we are - the producers of the show?) reserve the right to charge you for content that we don't think you should've seen and we're a bit surprised you have because you've really cheated yourselves out of $200 a seat.

I'm still mad.  QI has a philosophy about facts and curiosity.  Recycling facts, that are now wrong, goes against that philosophy.  The original idea for the show seems to have been going against accepting common knowledge and laziness.  The show seemed to scream, "Look it up!" or "Why do you think that?".  Recycling content was pure laziness.  Stephen and Alan knew that the content was recycled and it ruined the show or at least the idea behind the show.

In the end, it's hard to know if you should regret the things you don't do versus the things you do.

As an aside, some interesting facts (sticking with the Australian theme): all living marsupials are from South America not Australia (from Wikipedia) and echidnas and platypus can make custard (as they produce both eggs and milk).

Building a Network Over Transactions

The new meaning of customer value: a systemic perspective analyses providing value to customers from a systems perspective.

I had a thought, a little while ago, that Google is probably one of the first companies where the users and content providers are basically the same people and that they make money, through adverts, between connecting these two together using search.

It's probably not a new thought but at the time I started to draw a diagram of how it all works.  I happened across this diagram (on the left) in this paper (page 4). 

A perverse example is when you search for something and the first hit is your own blog.  You're now both the producer and consumer of the same content - with adverts sandwiched in the middle - hmm value sandwich.

In the paper they use Google and Apple as examples:

"Google has indeed realized the usability of systemic value-creation principles in building its offering. In contrast to Apple, it uses the value network to generate the revenues. Google provides free, easy-to-use tools for customers to use on the internet, the aim being to generate “eyeballs” for the ads of the advertising customers. In collecting these “eye balls” it has or it creates a product for every internet activity that attracts lots of traffic. From the firm's perspective, the offering elements are integrated to provide the audience for the ads, information being gathered in order to better scope the ads or just to make the customers happy and to promote other products."

There's an old idea, for the Web anyway, of building a network of customers above extracting value out of each transaction. Over-valuing the creation of the network lead to the whole dot com bubble and I have been thinking about how business models have progressed since then.

One horizontal and one vertical monitor for Ubuntu 11.04

I recently had problems configuring Ubuntu with dual screens using an NVidia card - the first screen is horizontal and the second screen is vertical - they are both Dell U2711 monitors. The idea is to be able to have Firebug running on the screen without obscuring the web page.

This should be a simple thing. But from what I can tell NVidia's TwinView driver doesn't support different monitor rotations (under Linux). But having X Windows it should still be easy using dual X Screens. It should just be a matter of going to NVidia X Server Settings, selecting "Separate X Screen" and then selecting "Enable Xinerama". Unfortunately for me, this caused general weirdness where the first screen was mostly black and the second screen was displayed horizontally.

The way I fixed the problem was to disable Compiz. The easiest way I found to disable Compiz was to log in using the "Ubuntu Classic (No effects)" session.

Then it was just a matter of enabling multiple XServers and Xinerama and enabling rotation (RandRRotation). Here are the bits in my xorg.conf to rotate my second monitor to the left:

Section "Monitor" 
    # HorizSync source: edid, VertRefresh source: edid
    Identifier     "Monitor1" 
    VendorName     "Unknown" 
    ModelName      "DELL U2711" 
    HorizSync       29.0 - 113.0
    VertRefresh     49.0 - 86.0
    Option         "RandRRotation" "on" 
    Option         "DPMS" 
EndSection

Section "Screen" 
    Identifier     "Screen1" 
    Device         "Device1" 
    Monitor        "Monitor1" 
    DefaultDepth    24
    Option         "TwinView" "0" 
    Option         "metamodes" "DFP-2: nvidia-auto-select +0+0" 
    Option         "Rotate" "left"
    SubSection     "Display"
        Depth       24
    EndSubSection
EndSection

When Kworkers Don't

I recently had the problem where kworker threads were taking up 100% on my Ubuntu box.

Various threads seem to suggest the problem lies with interrupts around PCI or power saving features. The thread with the answer that worked for me, "HELP !!! Zombie attack ... (kworker)"

To turn both of them off use:
noapic acpi=off

I found I only needed to turn off acpi though:
acpi=off

Put that in your grub configuration (sudo vi /etc/default/grub) and restart.

I hate not knowing why though.

End of JRDF

Many things have changed since I started JRDF in 2003. It feels like JRDF has come to a natural conclusion.

Some of them are things I've failed to do very well: get contributors, implement different file format parsers, find enough time to refactor existing bits, etc. I'm also not that interested in Java anymore (as was becoming increasingly obvious as it did have Scala in there at one point and has some Groovy DSL code in there).

The most recent change I've seen is that JSON has achieved some of what RDF was trying to do and I see it more and more in the way people use it to expose their data in a RESTful way. The tooling is less onerous and the ease of use is higher even if what you get is much less.

Also, external factors like W3C's official RDF API (for Java and Javascript) is largely the same thing but with official backing.

I've enjoyed developing it and meeting and talking to other people in other groups (especially Jena and Sesame). And of course, none of this would've happened if it wasn't for a lot of other people: Paul Gearon, Simon Raboczi, David Wood, David Makepeace, Tom Adams, Yuan Fang-Li, Robert Turner, Brad Clow, Guido Governatori, Jane Hunter, Imran Khan and Abdul Alabri and the other guys and girls Tucana/Plugged In Software/UQ.

Linkage

• Elastic Search another key/value store that allows multiple servers to cluster Lucene and uses a simple JSON API.


• Katta a project for using Hadoop and Lucene together.


• ElephantDB which uses MapReduce to produce indexes for key/values.


• How SimpleGeo Built a Scalable Geospatial Database with Apache Cassandra using Geohash (to flatten a 2d space into 1d) and Cassandra to provide a GIS database for exact match, range and proximity searching. Claims that rebalancing isn't necessary


• Google's Percolator paper about providing incremental updates to search indexes instead of using MapReduce.


• End of IPv4 or is it: IPocalypse and Vint Cerf and Geoff Huston at the recent Linx Conf Australia


• Efficient SPARQL joins, ontologies as games, IOS Semantic Web Journal


• TDD as a way of getting things done, Testing Javascript with Jasmine and The fallacy of trading quality in software.


• Brisbane Functional Users Group videos

Build Broken Images

One of the things that we encourage where I work is to visually and aurally notify you when a build has broken. I did some images a while back for this purpose.

They are:

• Lemmings (oh no)


• Lemmings (blocker - from original sprites)


• Watchman (Rorschach's Journal)


• Swedish Chef (borked build)