Showing posts with label google. Show all posts
Showing posts with label google. Show all posts

Saturday, December 31, 2011

Why the Cloud isn't the Internet

I think people are just starting to realize what some of the cloud vendors are providing and their drawbacks.  Steve Jobs is quoted in his biography describing the intent behind iCloud:
We need to be the company that manages your relationship with the cloud - streams your videos and music from the cloud, stores your pictures and information, and maybe even your medical data...over the next few years, the hub is going to move from the computer into the cloud...So we wrote all these apps - iPhoto, iMove, iTunes - and tied in our devices, like the iPod and iPhone and iPad...We can provide all the syncing you need, and that way we can lock in the customer.
The Mac has always been different to Windows.  One of those differences Windows users' notice is that you switch between applications in OS X compared to documents (or windows) in Windows. The Apple cloud maintains that pattern by syncing between applications rather than documents (or individual files).  This approach confuses a lot of people.

This is different to how most Mac users currently sync their files with Dropbox.  iCloud has ended up following its Mac heritage whereas Dropbox sticks to file syncing.  Matthew writes:
The difference between Dropbox and iCloud synchronization is that Dropbox is theoretically just a file system...If you have a document that you edit on your iPad and sync with Dropbox you can edit that same file, using a different application, on your PC...The iCloud experience is completely different. The only way to edit a document across platforms or devices is to use a version of the application for each device. Not a compatible application...it may actually make me change the desktop application that I use purely based on iCloud support.
If you want to read more about Dropbox and Apple there's a really good article in Forbes which details how Steve Jobs personally made an offer to buy Dropbox.

The edges of iCloud - the integration points to applications and the operating system - it's incomplete even if you buy into the idea of applications over documents.

For example, on iOS devices there is a Notes application but on OS X these notes are in a tab in the Mail application.  This seems like a weird and non-standard place to put it - if you are going to sync by application you'd think it should be the same application across platforms.

In iCloud for Windows, Windows users get more choice than OS X users.  Mail, Contacts and Calendar integrate with Outlook but you can choose your application for Bookmarks (IE or Safari) and Photo Stream.

Even within applications Apple haven't quite gotten syncing right with iCloud yet either including the new rules around where files are stored and what is automatically removed or backed up.

The cloud is about vendor lock as much as any other platform, like application servers or databases, but with the extra problem that your data is tied to the vendor's application, cloud and user base.  A stickier solution.

Some, like Google and Facebook, offer export services, but these almost don't matter, because you get an almost useless hunk of data, lose the ability to run the applications and you can't access users on their network (who may well have been collaborators).

With the Internet, the Web and open source you still have the possibility to use your data with applications shared by many people across different networks.

Wednesday, November 02, 2011

Building a Network Over Transactions

The new meaning of customer value: a systemic perspective analyses providing value to customers from a systems perspective.

I had a thought, a little while ago, that Google is probably one of the first companies where the users and content providers are basically the same people and that they make money, through adverts, between connecting these two together using search.

It's probably not a new thought but at the time I started to draw a diagram of how it all works.  I happened across this diagram (on the left) in this paper (page 4). 

A perverse example is when you search for something and the first hit is your own blog.  You're now both the producer and consumer of the same content - with adverts sandwiched in the middle - hmm value sandwich.

In the paper they use Google and Apple as examples:
"Google has indeed realized the usability of systemic value-creation principles in building its offering. In contrast to Apple, it uses the value network to generate the revenues. Google provides free, easy-to-use tools for customers to use on the internet, the aim being to generate “eyeballs” for the ads of the advertising customers. In collecting these “eye balls” it has or it creates a product for every internet activity that attracts lots of traffic. From the firm's perspective, the offering elements are integrated to provide the audience for the ads, information being gathered in order to better scope the ads or just to make the customers happy and to promote other products."
There's an old idea, for the Web anyway, of building a network of customers above extracting value out of each transaction. Over-valuing the creation of the network lead to the whole dot com bubble and I have been thinking about how business models have progressed since then.

Wednesday, September 24, 2008

Who Holds the Power?

On Monday I went and saw John Wilbanks talk on "Publishing in Today`s Environment" (I can't seem to find a decent URL but a good overview of some of the topics discussed is "The Open Access Interviews: John Wilbanks"). I didn't take many notes but a few things have stuck with me:
  • Libraries providing a role as repositories of data (both public and private).

  • The idea of "free as in puppy" in relation to digital curation - a great metaphor. When someone gives you a puppy it is initially free but the upkeep of it is anything but.

  • The Queensland Government has its own Creative Commons initiative and the Australian Government is following suite.

  • There's a power struggle occurring between researchers and publishers to make data, papers and the like freely available. The power used to be on the side of publishers but as the producers band together (by country, university, faculty and so on) the power is going back to them.

  • Creative Commons are continuing to fight, in cunning ways, to get back to decent copyright law.

The kind of behavior that publishers have exhibited appears to be on the way out and seems to be going the other way. This is where databases interoperate with each other, papers can link to the original data, and tools, data and papers are all part of an integrated experience. The linking and integration of papers and other artifacts seems to be a one way process, it's hard to imagine a developer, scientist, arts graduate or anyone else spending huge amounts of money and time finally wresting control from one bunch of people only to tie it to another.

This leads to the T-Mobile G1 announcement. The announcement was very underwhelming - guys speaking for the first 10-15 minutes saying what a wonderful job they had done and the guy talking about how good an experience the phone was at playing Pacman. The more intersting part is the open platform. Much like the publishing area the ability to have open access will be a massive differentiator. It reminded me a bit about the discussion on how AOL, Prodigy and the other closed networks quickly died when faced with the open web:
By contrast, the proprietary networks of CompuServe, AOL, Prodigy, and Minitel were out beating the bushes for content, arranging to provide it through the straightforward economic model of being paid by people who would spend connect time browsing it. If anything, we would expect the proprietary networks to offer more, and for a while they did. But they also had a natural desire to act as gatekeepers—to validate anything appearing on their network, to cut individual deals for revenue sharing with their content providers, and to keep their customers from affecting the network’s technology. These tendencies meant that their rates of growth and differentiation were slow.

The closed versus open network is not quite the whole picture with respect to the iPhone versus Android. While there is competition between iTunes versus Amazon, Streetview versus normal Google Maps and there will be other content battles, I don't think that's the source of the real innovation. I think it's probably the technical innovation that the online providers couldn't match that was decisive and will be for phones:
The software driving these communities was stagnant: subscribers who were both interested in the communities’ content and technically minded had few outlets through which to contribute technical improvements to the way the communities were built. Instead, any improvements were orchestrated centrally. As the initial offerings of the proprietary networks plateaued, the Internet saw developments in technology that in turn led to developments in content and ultimately in social and economic interaction: the Web and Web sites, online shopping, peer-to-peer networking, wikis, and blogs.

Is developing on an iPhone going to lead to more technical innovation over Android? Does the ability to have open source code on Android beat the Apple NDAs?

Apple will probably recognize that it's the developers that will ultimately have the power but like publishing it depends on the actions of both parties. At the moment both of these new phone platforms are a little limited - the ability for people to innovate on iPhones using iTunes, Mail is out but it doesn't seem terribly better for Android (being DRM free is a good start though). If history is any guide, it would seem to favor open development over closed.

Update: Similar more succinct explanation.

Update: No Pragamatic book due to NDA - the outrage!

Wednesday, September 03, 2008

Stats for Nerds

I read "Is google chrome under / partially reporting RAM usage ?" which seems to be the typical problem with Microsoft's task manager. A better way to check out Chrome vs others is to hit Shift-Escape, click on "Stats for Nerds" and it has a comparison there.

Wednesday, July 02, 2008

Round of Links

Saturday, April 19, 2008

Big Web Table

I thought I read someone ports Google's AppEngine to use HBase. Good idea but not quite. "Announcing A BigTable Web Service": "I then came up with the crazy idea to offer BigTable as a web service using App Engine. It would be an infinitely scalable database running in Google's datacenters. I spent my weekend learning Python and hacking together an implementation. Now I'm happy to present the BigTable Web Service. It models the API of Hbase—a BigTable clone. Now you can have simulated BigTable running atop App Engine, which itself provides an abstraction on top of the real BigTable."

What it actually does is use HBase's Thrift API on top of Google's BigTable or as they say BigTable as a Web Service (a RESTful one).

Saturday, January 19, 2008

Who Killed Functional Programming?

MapReduce: A major step backwards. Two good responses, "The Great MapReduce Debate" and "Relational Database Experts Jump The MapReduce Shark":
MapReduce is not a data storage or management system — it’s an algorithmic technique for the distributed processing of large amounts of data.

MapReduce has the same relationship to RDBMSs as my motorcycle has to a snowplow — it’s a step backwards in snowplow technology if you look at it that way.

I don’t think the authors understand distributed processing and distributed file systems when they think reduce must rely on FTP. The Wikipedia article says “each [reduce] node is expected to report back periodically with completed work and status updates,” pretty much the opposite of the “pull” DeWitt and Stonebraker criticize.


It's a fairly specious argument that more features means better and the comparisons are not equivalent - it would be better compare BigTable or HBase with databases not MapReduce - and that'd be silly because these guys are column database proponents which is (roughly) the same as HBase/BigTable (they do say it lacks views but the idea used by Google is copying rather mutating). An example of using MapReduce to do indexing, is Nutch. The distributed filesystem is important and it should be clear by now, you'd hope, that storing the data is not the problem, processing it is.

These types of comparisons, "X does something Y doesn't therefore X is better", reminds me of "Who Killed the Electric Car?". One of the reasons the EV1 was said to have failed was because "lacking an engine, it saves the driver the cost of replacement parts, motor oil, filters, and spark plugs" (and more) - all those moving parts, that had to maintained were gone. How can you change gears if you don't have a gearbox?

Does that remind you of a database or other piece of software you use? Isn't the software industry just like the car industry? There appears to be market forces ensuring that things stay complicated and expensive to maintain.

Meanwhile, Google is getting on with it by providing services for terabytes of data (via Wired), "Building on the company's acquisition of the data visualization technology, Trendalyzer, from the oft-lauded, TED presenting Gapminder team, Google will also be offering algorithms for the examination and probing of the information. The new site will have YouTube-style annotating and commenting features."

Update: It's been noted that bet on cheap and rickety if it's 10 times better (also mentions hypertable) and that they're comments are not even wrong.

Update 2: Part of my rant was going to be the whole Apple designs things with the fewest stuff in them - that's what makes them the leader what they leave out. It's the same gist that I read at "Heavier than Air".

Friday, August 03, 2007

Game Changer

Wag the dog
Arguing that EC2 has no intrinsic business value, is like arguing that an electrical grid or a telephone network has no intrinsic business value. Speculation: one reason business systems can't adapt is because the assumptions about what the business used to do, are embedded deep in the code. Very deep, not easy to pull out. And not just in the code but in the physical architecture the system is running on. Business "logic" is like bindweed - by the time you've pulled it out, you've ripped out half the garden as well.


So I missed lunch with a friend earlier this year because he was stuck in an Exchange upgrade. This was at the same time I was looking into Google's architecture and it struck me that there's no real upgrade process or data conversion process that needs hand holding for their architecture. It was at that point that I thought a lot of the jobs system administrators currently do will be greatly simplified or removed with the right software. The ratio of system administrators to servers at Google has to be much smaller (i.e. more servers to people) because they need so many more servers and there just aren't that many system administrators to look after software written in the usual way.

It came up again when I got roped into a meeting with a bunch of guys that administer several clusters. I was quite happy to be quiet but it was brought up that I was looking at cluster techniques. I explained about EC2 and how you can run a server for as little as 10 cents per instance hour and experiments for $2. I think it came as a bit of a shock to them - first the cheapness (obviously) and second the availability (basically its removing the gatekeeping to resources). The second benefit is something that really removes a lot of the politics - something not to be underestimated.

Tuesday, July 03, 2007

Google Culture

So a perspective of Google has been going around for a little while now. The story of when an interview gets widely distributed though is more interesting. It's not like Microsoft has anything to gain from distributing this information.

Via, "RE: Life at Google - The Microsoftie Perspective".

Wednesday, June 27, 2007

Scalability, Scalability, Scalability, Scalability

Dare Obasanjo has four recent postings on the recent Google Scalability Conference.

Google Scalability Conference Trip Report: MapReduce, BigTable, and Other Distributed System Abstractions for Handling Large Datasets:

The talk was about the three pillars of Google's data storage and processing platform; GFS, BigTable and MapReduce.

A developer only has to write their specific map and reduce operations for their data sets which could run as low as 25 - 50 lines of code while the MapReduce infrastructure deals with parallelizing the task and distributing it across different machines, handling machine failures and error conditions in the data, optimizations such as moving computation close to the data to reduce I/O bandwidth consumed, providing system monitoring and making the service scalable across hundreds to thousands of machines.

Currently, almost every major product at Google uses MapReduce in some way. There are 6000 MapReduce applications checked into the Google source tree with the hundreds of new applications that utilize it being written per month. To illustrate its ease of use, a graph of new MapReduce applications checked into the Google source tree over time shows that there is a spike every summer as interns show up and create a flood of new MapReduce applications that are then checked into the Google source tree.


Google Scalability Conference Trip Report: Using MapReduce on Large Geographic Datasets:

A common pattern across a lot of Google services is creating a lot of index files that point and loading them into memory to make lookups fast. This is also done by the Google Maps team which has to handle massive amounts of data (e.g. there are over a hundred million roads in North America).

Q: Where are intermediate results from map operations stored?
A: In BigTable or GFS


Google Scalability Conference Trip Report: Lessons in Building Scalable Systems:

The most important lesson the Google Talk team learned is that you have to measure the right things. Questions like "how many active users do you have" and "how many IM messages does the system carry a day" may be good for evaluating marketshare but are not good questions from an engineering perspective if one is trying to get insight into how the system is performing.

Specifically, the biggest strain on the system actually turns out to be displaying presence information.

Giving developers access to live servers (ideally public beta servers not main production servers) will encourage them to test and try out ideas quickly. It also gives them a sense of empowerement. Developers end up making their systems easier to deploy, configure, monitor, debug and maintain when they have a better idea of the end to end process.


And finally, Google Scalability Conference Trip Report: Scaling Google for Every User, which had some interesting ideas about search engine usage.

Update: links for 2007-07-06 includes links to the Google talk on MapReduce tasks on large datasets and other goodies (scalable b-trees, hashing, etc).

Update 2: Greg Linden has a much better list of videos and commentary.

Wednesday, June 06, 2007

All the Web is a Stage

Google Gears' WorkerPool - Message Passing, Shared Nothing Concurrency

By now, you‘ve probably already seen Google Gears, Google's solution for dragging AJAX applications offline. The one thing that jumped out at me was their WorkerPool component. This is a very nice solution for concurrency in Javascript.
In short: if you have any long running task, you can create a WorkerPool which is basically group of Javascript processes Note: they're not threads! The workers in a WorkerPool share nothing with each other, which makes them like OS processes or Erlang's lightweight processes (actually, they're more like the latter, as they're likely to run in the same address space).

And now guess, how these workers, these processes, communicate? Yep: messages, formatted as strings. Important to remember: if you format objects with as JSON strings, you can even send objects and structures along. The handler that receives messages also gets the ID of the sender, so if the sender implements a handler too, it‘s possible to return results asynchronously.

If you‘re reminded of Erlang or the old Actor concept you‘re right. I wonder what the Google Apps will do with this new concurrency approach (well, new for Javascript… yes I‘m ignoring Javascript Rhino).

I still hope that AJAX will die a quick death, like Java Applets, just for being so damn ugly and horrible to implement. But… things like this tell me that this will probably not happen. Good ideas like Google Gears will help paint over the ugly details of a solution, and it‘s all hip now, so it's easy to ignore many of the problems.


The same idea is also being implemented in Scala with their Actors.

Sunday, April 29, 2007

You Are Here, Now

Saturday, April 28, 2007

Planet RDF Roundup


  • Research Project: Pig Given Yahoo's usage of Hadoop it's good to see them building a query layer on top of it. And it's not SQL - hence the name - it's relationally based (it uses bags and still has DISTINCT) because that scales. Hah! Some of the documentation is gold, "In a conventional database management system, SQL queries are translated into relational algebra expressions, which are in turn translated into physical evaluation plans. Pig Latin queries are already an algebra, so we're bypassing the first layer." It even has nested relations and a flatten operation (which removes the nesting). A good read at "Yahoo Pig and Google Sawzall". Notes that Google's Sawzall looks like Scala.

  • REST Compile/Describe & WADL together with I finally get REST. Wow. all link to WADL. In what seems like an age ago, I remember several enterprise architects asking what the REST equivalent to WSDL was. The later has one of the best one liners to describe REST, "state machine as node graph traversed via URI". An example of how to use is give in "REST Describe first working Beta released"

  • Stavanger, oil, and the Semantic Web Talks about the "Norwegian Semantic Web Days" which covered things such as the OBO foundary.

Tuesday, March 13, 2007

Guice Boost

Guice "Guice injects constructors, fields and methods (any methods with any number of arguments, not just setters). Guice includes advanced features such as custom scopes, circular dependencies, static member injection, Spring integration, and AOP Alliance method interception, most of which you can ignore until you need it."

The moment I saw IoC wiring in code I thought of one man's wish for better IoC wiring in code (we were using Spring at the time).

Behold:

public class MyModule extends AbstractModule {
protected void configure() {i
bind(Service.class)
.to(ServiceImpl.class)
.in(Scopes.SINGLETON);
}
}


And how to inject the Service implementation:

public class Client {

private final Service service;

@Inject
public Client(Service service) {
this.service = service;
}

public void go() {
service.go();
}
}


Now you can test drive IoC wiring without feeling fruity (test driving XML always felt weird).

Which reminded me of a quote by Victoria Livschitz:

We now have a generation of young programmers who think of software in terms of angle brackets. An enormous mess of XML documents that are now being created by enterprises at an alarming rate will be haunting our industry for decades. With all that excitement, no one seems to have the slightest interest in basic computer science.


She was talking about web services but it seems applicable to IoC wiring too.

Via, Guice 1.0 Released.

Update: Similarly, Configuration in Java - It sure beats XML!. Mentions, using Java 6's hotpatch to reload the configuration too.

Thursday, February 01, 2007

Need for Speed

Bosworth On Why AJAX Failed, Then Succeeded links to which isn't the first time he has examined AJAX's success and failures or why other technologies fail such as RDF. Some commentary was interesting in the comments and a few blogs, including this summary:


All in all the speaker (intentionally) kept his talk simple and stupid but here are some points I think are worth noting:
  • Products fail because of [lack of] human psychology not poor engineering
  • Top 3 keypoints of UI:
    1. Speed
    2. Speed
    3. Speed
  • Your app should be a tool, it shouldn't leave the user feeling like one
  • Search isn't perfect but it is far better than any human can do so it is tolerable.
  • Analyze the frequency of use of feature and implement those first
  • If your app takes more than 2 seconds to load - forget about it
  • The Motorola RAZR was cool enough so the poor UI (narrow keypad) didnt matter

Wednesday, January 31, 2007

Semantic Spreadsheet

What happens when you combine a Google spreadsheet of Semantic Web tools and Simile Exhibit you get Sweet Tools (Sem Web).

The only problem seems to be that it only shows the first 10 or all results - it'd be good to have a previous/next button.

Sunday, December 03, 2006

Good Models - My Super-Turtle is Better than Yours

  • Standards and Pseudo Standards from Celebrating OWL interoperability and spec quality. Can a standard be based on a pseudo standard? Another posting also points to "An Investigation into the Feasibility of the Semantic Web" about the feasibility of integration using ontolgies.

  • Beyond Belief 2006 Session 5 includes Paul Davies (asks why the universe should even be understandable to human beings, why he's no longer a Platonist and levitating super-turtles) and Session 9 on why religion may have a place.

  • Agile Atheism "I am not agile because I don't believe in the agile religion and I don't accept its dogma. I like the engineering and planning practices that agile teams use - in the same way that I like people who do nice things (even when they do it because of fear of divine retribution). The difference is I don't want to be constrained by dogma into only doing those sensible things which are prescribed by agile. In the same way I don't like being prevented from doing sensible social things because of religious beliefs."

  • Jane's Rule for Loading Dishwashers "Compare this to test-driven development. There may be a little more effort when writing code, because you are writing programmer-tests to drive writing that code. It's an "unnatural" process, like sorting silverware into sub-bins when loading the dishwasher. But the true benefit comes later in the project (possibly just minutes or hours later), when you can rely on those programmer-tests to make refactoring safer. And since you fixed bugs during TDD, you have much less work fixing bugs later when it comes time to ship your product."

  • How can I get financial market information updated automatically to my spreadsheets?

  • 10 most intelligent / least intelligent dogs My dog is apparently the stupidest breed with regards to obedience. I'm not sure I'd call obedience a sign of intelligence (the complete opposite really).

Friday, November 10, 2006

Semantic Web 2.0

Some recent articles about people discussing the same ideas as the Semantic Web. The Great Database in the Sky "He went on to describe his vision of a skype for database access, combining my data, your data and public data into the next generation OLAP, running a trillion transactions per day. An example could be weather data and he asked what if you could run a SQL statement across all the data sources in the world"

"This is where it became evident that there is a deep disconnect between the traditional database community and the semantic web community. Mårten’s response was rather vague, that this wasn’t as broad as the semantic web and that the semweb includes unstructured data so wasn’t appropriate."

CEO of MySQL "Invents" the Semantic Web! "I have to say, his talk was both a validation of what we have all been working towards, and as Ian Davis explains, it is also a clear sign that the W3C and the Semantic Web community have not found a way to get the message accross."

And moving data around, owning your data seems to be another aspect of the Semantic Web overlooked.

WEB 2.0: Google CEO: Take your data and run "The more we can, for example, let users move their data around, never trap the data of an end user, let them move it if they don't like us, the better."

And my mind boggles at the idea of taking the proposed Australian Access card and their integration problems and fusing it with RDF. Some interesting points: "...the Access Card will be owned by the cardholder and not by the issuer...The effect of the issuer retaining ownership is that they control the card and the purpose for which it is used."

"In Centrelink alone we have a massive 275 kilometres of files...Medicare has to measure its records in a similar way. They have more than 3 square kilometres of storage space for forms with signatures."

"We collect, and almost never reuse, this information."

"The new card will finally put an end to this waste of time. We will be able to reuse the information that you have given us before, but only for the purposes for which you gave it to us. We can then pre-populate forms and take a lot of the pain out of the claim process."

Wednesday, November 08, 2006

Languages - Open vs Closed World

What did Javascript have that Java did not? "Languages based on static inheritance and typing are good for building complex silo (i.e. closed world) based applications. However, a global scale Architecture of Participation requires more dynamic structures like that found in prototype based inheritance and dynamic typing. In such a massively open world, the distinction between metadata, configuration and instances is simply impossible to pin down into well defined classes and configuration files."

"Javascript by its design is fundamentally messy, however that is its advantage over Java. The path to any sanity may just be what Google has shown, when building silo apps, hide the messy details under a clean well defined Java facade. Never forget though, that these facades are abstractions that leak. Always afford your applications the ability to escape into the Web and Javascript when necessary."

Bruce Tate has recently had a new article published, "Crossing borders: Delayed binding": "The more you dig into type and binding strategies, the more you find that waiting until run time to bind to an invocation or type fundamentally changes the programming process, opening a whole new world of possibilities. True, you find less safety. But you also find less repetition, more power, and more flexibility with fewer lines of code."

Wednesday, October 11, 2006

Agile Bigger than Jesus

Software Development: It's a Religion "But software development is, and has always been, a religion. We band together into groups of people who believe the same things, with very little basis for proving any of those beliefs. Java versus .NET. Microsoft versus Google. Static languages versus Dynamic languages. We may kid ourselves into believing we're "computer scientists", but when was the last time you used a hypothesis and a control to prove anything? We're too busy solving customer problems in the chosen tool, unbeliever!"