mobile geo social

I remember in January 2000, old media mogul Rupert Murdoch said he was not going to waste his money buying any ‘dotcom’ upstarts. The very next day, AOL bought Time Warner. Not the other way around!

Murdoch had apparently failed to grasp the significance of the interwebs.

However, ten years later Time Warner has regained its mojo and is now trying to offload a spent and jaded AOL. Did Murdoch get it wrong ten years ago, or did it simply take a whole decade for him to be proven right?

In 2009, the mob is rushing once again to the conclusion that Murdoch is losing his marbles, planning to charge for his online content and blocking the Googlebot from stealing it.

Personally I believe that Murdoch should worry less about the Googlebot, and more about how social media is turning his industry on its head.

The problem is that all of those dotcom upstarts have brought us information overload. There has been an exponential increase in the amount of information and content available to us, way beyond the capacity of the human brain to process.

The solution is social media, which empowers us to easily share the content we care about with our friends and contacts, and adds valuable metadata to that shared content, such as Likes or Retweet counts. This metadata helps us filter the signal from the noise, so that we can focus on just the best quality content from our trusted circle of friends.

This works great for movie reviews. People have always listened to the advice of friends when it comes to choosing what movie to watch. Social media simply provides an efficient and scalable way to do this.

The best example of this social filter is currently FriendFeed, although we can expect to soon see something equally impressive on Facebook. Twitter Search could do this even better if only it were possible to search the entire tweet history of just your friends, or a chosen social distance into your social graph, rather then merely search 7 days of the public timeline. I am hoping that the Google Social Search Experiment will enable this sort of social filter when Google completes its Twitter integration.

Back to Mr. Murdoch… Social media also works for the filtering of news content, however it’s more tricky than movie reviews because there is a need for trustworthy fact rather than mere opinion. This is why Eric Schmidt believes that figuring out how to rank real-time social content, perhaps based on a reliable measure of reputation and authority, is “the great challenge of the age“. It also explains why Twitter’s Retweet feature does not allow the original tweet to be modified, because this makes the Retweet count a more reliable indicator of authority.

So my advice to Rupert Murdoch and other media companies struggling with this; worry less about the Googlebot and more about social media. Focus on improving the quality of your content, so that people share it with their friends.

And if your own social media strategy is not delivering any tangible benefits, try moving it from your Marketing department to your Customer Service department. Use social media to listen more carefully to the needs of your customers, so you can improve the quality of your content to the point where a paid online content model becomes viable.

If Marketing and Customer Service argue about who owns the customer relationship, remind them both that thanks to social media it’s actually the customer who owns and controls the relationship with your business. Not the other way around!

I am pleased to announce the launch of GeoMeme, the fun way to measure and share real-time local twitter trends.

I got thinking about this when a recent Los Angeles earthquake was being measured in tweets per second rather than using the Richter Scale.

Then came the Magnitwude Calculator as a standard way to measure the magnitude of Twitter trends.

[Then came twotspot.com but that domain name was just too damn rude, so it was quickly renamed to GeoMeme.]

What does GeoMeme do?

GeoMeme measures real-time local twitter trends.

Tweeps are located on the map using public data from a number of iPhone twitter apps. When twitter launches its geolocation API, that will be used to locate even more people on the map.

GeoMeme measures and compares how many people on the map are tweeting about each of your two search terms:

The ‘magnitude’ of each search term is equal to the number of unique people tweeting per hour per square kilometer, so it increases when more people are tweeting in a smaller area.

Example: if 100 different people in an area of 10km² have tweeted about ‘love’ in the last 2 hours, the magnitude is 5.0 (100 divided by 10 divided by 2).

So you can search for ‘love’ and ‘hate’ and GeoMeme works out which one “beats” the other with the higher magnitude.

The default search terms are and smiley faces which provides a good measure of local happiness, as an example.

Can I use my iPhone?

Sure, or your iPod Touch. Here’s the screenshot:

Give me an example!

Thanks to some early coverage on The Register, Mashable, and Google Maps Mania, and winning Mashup of the Day on ProgrammableWeb, we’re off to a flying start. I’m glad GeoMeme is hosted on Google App Engine for scalability.

Here’s a selection of the most popular GeoMemes so far:

• beats in New York City
• Mega Shark beats Giant Octopus in LA
• Snow Leopard beats Windows 7 in Cupertino
• bridge beats swimming in San Francisco Bay
• wtf beats ftw in Washington

How does it all work?

I will leave the details of how it all works to another post, stay tuned for that.

Cloud computing is being sold as a hosting architecture to provide instantly scalable on-demand computing power, storage and bandwidth.

“The cloud’s resources scale with user demands. Pay only for what you use” says RackSpace, the latest to join the cloud gang.

One problem for the cloud gang, however, is that hosting has always struggled as a low margin commodity business.

Rackspace has just hired Robert Scoble to help spread the message, so we should expect this space to soon get hotter than an Sun SPARC with a loose heatsink.

But where exactly can some value be added in cloud computing, to increase the margins and keep Scoble funded so he can continue to filter the signal from the noise on FriendFeed? Okay, that’s slightly selfish but it’s an interesting question.

The interesting answer IMHO is cloud datasets.

Having useful datasets available in the cloud will unlock value from the data by allowing a new generation of mashup. These aren’t mashups that simply use data from remote web services, like plotting Craigslist ads onto a Google Map. This involves the mashup (joining) of datasets in the cloud using the power and speed of a relational database.

This cloud database approach might also provide Twitter and other owners of valuable data with a revenue model that doesn’t depend on advertising.

Here’s 10 cloud datasets that I’d personally like to mashup, to help explain:

1. Wikipedia. Funnily enough Amazon Web Services has just announced that it now offers a 66Gb dataset of Wikipedia. “The wiki markup for each article is transformed into machine-readable XML, and common relational features such as templates, infoboxes, categories, article sections, and redirects are extracted in tabular form.” One example: imagine the opportunities for a start-up social travel site to mashup its content with the wealth of travel information now available on Wikipedia. Massive.

2. Geonames. It bugs me that everyone who wants to use the geonames database needs to duplicate 800Mb of data. Move it into the cloud! Example: the travel site can now analyse reams of user-generated content (or Wikipedia content) for up-to-date categorization and geo-coding onto a map. Another example: most websites need a simple (but updated-more-often-than-you-would-think) list of countries on the rego form. Wouldn’t it be good if everyone used the same (geonames) list?

3. MaxMind IP address lookup. Turn an IP address into an always accurate city location. Example: targeted ad serving and traffic analysis.

4. Google PageRank. For any URL, what’s the PageRank measure of quality? If this is relational data (rather than from a remote web service), it can be combined with other measures of quality at database speeds.

5. Real-time stock market data.

6. Real-time sports data.

7. Dodgy credit card numbers.

8. Dodgy email addresses.

9. Twitter. Some of the above might be considered proprietary rather than public data, which brings me to Twitter and a potential revenue model for them and the cloud gang. If you’ve got valuable proprietary data like Twitter has got (some would say that’s all they’ve got), then replicating it into a relational cloud database will unlock more value than could ever be extracted (or sold) via a remote web API.

Example: when visiting an e-commerce site, it would be nice to see only the product reviews submitted by people I am following on Twitter, sequenced by a measure of quality based on how often those people have been retweeted. Of course, the cloud gang already have the billing infrastructure and monitoring in place to work out exactly how much proprietary data you have used, and what to charge you for it. Did I mention yet that Jeff Bezos is an investor in Twitter?

The advertising pie is not big enough to fund the whole of the interweb, so perhaps paid data consumption is the revenue model for Twitter and others. Businesses are happy to pay hosting providers for commodity services like CPU cycles and disk space, so why not pay Twitter (via a hosting provider) for valuable information? Did I mention yet that Jeff Bezos is an investor in Twitter?

10. This one is further out there; private foreign keys. Imagine the Twitter dataset including the email address of users, joined using that email address to a Facebook or Digg dataset, but not revealing that email address in the result set. That’s number 10 on my list. It would need to work in a similar way to Facebook’s FQL or Yahoo’s YQL or Google’s GQL, to expose enough information to be useful but to not expose anything that would violate privacy concerns. I hope to write some more about this and the privacy implications in another post.

So, who’s in the cloud gang? Google is well placed with AppEngine and plenty of valuable datasets to get started with. Amazon has all the billing machinery in place to sell proprietary data from Twitter and others. Sun now has MySQL which already supports remote replication and column-level permissions to enforce private foreign keys. And now RackSpace has Robert Scoble. This will be an interesting one.

Google today launched a rather massive change to its core search product.

SearchWiki adds some innocuous buttons to your search results page, enabling Digg -style voting and Friendfeed -style commenting on each result.

I think this feature might prove valuable for some users, at least the bad spellers among us and those who prefer to repeatedly type the same search term into Google rather than use bookmarks or their memory.

However this feature becomes massively valuable for Google if enough people bother to vote for their favourite sites and add comments. Harnessing the collective wisdom of all those users is a great way for Google to improve upon its not-so-secret-anymore search algorithm.

Currently your own SearchWiki wisdom impacts only your own search results, nobody else’s. But the words chosen to explain SearchWiki do leave the door open for Google to evolve into a social search engine; “Customize your search results with your rankings, deletions, and notes — plus, see how other people using Google have tailored their searches.”

Personally, I’m not sure how much I want strangers (or bots) to influence (or game) my search results.

But I might want my friends and social networks to influence some of my search results.

If only Google could somehow identify all my friends in all my social networks, and keep track of their searching activity. Wait a minute…

SearchWiki + OpenSocial = mainstream social search.

The web is the their platform.

For a while I’ve had a niggling problem with social networking sites.

I’ve already set up my LinkedIn network and my Facebook friends, so why should I have to do it all again on every other site that has decided to go social on me?

When I heard the OpenSocial announcement last year, while I was F5ing the API URL, waiting to see the campfire video, I was imagining that the problem had been solved, by allowing any social networking site to share its social data with any other.

But the first incarnation of OpenSocial, actually the 0.7th as I write this, is more aimed at developers re-using code to make applications more portable, rather than data portability.

Then Plaxo released a LinkedIn sync feature which looked promising, but that was just two social sites, what about all the others?

Now it looks like Google has provided the solution, not as part of OpenSocial but with its new Social Graph API. Social data becomes portable simply by adding some XFN tags to the hyperlinks between your pages and your friends’ pages and your other pages (view the source of this page and search for rel=”me” to get the idea), then letting the Googlebot spider those links to work out the connections. Very simple and powerful. The internet is the platform.

This must upset Facebook, because the social data representing all those friend connections is a big part of their crown jewels. If Facebook changes profile pages to become publically available (or less revealing profile summaries, as LinkedIn has done), and adds some XFN tags, then that social data and the ad revenue extracted from it will start to trickle out onto the wider web. How long can Facebook resist?

Anyone could start to work out who knows who by using the Social Graph API. Reputable sites will put the decision of how to use that social data in the hands of the user. But there’s also a privacy risk here. Perhaps the answer to that is something along the lines of OpenId which puts the user firmly in control of how portable their identity data is.

My mind spins with the opportunities and challenges created by this great innovation. My favourite: combine social graph data portability with always-connected location-aware mobile devices (“phone”), and you can mashup the social landscape with the physical landscape you’re walking through.

Maybe that’s what Judge Dread’s helmet did.

Here’s another blast from the past. In the early 1980s, my dad bought me a Sinclair ZX Spectrum to celebrate my passage into the teenage years.

When I got fed up with the games that you could play by copying Spectrum Basic code from the pages of a magazine (and fed up with the seek / transfer times involved), I decided it was time to learn programming. When I got fed up printing rude words for my mates using Spectrum Basic, it was time to get serious and learn Z80 assembly programming.

The outcome was a fruit machine simulator called ‘MegaFruit’ which had some revolutionary graphics and sound for its day, and also speech synthesis! MegaFruit took 7 squeeking minutes to load its 16,384 bytes of Z80 code from an audio cassette tape.

To my delight, I was able to strike a distribution deal with Thor Computer Games, who marketed and sold the game, and paid me money! Here’s the cassette sleeve artwork:

Sometime in the 1990s, my last remaining copy of the MegaFruit cassette tape was lost, I suspect as I moved my possessions around England during Uni days. I was devastated that I would never again see my creation working.

Then came Google to the rescue. I’ve been a fan of Google since their early days, and they did good. My Google Moment came in 2005, when I searched and found the website of a Spectrum addict in Russia who had been creating ROM images of games that could be interpreted by a Spectrum emulator written in Java. Before long I was playing MegaFruit again with a massive smile on my face. I even found a port of the emulator that let me play on my Smartphone.

And here it is using the excellent QAOP emulator (so named after the forward / back / left / right keys of choice back then):

Keys:

F11 = mute
PgUp/PgDn = sound volume