What’s the difference between user generated content and user generated rubbish? Comments please…

Some user generated content (UGC) is genuine, honest, credible, reputable, trustworthy, valuable, quality information. But some is rubbish (let’s call that UGR), including deliberately misleading propaganda, biased blog comments, bogus product reviews, spam, veiled advertising, and bad poetry (or is it just my blog that attracts poetry bots?)

Google’s PageRank algorithm does a good job of measuring the quality of a simple web page, based on the number of incoming links to that page, and recursively weighted on the quality of those linking pages. However, web2.0 has given us blogs, wikis, forums, media sharing, customer product reviews and ratings, social bookmarking, and more recently aggregation of all of the above; resulting in web pages that contain an increasingly complex array of UGC and UGR, making it increasingly difficult for algorithms, and site visitors and site owners to filter the signal from the noise, the UGC from the UGR.

So I wanted to write a post about some of the emerging technology innovations attempting to solve this problem. Readers are kindly asked to add a comment at the bottom of the post. All comments will be shown, even bad poetry, for purposes of research and experimentation.

Measuring quality is relatively easy for eBay. Its Feedback Ratings provide an excellent indicator of trustworthiness, because online auctions involve measurable user actions such as ‘Was the product description accurate?’ and ‘Did the buyer pay up?’ Such actions speak louder than the mere words of a blog comment or product review.

Amazon now owns a valuable database of customer product reviews to help people through their purchasing decisions. Innovation by Amazon in this area has included the ability to provide feedback on the usefulness of other users’ comments, and a Reviewer Rank algorithm which provides a measure of reviewer quality (interestingly, this algorithm was recently improved to include some PageRank-like recursiveness).

In a past life I had the pleasure of working for Lonely Planet, a travel publisher whose credibility and quality has been built upon the independence of its authors and their unbiased travel reviews. Lonely Planet and its peers have long struggled with the opportunity to harvest UGC from loyal and passionate travelers, because it is just so difficult to measure the independence and quality of contributing users.

TripAdvisor was allowed to emerge as a disruptive force in the market for travel advice, allowing anybody to review any hotel or restaurant. That created a lot of quality content for a while, but ever since hotel owners found out about TripAdvisor and began to review their own hotels, it’s been difficult to tell the UGC and UGR apart. TripAdvisor still desperately needs a reliable measure of user generated quality to restore its credibility.

Perhaps social networking can help TripAdvisor; being able to filter your travel advice to that written only by your friends would eliminate biased reviews (unless you are friends with a bunch of hotel owners, in which case you’re probably going to stay in their hotel anyway). But until the internet settles on a standard for social data portability, not many of us will have enough online friends who have traveled enough and generated enough online travel content for such a social filter to work reliably, even allowing for recursive algorithms.

If it’s just travel advice and inspiration you’re looking for, you could wait for Lonely Planet’s upcoming blog syndication feature, which promises a novel solution to the problem.

But more generally, I think we all need a universal reputation system, one which aggregates lots of measures of quality from lots of different sites. Imagine if you could easily see a summary of my quality metrics from eBay and Amazon and Yahoo Answers and LinkedIn Answers and GetSatisfaction, perhaps even my Bugzilla and Basecamp metrics too; would that be enough for you to trust my travel advice and any other content that I generate?

Site visitors would benefit from increased visibility of users who generate content. Genuine contributors would be encouraged by being able to build a universal reputation for quality UGC, and discouraged from the risk of creating UGR. And site owners would benefit from data to filter out the UGC from the UGR.

A universal reputation system could also help to eliminate online vote rigging, astro-turfing (all those reviews of iPhone apps posted by the developers themselves), and space-faking (setting up false identities on social networking sites).

Who are the players?

SezWho SezWho provides a plugin for blog commentary which presents a useful summary of UGC history for each contributor, and allows customizable 5-point rating scales for site owners.
Intense Debate Intense Debate has a great interface design. It’s recently been acquired by Automattic, the owners of the WordPress blogging platform, which will provide some valuable distribution, perhaps critical mass. But will the other blogging platforms want to adopt or integrate with a standard controlled by a competitor?
Google Friend Connect Google Friend Connect allows any site to embed a comments or ratings gadget onto any page. The universal view of previous UGC is not there yet, however this will become powerful when integrated fully with Google’s other stuff; Blogger and SearchWiki and the Social Graph API and YouTube (arguably the site most in need of a UGR filter!)
Disqus Disqus is getting lots of press for its prompt Facebook Connect integration which takes the hassle out of commenting. Video comments can by posted, powered by Seesmic. Readers can nudge comments up and down the list by voting on them. Try it out below.

If you have a view on who will win the race to become the universal reputation system, please comment below. Are there any other players that I have missed out? (Yes I know that is exposing me to some comments on the quality of this post!)

Also here’s some further questions to inspire some commentary:

  • Should we settle on a word for what is being measured here? Quality, importance, value, trust, reputation, credibility, honesty, transparency? Or will the winner of the race provide a web2.0 brand name to describe this concept of a universal measure of user generated content?
  • Is it even possible to determine an objective universal score? The success of PageRank would suggest yes. Or is quality in the eye of the beholder? Is one person’s signal another person’s noise?
  • Would a universal metric destroy the democratic level playing field that is UGC / UGR?
  • What are the consequences of such a universal reputation system being gamed?
  • How likely are eBay and Amazon to open up their reputation data? What are the privacy implications?

Thoughts please. Don’t be shy!

This entry was posted in mobile geo social and tagged , , , , , , , , , , , , , , . Bookmark the permalink.
  • http://www.profero.com.au Mike

    This ties in to the Web3.0 debate, or Contextual Web – value of content is determined by context, in most cases, the context being the credentials and background of the contributor. Depending on what i'm looking for, that context changes – i may be quite happy to believe the review of a stranger, provided i know they have a journalistic background and have no obvious vested interest, or it may be only friends who's opinion's i rate that i'd want. As in the real world, i'd value my geek mate's review of a new laptop, but not my dad's. So i think there will need to be an overlay of “expertise in field” or fit-for-purpose across this – i could give a plausible review of a digital marketing campaign, but no-one would want to know my thoughts as an art critic, for example. But i agree with you, something has to emerge – the current format is becoming too open to gaming by dodgy marketing agencies and people with vested interests, and all that will do is ultimately destroy the value of all UGC in the eyes of the consumer as everything gets tarred with skepticism.

  • http://obscurelyfamous.com Daniel Ha

    When we first started Disqus, we decided to place a heavily emphasis on user reputation and quality filtering/tracking across websites.

    This was a good idea then and it's still a good idea now. But we realized we weren't doing a very good job, and we decided to focus on what we were nailing – pure social commenting functionality that people enjoyed using. The reputation points aspect is still a part of Disqus but we acknowledge that it still needs a bit of work for it work as nicely as we had hoped earlier on.

    As recently as yesterday, we discussed amongst ourselves some ideas that we're pretty excited about. I don't think it'll be a solution to the reputation problem, but it'll help surface some quality contributions on sites.

    Thanks for the mention.

    Daniel @ Disqus

  • http://intensedebate.com Michael Koenig

    Hi Bob. Thanks for dropping me a line about your post and for including IntenseDebate in your assessment.

    IntenseDebate is all about making comments useful and bringing valuable comments into the forefront. Our comment voting and user reputations are one of the ways we do this – it's one of our solutions to help you cut through the comment noise and bad poetry and help you easily find quality comments made by credible commenters.

    IntenseDebate commenter reputations are based on the number of comments made, but more importantly the number of times their comments are voted as being worthwhile or not. Each commenter's reputation is immediately visible and is located right next to their avatar so you don't have to go digging around to find it. The comment rank is just to the right of the reputation. Add in our filters to sort the highest rated comments to the top and IntenseDebate makes it a piece of cake to find the good stuff.

    Our reputation system will be getting a major overhaul based on our community's feedback sent to us directly and voiced in our Get Satisfaction forum. We've always got our ears open.

    In regards to your concern about IntenseDebate being an Automattic Joint now, rest assured IntenseDebate will always be platform agnostic. The only things that have changed (among others) are that our system is more secure and scales better than ever, we can guarantee that we're not going anywhere, and we've got some really innovative enhancements coming down the line with other Automattic products! Add access to a massive userbase and it's a great time to give IntenseDebate a try.

    Thanks again!

  • http://www.avanta.co.uk/UK/meeting-rooms/offices/Heathrow_MTR meeting rooms heathrow

    Great post, just posted it up in twitter. Hope you don't mind

  • Wayne

    And why exactly should I listen to you? Eh? Who set you up as an 'expert' on UGR? Why i might as well ask my cat what she thinks…. hold on…. hmmmm…. she seems to think her bottom is an ice cream…. Now that's what I call thinking outside the box.

  • http://www.facebook.com/profile.php?id=527809107 Bob Hitching

    Thanks Wayne, great example!

  • http://www.hotelbackpacker.com Backpackerresources

    Building quality and good reputation takes a lot of courage and sweat. Unquestioned credibility matters most and as what you wrote, truth in all aspects especially in dealing with hotel accommodation during travel is important. But not all “checklist” are credible. How to spot the Not? Just follow your instinct.

  • golfman_story

    This is quite impressive, I am pleased to read this post, keep posts like this coming, you totally rock!

    http://modestadventurer.com

  • http://www.elite-proxy.net Craigslist Proxy

    Was a good read. I'll be back for more.

  • http://www.diaspora.ie/ Mick

    Hi weblov,

    Hi Bob,

    just a short one to let you know I have Facebook Connect running in Intensedebate here: http://diaspora.ie/starship/2009/06/intensedeba

    It's Intensedebates limited roll-out trial so feel free to try it out / leave a message.

    Slainte!

    Mick

  • http://modestadventurer.com Traveller_Adventure

    Great post, really help me alot. Thanks.
    sain-web.com

  • http://www.club-penguin.org/ clubpenguincheats

    I think there will need to be an overlay of “expertise in field” or fit-for-purpose across this – i could give a plausible review of a digital marketing campaign, but no-one would want to know my thoughts as an art critic, for example.

  • max191

    I am about to start a blog and your blog gave me much hint how to do it. I really loved to visit your blog. Hope to see more inputs from you in your blog.
    regards
    charcoal grill

  • http://www.hostelio.com Hostelio

    I started using Google Friend Connect a while ago, however i cant say i am fully satisfied. Facebook integration with API looks like a much better solution.

  • http://www.besttabletlaptops.info/ tablet laptops

    Wow! this is a great description helps people to know about the good comments and rubbish comments. Also helps blog owners to use the latest software to eliminate the rubbish comments.

  • Ekaraj

    Woo this is a nice blog, i would love to read more.

    regards
    smith
    ______________________________________________
    Cheap Aion kinah | aion kinha

  • Lisa909

    i added this to my article site.

    Thanks
    paul
    ______________________________________________
    aion gold | Cheap Aion kinah

  • http://twitter.com/hitching bob hitching

    Google's new Social Search Experiment http://bit.ly/4DgQX8 shows that Google is working hard on this problem.

    According to Eric Schmidt, figuring out how to rank real-time social content is “the great challenge of the age” – see http://www.readwriteweb.com/archives/google_web

    Perhaps PeopleRank will emerge as the successor to PageRank, and a solution to ranking real-time social content / filtering out the user generated rubbish.

  • Anonymous

    Your hotel travels post is very nice I like your post. You can also get good information and best facilities of Cheap Hotels Deals for travel and you will get all the package of traveling.

  • http://www.artndesigns.org/2009/06/latest-arts-designs-in-wedding-dresses-ultimate-collection-of-stylish-wedding-dresses/ Fashion Trends

    Hey friend finding continues good news of latest art please give me some ideas to find updated news. I also want to promote my site by updated news of latest arts and designs.

  • http://theoutsourcebot.com seo company

    I subscribed to your blog when is the next post

    Have a nice day
    mike jumper
    ______________________________________________

  • http://www.iintense.com backlinks

    nice post, thanks for the share

  • http://www.jrhandbags.com/ gucci bags

    “Good article, I like it, thanks for share!
    http://www.galaxyhandbags.com

  • http://www.jrhandbags.com/ gucci bags

    “Good article, I like it, thanks for share!
    http://www.galaxyhandbags.com

  • http://www.sinohotel.com/english/hotels/beijing/index.html Beijing Hotels Reservation

    That Sounds interesting, I agree with you.Please keep at your good work, I would come back often.

  • teenxxx1
  • http://www.sinohotel.com/english/hotels/beijing/index.html Beijing Hotels

    Interesting post. I have made a twitter post about this. Others no doubt will like it like I did.

  • http://www.sinohotel.com/english/hotels/beijing/index.html Beijing Hotels

    Interesting post. I have made a twitter post about this. Others no doubt will like it like I did.

  • http://gisnap.com gisnap

    Its really cool, I came to know this really worth visiting, just bookmarked your site.

    http://gisnap.com/
    The place where fun never ends

  • http://gisnap.com gisnap

    Its really cool, I came to know this really worth visiting, just bookmarked your site.

    http://gisnap.com/
    The place where fun never ends

  • rockymeet

    This is fantastic news! Sharon has tremdous talent – and I am so glad she will share that with us!
    anti keylogger

  • http://www.lv-cheap.com/ louisvuittonoutlet

    The most important thing http://www.lv-cheap.com to consider is your professional life when it comes to choosing a gucci handbags. http://www.gucci2sale.com

  • https://www.jobsindubai.com/job_list.asp?lstIndustryID=15 Dubai Hotel Jobs

    Trip advisor is one of my favorite hotel and restaurant reviewer. There's a lot of information there regarding cheap hotel accommodations, features and others. But when it comes to Hotel Jobs, here's a wonderful resource that will surely be liked by Hotel jobseekers: https://www.jobsindubai.com/job_list.asp?lstInd

  • http://www.lv-cheap.com/ louisvuittonoutlet

    nice disqus.

  • http://www.tuktukholiday.com/ Hirochisung

    Good job.Good post ,Good comments.

  • http://www.tuktukholiday.com/ Hirochisung

    good idea, and very interesting.

  • http://www.gucci2sale.com/ gucci2sale

    Thanks for posting this. Very nice recap of some of the key points in my talk. I hope you and your readers find it useful! Thanks again. gucci2sale.com

  • http://www.jrconsumer.com/ RV Ratings

    anything should really must be useful to us…

  • http://www.longnightgown.net/ long nightgown

    I liked your article very useful to me. I will come back and read the article again.
    More info on the web site long nightgown

  • http://www.webaddlink.com/ addlink addurl

    Your website is beautiful and useful. How do I access your site often. Come visit my website at addlink addurl and Linking to websites

  • http://peninggi-badan.com Bill Andreas

    Site visitors would benefit from increased visibility of users who generate content. Genuine contributors would be encouraged by being able to build a universal reputation for quality UGC, and discouraged from the risk of creating UGR. And site owners would benefit from data to filter out the UGC from the UGR.cara menambah tinggi badan

  • Pingback: Social Recommendations For Every Site On The Web | mobile geo social

  • http://peninggi-badan.com Bill Andreas

    Disqus is getting lots of press for its prompt Facebook Connect integration which takes the hassle out of commenting. cara meninggikan badanVideo comments can by posted, powered by Seesmic. Readers can nudge comments up and down the list by voting on them. Try it out below.

  • http://peninggi-badan.com Bill Andreas

    If you have a view on who will win the race to become the universal reputation system, please comment below. Are there any other players that I have missed out? (Yes I know that is exposing me to some comments on the quality of this post!)
    tinggi badan

  • http://www.voxcall.com/ international call

    All I can say is, disqus is great. I've been using it for a year and it helps me to an easy commenting process, no need to make a lot of accounts.

  • http://www.goahats.com newerahats

    Do you want to be more fashion, more charming,just do a little thing. A exciting purchasing is ready to go! Just pay attention to. We are specializing in providing new era hats ,wholesale new era hats,new era caps ,one industries hats,rockstar energy hats,Monster Energy Hats which would be your final choice. Just do what you want alonging with your active heart. Our website is persisting on best customer service. Please come and enjoy it!

  • http://tuktukholiday.com cheap hotels

    Very nice recap of some of the key points in my talk.

  • http://marriedwomenlookingforaffairs.org Married And Cheating

    Yep its disqus all the way for me, i find it really helpful and easy to use.

  • http://sarakeane.com Married Women Seeking Married

    I gotta agree with the majority here…I use disqus all the time and I love it :-)

  • http://www.stajump.com NFl Jerseys

    I very much enjoyed your website. Excellent content.Your favorite team's Cheap jerseys delivered right to your door.
    Cheap Nfl Jerseys
    Cheap Jerseys
    All at unbeatable prices!!