Beta Labs - Yellowpages.com

Ratings and Reviews on YELLOWPAGES.COM

Last week we launched the Review2Win Sweepstakes on yp.com and YELLOWPAGES.COM. The sweepstakes should generate increased awareness of our ratings and review feature (games, prizes and trips are gauranteed to get our attention!) so we decided that this was a great opportunity to introduce Nicholas Perry – the ratings and reviews feature manager. We asked him to give us an overview of the feature and highlight why ratings and reviews matter in local search.

“People generally trust online reviews more than critic reviews and editorials. They want to read what other people thought about a business, learn why others rated a business highly or poorly, and get a better idea of whether they want to use that particular business or not. Ratings and reviews help a user differentiate between businesses that otherwise seem very similar.

On YELLOWPAGES.COM and yp.com we allow users to rate businesses on a scale of 1-5 stars and write a review of their experience with a business. We also aggregate reviews from other sites to increase the number of ratings and reviews that we provide to our users.

On the yp.com site, we use the reviews to help provide relevant results. If many users are praising a restaurant’s lasagna in their reviews, then that business may be a better match for a user searching for “lasagna” than other Italian restaurants

We filter reviews* for violations of our specific guidelines, such as profanity, slander, or spam. If a user writes an honest, factual review of a bad experience they had with a business, then that review will remain on our sites. We can’t control everything, but the more real users we have reviewing each business, the less impact fake reviews will have.

We present the reviews in order of review date. Recent experience with a business is most relevant to how the business is performing now, helping a user make a decision on which business to use.

We recognize the importance of including businesses in the ratings and reviews process. We allow businesses to reply directly on the site to each review so business owners can have an open dialogue with their customers for positive or negative reviews. This dialogue between consumers and the business can be very valuable to everyone. And, since we recognize that some businesses are also concerned about the quality of reviews we do allow businesses to opt out of reviews altogether. However, as online reviews are extremely valuable and important to consumers, the vast majority of businesses do allow users to review their business.

There are many prizes available during our Review2Win Sweepstakes, so take this opportunity to provide very valuable insight into local businesses that you love or hate. Your voice will be heard by our large user base and may help your neighbors make good decisions on which local businesses to use.

Watch for new innovations with our Ratings and Reviews feature in the future. We are currently in the process of developing new functionality which we plan to introduce in 2010. Of course, if you have any ideas that you’d like to see on the site, just drop us a note. All ideas are welcome!

Nicholas Perry
Sr Product Manager

* “We are under no obligation to monitor any reviews; however, we reserve the right, in our sole discretion, to monitor, edit and/or remove any reviews for any reason, investigate possible violations of our User Review guidelines, and enforce any rights available to us as a result of any such violation.”

User Feedback on Search Relevance

When writing the retrieval & ranking code that’s at the heart of any search engine — code that answers the question what results should be returned for this search, and what order should they be in? — engineers have a plethora of algorithms to choose from. And given the critical importance of getting relevance right, one might reasonably wonder whether, by now, the search industry has settled on a standard automated measurement of relevancy. (That would certainly help us choose the right search algorithms!)  This is not the case.  While some work has been done to analyze server logs to report proxy measures such as click-through rates and page dwell time, automated tools have yet to replace human raters. This is largely because relevancy is highly subjective.

Many local search engine firms have specialized in-house teams whose job it is to judge the relevance of the results produced by their engine in response to real-world queries from users. But this raises some questions: Are the scores from a group of ‘professional judges’ applicable to the general population? Without local knowledge, can the testers determine whether an ambiguously named business is properly categorized or whether the most popular restaurant in a neighborhood is ranking appropriately?  Are the testers aware of colloquial differences in names for businesses or products?  Do the testers understand local preferences around how far people are willing to travel for a particular type of business?

One way to address such concerns is simply by asking for direct feedback from real users. So today, yp.com is launching a tool to gather relevancy feedback on individual search results. Next to any business listing show on a search results page on yp.com, you’ll notice a pair of Yes/No buttons under the words “Listing relevant?” We hope you’ll take the time to click a few of them — especially when we return the wrong stuff, but also on those occasions when we manage to get it right, too!