The topic of what effect, or if any at all, reviews have on ranking has been a subject of hot debate for some time. Experts with years of experience aren’t even on the same page when it comes to their speculation about review impact. The conversation has been the breeding ground for everything from the conspiratorially insane to the fear saturated, downright paranoid and all that lay in between.

And despite all the chatter, people still can’t seem to agree on a consistent answer to the burning question; do reviews affect ranking?

So of course, I had to dig in and see what the data says. Regardless of what your points of view are on ranking factors, when you analyze a bunch of listings, the data always paints a picture. For this experiment in controversy I took a small sample of 540 listings across a number of random categories and random key phrases, analyzing page one of search results.

Before I go into what the listing information gave me, I want to express what my own views were on the subject.

What Is Our Position?

I, as well as my colleagues, have had the great pleasure of seeing the inner workings of hundreds of listings, seller accounts and products. With that experience we are able to put together a fairly accurate depiction of how Amazon’s algorithm works. Couple that experience with constant interaction with Amazon and a lot of the holes can be filled.

Through all that, we concluded that product reviews are NOT directly a factor in search rank. This would mean the algorithm does not account for reviews in its decision to order listings for key terms input in the search bar.

Our experience has shown us no reason to believe such things as review velocity, review rating and origin of reviews, coming at varying intervals of high or low, have not impacted ranking for keywords on any listings we’ve tested or monitored.

Our position has always been that it is customer sentiment that drives conversion rates and THAT is the indirect factor that has impacted ranking. With that said, we should look at what 540 listings had to say on the subject.

What Does the Data Say?

After reviewing nine categories, 45 keywords and the top 12 search results on page one for each (total 540) I discovered that the number of reviews and star rating didn’t appear to have an obvious impact on ranking. This is because many well established, higher ranking products were being outranked by listings with fewer reviews or lower stars.

Now, I said obvious impact, but there is certainly some compelling data to look at. For instance, out of the sample, 70% of all of page one, top twelve results had over 100 reviews. And 83% had over 50. So, while it wouldn’t appear that number of reviews guarantees a higher search rank, it is interesting to note that you appear to need at least 50 reviews to be a player in this game.

The next metric I paid attention to was star rating. While star rating seemed to vary from one ranked listing to the next, 87% of all top twelve results had AT LEAST a 4-star rating. And 92% had at least 4 stars showing.

Again, we find ourselves pondering the age old philosophical question “which came first, the chicken or the egg?” in that we don’t know if the listings ranked because of their reviews or if they have great reviews because they are so popular. However, seeing that the majority of top results have over 50 reviews consistently and at least four stars tells us more about what we as sellers are up against.

Further, the implications of what it means that 92% of top search results show at least four stars are kind of astounding. This really leads us to a better understanding of what can happen to a listing if it dips below four stars. The data supports that the horror stories we’ve all heard are, unfortunately, true.

The Verdict

It is still a complex issue. And while the data points to the clear fact that page one listings have a certain number of reviews and review rating, there is still not enough to convince us that the algorithm specifically accounts for reviews.

Be that as it may, we now have a comprehensive understanding of the landscape of page one results. As such, the strategies we employ as sellers remain the same:

Get initial reviews however possible. Put a system in place to get a continuous stream of reviews over time. Provide stellar customer support and sell a quality product so that those reviews are generally high. This will only aid in keeping your conversion rate as optimal as possible so ranking can increase or maintain elevated levels.

If you find yourself struggling to put together a system that gets your products the reviews they need, we may be able to help. With our Review Rush product, we provide for you a professional template that will make your brand stand out and pull in more reviews. And our SixLeaf products allow you to rank your listing for important keywords so you can garner more organic sales that lead to more reviews as well.