Do “Helpful” Review Votes Affect Ranking?

In our constant obsession with what exactly makes Amazon’s ranking algorithm tick, we decided to turn our attention to certain nuances that have long been under speculation. One is listing “activity” or otherwise helpful and unhelpful review votes.

Some have wondered whether tampering with reviews in any way would affect, positively or negatively, keyword ranking. While this is one of those small factors that it would be impossible to determine fully, we still thought it would be worthwhile to look for any obvious correlations. So we narrowed our focus to helpful reviews, and set out to see if those helpful votes appeared to be connected at all with the ranking of the product.

We only looked at top ranking products (first 10 on page one for major keywords). Then we analyzed the top 10 most helpful reviews. We analyzed almost 1300 reviews and now we’re here to share what we found.

What We Are Supposing

If helpful votes were to have a positive effect on ranking, we would necessarily see patterns such as the listings with the largest number of helpful votes outranking those with less. Or perhaps listings with large numbers of helpful votes for positive reviews would dominate page one search results.

Other correlations may also appear but we wouldn’t see them until the data was analyzed.

1264 Reviews Worth of Data

Because we were looking at so many reviews, and so many listings, we weren’t able to look at a large number of keywords. But in our small sample size of keywords we looked to see if any of the listings with the most helpful reviews (as in, largest number of up votes) made it to the top three on the page. Only seven out of thirteen instances showed listings with the largest number of helpful votes in the top ten most helpful reviews within the first three search results. This even split does not support a strong correlation.

19% of the 1264 most helpful reviews were negative reviews (1, 2 or 3 stars). Only 17.6% of the top most helpful voted (the one with the MOST votes on each listing) were negative reviews. Whereas a WHOPPING 60.5% of all the helpful reviews were positive (4 and 5 star) and 72% of the top most helpful voted were positive.

So, there is a very low number of listings on page one that have “helpful” negative reviews or the most votes going to negative reviews. Meanwhile there is a large number of listings on page one of a number of popular keywords that have positive reviews voted as helpful as well as a large number whose largest number of votes went to a helpful review.

What Does This Mean?

Remember correlation does NOT mean causation, so perhaps these highly ranked listings were doing well before they garnered so much review activity. It is entirely plausible that these listings were high ranking already and the reviews were already there to be seen and voted on for a long time.

To support that, only 10% of all the reviews were from 2017 and of those only 8 of them had more than five helpful votes. The idea here being that if all it took was helpful votes, then newer ranking products would have more recent reviews and tons of helpful votes (since, you know, there are always people out there gaming the system).

Despite the fact that there are so many helpful votes for positive reviews on listings featured on page one of major keywords, it is our opinion that there isn’t a strong enough correlation to signify a key ranking factor.

More, this information just leads us to what we already know and that is if you sell a quality product you are more likely to garner positive reviews and support for those reviews. And top search rankings are typically dominated by quality products.

Other Interesting Findings

One interesting thing to note is that 27.6% of all of the listings we analyzed had ONLY positive reviews in their top 10 most helpful. We didn’t find any that only had negative reviews, but we found a few that had mostly negative.

In fact, if you were thinking about selling yoga mats….don’t. That niche is ruthless. So many listings had hundreds of helpful votes on multiple negative reviews that it is hard to believe that is organic. Especially for listings ranking so high for such a strong keyword.

By | 2017-08-10T17:28:55+00:00 February 1st, 2017|Ranking Factors|

About the Author:

  • Mark Norton

    Hi

    While there may be no evidence that it affects ranking directly, do you think its a reasonable assumption that there would be a significant increase in conversion rate if a products visible reviews were all positive, versus if they weren’t?

  • Robert Goodwin

    My experience with my own products is that increasing order trends by product results in higher page ranking for the products and the opposite that decreasing order trends by product result in lower page ranking.

    I also see a correlation between number and frequency of positive reviews in higher page ranking.

    Also, FBA products seem to influence page ranking.

    And finally, lower price seems to rank higher if all things above are equal including the same keyword.

    It is difficult to point to any single algorithm; they are all important and worth improving each.

    Rob Goodwin
    Two Cool Inc

Share This

Share This

Share this post with your friends!