In our constant obsession with what exactly makes Amazon’s ranking algorithm tick, we decided to turn our attention to certain nuances that have long been under speculation. One is listing “activity” or otherwise helpful and unhelpful review votes.
Some have wondered whether tampering with reviews in any way would affect, positively or negatively, keyword ranking. While this is one of those small factors that it would be impossible to determine fully, we still thought it would be worthwhile to look for any obvious correlations. So we narrowed our focus to helpful reviews, and set out to see if those helpful votes appeared to be connected at all with the ranking of the product.
We only looked at top ranking products (first 10 on page one for major keywords). Then we analyzed the top 10 most helpful reviews. We analyzed almost 1300 reviews and now we’re here to share what we found.
What We Are Supposing
If helpful votes were to have a positive effect on ranking, we would necessarily see patterns such as the listings with the largest number of helpful votes outranking those with less. Or perhaps listings with large numbers of helpful votes for positive reviews would dominate page one search results.
Other correlations may also appear but we wouldn’t see them until the data was analyzed.
1264 Reviews Worth of Data
Because we were looking at so many reviews, and so many listings, we weren’t able to look at a large number of keywords. But in our small sample size of keywords we looked to see if any of the listings with the most helpful reviews (as in, largest number of up votes) made it to the top three on the page. Only seven out of thirteen instances showed listings with the largest number of helpful votes in the top ten most helpful reviews within the first three search results. This even split does not support a strong correlation.
19% of the 1264 most helpful reviews were negative reviews (1, 2 or 3 stars). Only 17.6% of the top most helpful voted (the one with the MOST votes on each listing) were negative reviews. Whereas a WHOPPING 60.5% of all the helpful reviews were positive (4 and 5 star) and 72% of the top most helpful voted were positive.
So, there is a very low number of listings on page one that have “helpful” negative reviews or the most votes going to negative reviews. Meanwhile there is a large number of listings on page one of a number of popular keywords that have positive reviews voted as helpful as well as a large number whose largest number of votes went to a helpful review.
What Does This Mean?
Remember correlation does NOT mean causation, so perhaps these highly ranked listings were doing well before they garnered so much review activity. It is entirely plausible that these listings were high ranking already and the reviews were already there to be seen and voted on for a long time.
To support that, only 10% of all the reviews were from 2017 and of those only 8 of them had more than five helpful votes. The idea here being that if all it took was helpful votes, then newer ranking products would have more recent reviews and tons of helpful votes (since, you know, there are always people out there gaming the system).
Despite the fact that there are so many helpful votes for positive reviews on listings featured on page one of major keywords, it is our opinion that there isn’t a strong enough correlation to signify a key ranking factor.
More, this information just leads us to what we already know and that is if you sell a quality product you are more likely to garner positive reviews and support for those reviews. And top search rankings are typically dominated by quality products.
Other Interesting Findings
One interesting thing to note is that 27.6% of all of the listings we analyzed had ONLY positive reviews in their top 10 most helpful. We didn’t find any that only had negative reviews, but we found a few that had mostly negative.
In fact, if you were thinking about selling yoga mats….don’t. That niche is ruthless. So many listings had hundreds of helpful votes on multiple negative reviews that it is hard to believe that is organic. Especially for listings ranking so high for such a strong keyword.