How we score Reddit sentiment

We’re adding a new feature to our website that allows you to see how people on Reddit view a particular product. We’re calling this feature the “Reddit Sentiment Summary” and you’ll see it appear across the website on reviews, best lists, and VS articles as a widget such as the one pictured above.

So, where does this score come from?

We find Reddit threads related to a particular product. Then, we use a JavaScript library called “Sentiment” to classify each comment as positive (for example, “I love the battery life”), negative (“the battery life is very poor”), or neutral (“do you know how to turn ANC on?”).

We tally up how many comments fall into each sentiment category—positive, negative, and neutral. Additionally, we incorporate the community’s feedback by counting upvotes and downvotes on each comment. This helps us capture cases where the broader Reddit community might disagree or strongly agree with individual comments.

Here is an example of a comment from Reddit that our sentiment tool identified as positive sentiment:

• “I love my galaxy buds…Not an audiophile, but the audio is so good” (Upvotes: 5, Downvote: 0)
  • Total Positive Sentiment = 6 (1 for the comment itself + 5 upvotes)

Here is an example of a comment from Reddit that our sentiment tool identified as neutral sentiment:

• “I’ve only tried the Buds 2 and they are a tad bright but mostly neutral. Tonality wise, can’t go wrong with Samsung products after their AKG purchase. The other alternative is Galaxy Buds Plus, they follow the Harman Target even closer. No noise cancelling though.” (Upvotes: 0, Downvote: 0)
  • Total Neutral Sentiment = 1 (just the comment itself)

Here is an example of a comment from Reddit that our sentiment tool identified as negative sentiment:

• “The lack of multipoint is what makes it a no go for me. Most premium headphones for everyday use (over ear and ear buds) support multipoint so not sure why they decided against it.” (Upvotes: 5, Downvote: 1)
  • Total Negative Sentiment = 6 (1 for the comment itself + 5 upvotes)
  • Total Positive Sentiment = 1 (due to 1 downvote, implying disagreement)

Since we are using Javascript Sentiment library to calculate the Reddit Sentiment Summary, it’s not perfectly accurate, nor are we trying to claim that this is the best method to determine whether or not a product is good. We built out an objective SoundGuys testing lab for a reason, after all. That said, crowdsourcing sentiment about a product can provide a valuable additional piece of information for our readers. Ultimately, we serve the readers of SoundGuys, and we will always provide any information we can to help you make an informed decision about what audio products to buy.

In the future, we hope to improve upon our Reddit Sentiment Summary in the following ways:

• Use more advanced sentiment analysis tools (data science libraries, ML, AI models).
• Calculate the trend of sentiments that could reflect a sudden issue with a product (e.g. +20% negative sentiments the past week for product X).
• Be more granular on the sentiment analysis for each product (battery sentiment, sound sentiment…).
• Display the Reddit threads to the user to be more transparent on the source content.

For now, you can cautiously use our Reddit sentiment scores to get a feel for what the public thinks of a certain product.

Is this a feature that is useful to you? What features do you want us to add to the website next? Let us know in the comments below.