This page is going to cover the Impact analysis for NPS data points. How it is calculated, what we use to determine if a comment is positive or negative, and more.

Question: I don't know what the numbers next to my negative and positive Topics mean.

Answer: They are called "Impact Numbers" and exist to show you how much that Topic affects your NPS

Example Impacts

For something like NPS score, the potential range is 200 total points: from -100 to positive 100. A normal score starts at 0, and then changes based on the type of feedback it receives.

In the above image, you can see the "App" Topic is bringing the NPS down by 9.7 points, and bringing the NPS up by 13.5 points. By taking (-9.7 + 13.5), it leaves us a total of 3.8 positive points contributing to the overall NPS. If you were to turn all of the negative app comments positive, you would gain 9.7 NPS points to your overall score.

Impacts are done on the scale of 200. The negative impacts explain the deviation from the maximum value (+100) and the positive impacts explain the deviation from minimum value (-100). A handy formula can be seen as: positive impact + |negative impact| = 200

Question: How is it determined if a comment is put into the Negative or Positive Impact for a Topic?

Answer: The score, the sentiment, and associated topics

If I submit a comment saying "I could not log in and the customer rep was rude" and I leave a score of 3, Lumoa will put that comment into Topics like Login and Customer Service Attitude. It will add them to the Negative Impact of those categories because the customer left a Negative score and is speaking negatively in the open text response. However, it's worth noting that resolving JUST ONE of these issues will not turn this comment into a positive score. They are talking about two separate issues, and you will need to fix both in order to make this customer a promoter.

Example Issue

Additionally, neutral comments can be flagged as either positive or negative depending on the algorithm the AI uses when looking at customer sentiment in the open text response. Our system uses AI to analyze comments and determine whether or not a sentence is positive or negative, and then uses that sentiment to push a comment in either direction.

As an example, if I submit a comment saying "great customer service" with a score of 7, that comment will be placed into the Positive Impact for the Customer Service Topic. This is because even though in a vacuum a score of 7 is neutral, our AI can determine that this comment is overall positive, and assign it to an Impact accordingly.

In the below image you can see this in action. Here is a comment with a score of 7, being placed in to the negative Impact Topic of Stability, Bugs, and Freezes, because the comment sentiment is Negative.

Fully Resolved Issue

Note: Comments that have a negative sentiment in their open text response, but leave a score of 10, will not show up in the negative Impact for a Topic. This is because while the customer is talking about the Topic negatively, it is not enough to actually affect the score. Additionally, resolving the issue wouldn't gain you any additional NPS, since they have already left you the highest possible score.

Question: How are the Impact numbers next to my Topics calculated?

Answer: The Impact is based off of the amount of associated feedback and the sentiment of those feedback

The exact numbers are determined by analyzing sentiment, scores, and amounts of feedback associated with a certain topic.

First, we split each comment into Statements, which are usually sentences. Feel free to go and read more about what a Statement is. Then for each of those Statements, we use AI to determine sentiment and topics being talked about. We compare those Statements with the scores, and use predictive analysis to determine how fixing those issues would Impact your score (note that there can be multiple issues for each open text response as explained above). We then aggregate that data, factoring in the amount of feedback associated with a certain topic.

As an example, the Impact will be lower for a topic with 5 comments than one with 100 comments, because you have more to gain from fixing the issue with 100 comments. A higher Impact score means that more people are talking about this topic in their open text response, and you can use the positive and negative impact to determine how your customers are feeling about a topic overall.
Was this article helpful?
Thank you!