Skip to content

What is Real User Monitoring (Apdex)

Apdex is a standardized way to report on just about any user experience by converting many measurements into a single simple to understand zero-to-one score. Apdex is similar to NPS (Net Promoter Score) or MOS (Mean Opinion Score) in that it measures the user experience rather than the internal state of a system. However, the key difference is that Apdex is calculated automatically as shown in the figure below, whereas NPS and MOS require manual input from customers.

image

(Source: https://www.apdex.org/)

Examples

Here are a few examples to help you better understand how Apdex is calculated.

Example #1

Suppose we want to monitor the user experience of a RESTful API, and the only dataset we have is its access logs. We can define its Apdex as follows.

  • Dataset: Access Logs
  • Definitions of Satisfied (Good), Tolerating (OK), and Frustrated (Bad)

    - Satisfied: HTTP Code == 200 AND Response Time <= 1 sec - Tolerating: HTTP Code == 200 AND 1 sec < Response Time <= 5 sec - Frustrated: (HTTP Code == 200 AND Response Time > 5 sec) OR HTTP Code != 200 # NOT Satisfied AND NOT Tolerating

  • Apdex = (Satisfied Count + 0.5 * Tolerating Count) / Total Count

    - Tolerating users are considered 50% satisfied, so the tolerating count is multiplied by 0.5 - Apdex falls into [0, 1]

As you can see, Apdex condenses multiple measurements into a single simple to understand zero-to-one score.

Example #2

For practical purposes, we can simplify the Apdex defined in the previous example by removing the tolerating category. Remember, don't let perfect be the enemy of good.

  • Dataset: Access Logs
  • Definitions of Satisfied (Good), Tolerating (OK), and Frustrated (Bad)

    - Satisfied: HTTP Code == 200 - Frustrated: HTTP Code != 200

  • Apdex = Satisfied Count / Total Count # Apdex = Success Rate in this simplified version

    - Apdex falls into [0, 1]

To sum up, what Apdex does is to objectively assess the user experience of a system in an automated manner. This involves 1) collecting relevant metrics each time a function or feature is used, and 2) applying a set of pre-determined rules or criteria to evaluate the user experience. It is normal for the objective score provided by Apdex to differ slightly from the subjective score provided by human beings. For practical purposes, it is also acceptable to use only two categories: satisfied (good) and frustrated (bad). Keep in mind that it is better to strive for improvement rather than perfection.