How can Apdex help you ¶

Apdex assesses user experience and complements traditional monitoring tools that monitor the internal state of the system. Apdex, being effect-oriented, focuses on the impact of system performance on user experience, while traditional monitoring, being cause-oriented, targets the identification of the source of problems. In general, Apdex benefits system owners in the following ways.

Helps improve user experience ¶

Apdex, like NPS and MOS, is a measure of user experience. As Peter Drucker once said, "what's measured improves". However, unlike NPS and MOS, which rely on subjective feedback from users, Apdex requires system owners to implement a solution that automatically and objectively evaluates the user experience of their systems. This involves collecting relevant metrics each time a function or feature is used, and applying a set of pre-determined rules or criteria to assess the user experience. This requires some extra effort, but it is a must for system owners that are looking to improve the user experience of their systems.

Provides a common language for stakeholders with different backgrounds ¶

Apdex condenses multiple metrics into a single score that is easy to understand even by non-technical stakeholders, such as customers. Thus, it provides a common language for stakeholders with different backgrounds to communicate about the user experience of a system. If you've ever used NPS or MOS, you know what we are talking about.

Allows for meaningful performance comparison ¶

Apdex measures a system's performance from the user's perspective (i.e. user experience), making it a performance measure independent of system implementation details. As a result, Apdex can be used to compare the performance of different systems that provide the same functionality, or to track the performance of an evolving system over time.

Supports effect-oriented alerting ¶

There is a significant difference between the alerts set on Apdex and those set on internal system status (such as CPU utilization): the former are effect-oriented, while the latter are cause-oriented.

The cause-oriented alerting approach excels at pinpointing which part of a system is experiencing a problem, but it has several drawbacks. In a complex system, it is difficult to set up alerts with appropriate thresholds at all potential points of failure, leading to blind spots and the possibility of not becoming aware of a problem until it is reported by customers. This approach also cannot provide information about which features and users are affected by a problem and to what extent, leading to the possibility of false alerts (such as those that occur during periods of low user activity and those that do not result in noticeable degradation of user experience due to built-in automatic retry mechanisms) and difficulty in distinguishing real alerts from false ones. This can cause system owners to either overreact or underestimate the severity of a problem until it is reported by customers.

In contrast, the effect-oriented alerting approach does not reveal the root cause of an issue. However, it requires fewer alerts to be configured as it monitors user experience by feature and there are typically fewer customer-facing features compared to potential failure points in a complex system. It is also less likely to have blind spots as long as all relevant features are monitored. Last but not least, it does not have false alerts because it only alerts if there is sufficient user activity and a noticeable decline in the user experience.

Supports fine-grained availability calculation ¶

If you consider a system 100% available to satisfied users, 50% available to tolerating users, and unavailable to frustrated users, then its Apdex score and its availability are actually the same thing. An Apdex score of 99% means 99% availability. In our opinion, this approach is superior to using just time to calculate availability because user activity levels can vary significantly over different time periods.