Data Process ¶
The following is the complete data workflow of UAS, and we will provide detailed explanations of each component separately.
ETL Service: ¶
This service receives alerts from external parties via APIs and transmits them directly to a Kafka message queue. Within the ETL service, Kafka consumers are responsible for retrieving alerts and determining if they are legacy data. If they are, the alerts are enriched with labels queried from the inventory database. Once this process is complete, a normalized alert is then sent to the Kafka message queue.
Aggregation Service: ¶
The Kafka consumer within this service is tasked with retrieving normalized alerts. Subsequently, the service queries monitoring rules from the MySQL database using the app_group or business service as parameters. If an alert meets any monitoring rule, with the exception of continuous firing, the service calculates continuous firing in the cache. Once the continuous firing threshold is reached, the labels in the monitoring rules are added, and the processed alert is sent to the notification service. Additionally, this processed alert is stored in Pinot for persistence.
Simultaneously, there is an additional notification auto-triggering flow. If an alert contains the label "graphQL-Pool," the failrate of a specific pool is calculated. Upon the failrate reaching the upper threshold, a firing alert is sent to the notification service. Conversely, when the failrate drops to the lower threshold, a resolved alert is sent to the notification service.
Furthermore, if monitoring rules are configured, the notification frequency can be adjusted by the threshold for continuous firing.
Notification Service: ¶
It closely collaborates with the central Alertmanager, bypasses alerts, fetches receiver definitions from a Git repo or MySQL database, and sets configmaps of routers and receivers.
Analysis Service: ¶
This service integrates with third-party systems, such as the inventory graphQL database.
AlertMananger: ¶
This service will receive alerts with firing or resolved status from the notification service, and sync receiver definitions and routers to its memory. Additionally, it is responsible for managing the lifecycle of individual alerts and notifications.
