Skip to content

Operations Guide

This section is for teams that consume the observability platform — NOC, SREs, and operations engineers.

First Response

flowchart TD
    A[Alert or Reported Symptom] --> B[Capture Customer Region and Time]
    B --> C{Telemetry Current?}
    C -->|No| D[Investigate Inventory Discovery or Ingress]
    C -->|Yes| E{Service Signal Degraded?}
    E -->|Yes| F[Correlate Metrics Logs and Traps]
    E -->|No| G[Compare Peer and Expand Time Range]
    D --> H[Route to Observability Owner]
    F --> I[Route to Device or Service Owner]
    G --> F
  1. Record the customer, region, data center, symptom, and incident time range.
  2. Confirm the alert or report against current Grafana and/or Splunk evidence.
  3. Check data freshness; stale or missing telemetry can look healthy.
  4. Compare a healthy peer to determine whether the scope is one node, cluster, customer, region, or the observability platform.
  5. Correlate service metrics with logs or traps before assigning ownership.
  6. Include dashboard/search links, filters, timestamps, and target status in the handoff.

If the Prometheus target is absent or down, treat that as a visibility problem until service health is confirmed through another source.

Getting Access

Platform Guide
Grafana How to request and configure Grafana access
Splunk How to request and configure Splunk access

Using the Tools

Guide Description
Using Grafana Navigate dashboards, read alerts, create panels
Using Splunk Search logs, use saved queries, create reports

Resources