Skip to content

EMS Incident Design

Overview

EMS Incident will re-define the concept of Keep Incident, which will be a copy of ServiceNow Incident. So in Incident view, there will be only a list of SNOW Incidents and user can associate EMS Alerts to SNOW Incident. The EMS Incident will sync the status and comments from SNOW Incident but no sync back.

Incident Creation/Modification/Resolve

  • No manual status change for incident function in EMS.
  • Incident will be created when EMS get the SNOW Incident created message from rabbitmq.
  • Incident will be updated when EMS get the SNOW Incident updated message from rabbitmq.
  • Incident will be resolved when EMS get the SNOW Incident resolved message from rabbitmq.

Alert Association

  • Only single alert can associate to one Incident, if group alert associated to an Incident, only sub-alerts directly associate to the Incident.
  • User can associate one or more alerts to an Incident manually from alert detail page or bulk action.
  • Once new Incident created in EMS(sync from SNOW), there will be a backend job to check if there is any alert need to associate to the new Incident.
  • Once alert associated to an Incident, the alert detail page will show the Incident info and link to the Incident detail page.
  • Once alert associated to an Incident, the Incident detail page will show the list of associated alerts.

Incident Functionality

  • Remain current Incident view page layout but show SNOW Incident info instead of Keep Incident info.
  • Incident will support similar functionality as current Incident, such as: view detail, comment, associate alert, view associated alerts, filter, search, etc.
  • Incident will not support status change, merge, or modify SNOW related fields.
  • Incident can be enrich fields.
  • Remove some fields modification function for not modify SNOW related fields make data consistent.
  • Some fields from SNOW not included in EMS Incident view will show same as other fields in Incedent view, such as: Impact, Urgency, Priority, Assignment group, etc.
  • Support filter Incident by SNOW fields, such as: Impact, Urgency, Priority, Assignment group, etc.
  • Incident can trigger AI analysis and workflow similaer as group alert.
  • Incident resolved when SNOW Incident resolved, associated alerts status will stay no change.

Data Synchronization Strategy

Message Processing:

  • Process ServiceNow incident messages from RabbitMQ queue
  • Upsert logic: Create new incident if not exists, update if exists based on sys_id

Field Mapping Logic:

  • Map some key fields from message to 'incident_servicenow' table column
  • Store all other fields in additional_data column as JSONB

API Design

Incident Endpoints:

  • Need modify existing incident APIs to support ServiceNow fields and data.
  • Remove some incedent APIs which are not needed for SNOW Incident.

Incident Object

Remain current Incident object, continue use 'incident' table and 'lastalerttoincident' table to store the Incident info and alert-association info, add new 'incident_servicenow' table to store the SNOW specific fields.
Main fields will store in table's fields and other additional fields will store in 'additional_data' as JSONB. 

CREATE TABLE incident_servicenow (
  incident_id UUID PRIMARY KEY REFERENCES incident(id),
  sys_id VARCHAR(32) UNIQUE NOT NULL, -- sys_id
  number VARCHAR(40), -- number
  state INTEGER, -- state
  incident_state INTEGER, -- incident_state (same as state but kept for completeness)
  priority INTEGER, -- priority
  impact INTEGER, -- impact
  urgency INTEGER, -- urgency
  caller_id VARCHAR(100), -- caller_id
  assignment_group VARCHAR(100), -- assignment_group
  assigned_to VARCHAR(100), -- assigned_to
  opened_by VARCHAR(100), -- opened_by
  resolved_by VARCHAR(100), -- resolved_by
  closed_by VARCHAR(100), -- closed_by
  opened_at TIMESTAMP, -- opened_at
  resolved_at TIMESTAMP, -- resolved_at
  closed_at TIMESTAMP, -- closed_at
  due_date TIMESTAMP, -- due_date
  short_description TEXT, -- short_description
  description TEXT, -- description
  cause TEXT, -- cause
  close_code VARCHAR(50), -- close_code
  close_notes TEXT, -- close_notes
  resolution_code VARCHAR(50), -- u_resolution_category
  resolution_notes TEXT, -- resolution_notes (not in message)
  business_service VARCHAR(100), -- business_service
  impacted_services VARCHAR(255), -- u_impacted_service
  business_impact VARCHAR(100), -- business_impact
  sys_created_by VARCHAR(100), -- sys_created_by
  sys_updated_by VARCHAR(100), -- sys_updated_by
  additional_data JSONB, -- all u_* custom fields and other fields
  created_at TIMESTAMP DEFAULT NOW(),
  updated_at TIMESTAMP DEFAULT NOW()
);

Database Indexes and Constraints:

-- Primary indexes for performance
CREATE INDEX idx_incident_servicenow_sys_id ON incident_servicenow(sys_id);
CREATE INDEX idx_incident_servicenow_number ON incident_servicenow(number);
CREATE INDEX idx_incident_servicenow_state ON incident_servicenow(state);
CREATE INDEX idx_incident_servicenow_priority ON incident_servicenow(priority);
CREATE INDEX idx_incident_servicenow_assignment_group ON incident_servicenow(assignment_group);
CREATE INDEX idx_incident_servicenow_opened_at ON incident_servicenow(opened_at);
CREATE INDEX idx_incident_servicenow_resolved_at ON incident_servicenow(resolved_at);

-- Composite indexes for common queries
CREATE INDEX idx_incident_servicenow_state_priority ON incident_servicenow(state, priority);
CREATE INDEX idx_incident_servicenow_assignment_state ON incident_servicenow(assignment_group, state);

-- JSON field indexes for custom fields (if needed)
CREATE INDEX idx_incident_servicenow_additional_data_gin ON incident_servicenow USING GIN(additional_data);

-- Foreign key constraint
ALTER TABLE incident_servicenow ADD CONSTRAINT fk_incident_servicenow_incident 
  FOREIGN KEY (incident_id) REFERENCES incident(id) ON DELETE CASCADE;