EMS - Harness Deployment ¶

The EMS consists of three deployable components — ems (backend), ems-ui (frontend), and ems-redis (cache cluster) — running across two Kubernetes clusters (wjfkgen-p-1 as primary, wdfwgen-p-5 as secondary) in both prod and alpha environments. Deployment is driven by Harness pipelines, with service images built via Jenkins and configuration managed in a dedicated configuration repo.

Chart ¶

Configuration ¶

• We configure application settings in /config/ems-ui/em-ui.yaml and /config/ems/ems.yaml, and deployment-related settings in /deploy/ems-ui.yaml, /deploy/ems.yaml, and /deploy/ems-redis.yaml. All these configuration files are stored in the Configuration repository.

• Some secret paths are specified in the YAML files under /secrets/apps/ems.

• Chart definitions are configured in /deploy/em-ui.yaml and /deploy/ems.yaml.

Repo: https://sqbu-github.cisco.com/WebExSquared/configuration

Vault ¶

Login

Secret Path: /secret/apps/wbx-ems

Image Build ¶

Jenkins URL: https://sqbu-jenkins.wbx2.com/service05/job/team/job/wbx-ems/job/pipeline/

Pipelines ¶

Harness Pipelines URL: https://app.harness.io/ng/account/o0yschY0RrGZJ2JFGEpvdw/all/orgs/Webex/projects/Webex/pipelines

Frontend Service (ems-ui) ¶

Backend Service (ems) ¶

Redis Cluster ¶

Deploy Workflow ¶

Taking the EMS backend service as an example.

Step 1: Click the pipeline you want to deploy (Please choose either ems-ui or ems, and also be sure to specify which environment you want to deploy to).

Step 2: Click the RUN button.

Step 3: Configure pipeline variables and click Run Pipeline.

Tip: Every time you push a change to the Configuration repo, a new hash will be sent to the Jenkins Pipeline Notifications room.

Cluster Login ¶

Primary — wjfkgen-p-1 ¶

The Vault token must be copied from Vault with namespace=meetpaas/mccprod.

export VAULT_TOKEN=<your_own_token>
export CLUSTER_NAME=wjfkgen-p-1
export CLUSTER_DOMAIN=prod.infra.webex.com
export CNC_DOMAIN=prod.infra.webex.com
export CNC=mccprod
export VAULT_ADDR=https://keeper.cisco.com
export VAULT_NAMESPACE=meetpaas/mccprod
export INFRA_SERVICE_URL=https://infra.int.mccprod.prod.infra.webex.com
kubectl wbx3 login wjfkgen-p-1 --role wbmaas-mct-prod-admin

Namespaces:

Secondary — wdfwgen-p-5 ¶

export VAULT_TOKEN=<your_own_token>
export CLUSTER_NAME=wdfwgen-p-5
export CLUSTER_DOMAIN=prod.infra.webex.com
export CNC_DOMAIN=prod.infra.webex.com
export CNC=mccprod
export VAULT_ADDR=https://keeper.cisco.com
export VAULT_NAMESPACE=meetpaas/mccprod
export INFRA_SERVICE_URL=https://infra.int.mccprod.prod.infra.webex.com
kubectl wbx3 login wdfwgen-p-5 --role wbmaas-mct-prod-admin

Namespaces:

Important Notes ¶

1. Traffic management differs between clusters: - wjfkgen-p-1: traffic managed via Ingress - wdfwgen-p-5: traffic managed via Istio

2. Backend service runs on Python 3.12. Use the specified base image from containers.cisco.com/repository/webexott/base/python.

3. Frontend service runs on Node.js 22.16.0. Custom builds are required since standard images were not available: - Build repo: xiaowfu/wbx3-nodejs - Jenkins job: nodejs-build - Image pushed to: containers.cisco.com/repository/webexott/base/nodejs

4. Node labels and nodeSelector differ between clusters:

   wjfkgen-p-1:

   wb3:
     service:
       nodeSelector:
         type: ems-p0
       tolerations[0]:
         key: dedicated
         operator: Equal
         value: ems
         effect: NoSchedule
   

   wdfwgen-p-5:

   wb3:
     service:
       nodeSelector:
         dedicated: ems
         type: worker
       tolerations[0]:
         key: dedicated
         operator: Equal
         value: ems
         effect: NoSchedule