Skip to content

WebexKubed Platform Arch

What is a WebexKubed Cluster

Need to setup a kubed cluster requires infra, orchestration, package, config, deployment, LB, DNS, etc. kubed-cluster-model-layout

Infrastructure

  • Infrastructure internal provider: Openstack
  • Infrastructure public provider: AWS, Azure, GCP

Orchestration - Kubernetes

Container deployment era: Containers are similar to VMs, but they have relaxed isolation properties to share the Operating System (OS) among the applications. Therefore, containers are considered lightweight. Similar to a VM, a container has its own filesystem, share of CPU, memory, process space, and more. As they are decoupled from the underlying infrastructure, they are portable across clouds and OS distributions.

container_deployment

Kubernetes Components

Control Plane Components

  1. kube-apiserver
  2. etcd
  3. kube-scheduler
  4. kube-controller-manager
  5. cloud-controller-manager

Node Component

  1. kubelet
  2. kube-proxy
  3. Container runtime

Package

Helm

Package manager for Kubernetes. For reference usage scenarios, refer to tooling/helm 

helm install <release_name> -n <namespace> .

Helm Charts

Charts are Helm’s way of packaging Kubernetes resources into applications. WebexKubed keeps all Helm charts in the kubed-charts repo.

Configuration

Helm-cfg

Use cfg plugin to rendering helm charts from different config source, like Vault, infra service, app repo 

helm cfg build --define env=${ENV},infra_repo=${INFRA_REPO},cloud=${CLOUD},runOnPGEnable=false,service=argo

Vault

Manage Secrets & Protect Sensitive Data with Vault

Visit vault portal to check secret.

Deployment

Argo

Cloud native workflow and pipelining framework, used for various tasks in WebexKubed. The general flow is as follows: 1. Argo command is set in a PR. 2. Githook is triggered. 3. Argo Sensor picks up the message. 4. Argo workflow is triggered and begins running steps. For more information on Argo workflows, refer to argo-deployment-pipeline * Deploy pipeline workflow doc

Harness/Istio

Harness Doc: https://sqbu-github.cisco.com/pages/WebexPlatform/docs/documentation/architecture/deployment_pipeline/harness-webexkubed-deployment

Istio mesh Doc: https://cisco.box.com/s/6iqynkyjn9y2qn9zwawfoltfkuwm6bpz

Architecture Overview

component overview

Cluster Environment

  • dev
  • int
  • prod

CnC cluster

The Command and Control (CnC) Cluster is the main Cluster that kickstarts a cloud, such as mccdev, mccint, mccprod, and more. This Cluster runs SinglePane and all the tools we need to bring up separate Application Cluster. For more information, refer to the CnC docs page.

Application cluster

Application Clusters runs the actual applications managed by various application teams. They are the most common Cluster type in the WebexKubed Platform.

Observer cluster

Observer Clusters aggregate metrics from all Clusters and host Grafana dashboards to display the metrics. Thanos powers the metric aggregation from all Cluster to the Observer Cluster.