ControlPlane resource exposes a set of configurations necessary to automatically install the Service Bridge control plane on a cluster. The installation API is an override API so any unset fields that aren’t required will use sensible defaults.

Prior to creating the ControlPlane resource, a cluster needs to be created in the management plane. Control plane install scripts would create the following secrets in the Kubernetes namespace the control plane is deployed into. Make sure they exist:

  • oap-token
  • otel-token

If your Elasticsearch backend requires authentication, ensure you create the following secret:

  • elastic-credentials

A minimal resource must have the container registry hub, telemetryStore, and managementPlane fields set.

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
  name: controlplane
  namespace: istio-system
spec:
  hub: docker.io/tetrate
  telemetryStore:
    elastic:
      host: elastic
      port: 5678
  managementPlane:
    host: tsb.tetrate.io
    port: 8443
    clusterName: cluster

To configure infrastructure specific settings such as resource limits in Kubernetes, set the relevant field in a component. Remember that the installation API is an override API so if these fields are unset the operator will use sensible defaults. Only a subset of Kubernetes configuration is available and only for individual components.

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
  name: controlplane
  namespace: istio-system
spec:
  hub: docker.io/tetrate
  imagePullSecrets:
  - name: my-registry-creds
  telemetryStore:
    elastic:
      host: elastic
      port: 5678
  managementPlane:
    host: tsb.tetrate.io
    port: 8443
    clusterName: cluster
  components:
    collector:
      kubeSpec:
        resources:
          limits:
            memory: 750Mi
          requests:
            memory: 500Mi

ControlPlaneComponentSet

The set of components that make up the control plane. Use this to override application settings or Kubernetes settings for each individual component.

Field Description Validation Rule

collector

tetrateio.api.install.controlplane.v1alpha1.OpenTelemetryCollector

oap

tetrateio.api.install.controlplane.v1alpha1.Oap

xcp

tetrateio.api.install.controlplane.v1alpha1.XCP

istio

tetrateio.api.install.controlplane.v1alpha1.Istio

rateLimitServer

tetrateio.api.install.controlplane.v1alpha1.RateLimitServer

hpaAdapter

tetrateio.api.install.controlplane.v1alpha1.HpaAdapter

onboarding

tetrateio.api.install.controlplane.v1alpha1.Onboarding
Workload Onboarding.

satellite

tetrateio.api.install.controlplane.v1alpha1.Satellite
Satellite provide load balancing capabilities for data content before the data from Envoy reaches the SPM in Control Plane. When envoy points the address to Satellite, it can load balance the traffic to the SPM service.

ngac

tetrateio.api.install.controlplane.v1alpha1.NGAC

gitops

tetrateio.api.install.common.GitOps
Configuration for the integration of the Control Plane with Continuous Deployment pipelines.

internalCertProvider

tetrateio.api.install.common.InternalCertProvider
Configure the Kubernetes CSR certificate provider for TSB internal purposes like Webhook TLS certificates. This configuration is required for kubernetes version 1.22 and above.

defaultKubeSpec

tetrateio.api.install.kubernetes.KubernetesSpec
Configure Kubernetes default settings for all components. These settings will be merged to all components’ settings, only if the component does not define the same setting. In that case, the setting defined at the component level prevails over the global default.

wasmfetcher

tetrateio.api.install.controlplane.v1alpha1.WASMFetcher
Configuration for the WASM Fetcher component.

defaultLogLevel

string
The default log level for all components if the per component log level config is not specified. Note that the supported log level for different components can be different.

ControlPlaneSpec

ControlPlaneSpec defines the desired installed state of control plane components. Specifying a minimal ControlPlaneSpec with hub, clusterName, and managementPlane set will create an installation with sensible defaults.

Field Description Validation Rule

hub

string
REQUIRED
TSB container hub path e.g. docker.io/tetrate.

string = {
  min_len: 1
}

imagePullSecrets

List of tetrateio.api.install.kubernetes.LocalObjectReference
Pull secrets can be specified globally for all components, or defined into the kubeSpec.serviceAccount of every component if needed. In case both are defined, the most specific one (the one defined at the component) level is used.

List of references to secrets in the same namespace to use for pulling any images in pods that reference this ServiceAccount. ImagePullSecrets are distinct from Secrets because Secrets can be mounted in the pod, but ImagePullSecrets are only accessed by the kubelet. More info: https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#service_account-v1-core

components

tetrateio.api.install.controlplane.v1alpha1.ControlPlaneComponentSet
The set of components that make up the control plane. Use this to override settings for individual components. These components assume the following secrets are present: oap-token and otel-token.

providerSettings

tetrateio.api.install.controlplane.v1alpha1.ProviderSettings
Configures Kubernetes provider specific settings.

managementPlane

tetrateio.api.install.controlplane.v1alpha1.ManagementPlaneSettings
REQUIRED
Configure the management plane to retrieve configuration from.

message = {
  required: true
}

meshExpansion

tetrateio.api.install.controlplane.v1alpha1.MeshExpansionSettings
Configure mesh expansion to connect workloads external to Kubernetes to the mesh.

telemetryStore

tetrateio.api.install.controlplane.v1alpha1.ControlPlaneSpec.TelemetryStore
REQUIRED
Configure the store that TSB will use to persist application telemetry data.

message = {
  required: true
}

meshObservability

tetrateio.api.install.controlplane.v1alpha1.ControlPlaneSpec.MeshObservability
Configure how the mesh should be observed, which observability functionalities should be enabled to observe your registered services in the mesh, and the store properties that TSB will use to persist application observability data like metrics, traces, logs. If omitted, a demo grade mesh observability setting will be configured for your convenience.

tier1Cluster

bool
DEPRECATED**: This should not be set through Control plane API Instead use TSB Cluster API. Indicates that this cluster is used for tier1 gateways. Tier one clusters can only contain tier 1 gateways. Non-tier1 clusters contain tier2 gateways but not tier 1.

MeshObservability

Configure how the mesh should be observed, which observability functionalities should be enabled to observe your registered services in the mesh, and the store properties that TSB will use to persist application observability data like metrics, traces, logs. If omitted, the operator will assume a demo installation and for your convenience install a demo grade mesh observability setting. Select one of the MeshObservability settings to see complete examples.

Field Description Validation Rule

demoSettings

tetrateio.api.install.common.MeshObservabilitySettings oneof mesh_observability

settings

tetrateio.api.install.common.MeshObservabilitySettings oneof mesh_observability

TelemetryStore

Configure the store that TSB will use to persist application telemetry data. Select one of the TelemetryStore settings to see complete examples.

Field Description Validation Rule

elastic

tetrateio.api.install.controlplane.v1alpha1.ElasticSearchSettings oneof telemetry_store

HpaAdapter

Kubernetes settings for the OAP (SkyWalking) HPA adapter component.

Field Description Validation Rule

kubeSpec

tetrateio.api.install.kubernetes.KubernetesComponentSpec
Configure Kubernetes specific settings

IsolationBoundary

IsolationBoundary is an isolated Istio environment which can spread across multiple revisioned control plane clusters.

Example:

isolationBoundaries:
- name: prod
  revisions:
  - name: stable
    istio:
      tsbVersion: 1.6.0
- name: staging
  revisions:
  - name: v1_6_3
    istio:
      tsbVersion: 1.6.3
  - name: v1_6_1
    istio:
      tsbVersion: 1.6.1
      disable: true

The tsbVersion field can be left empty, which would then default to the current TSB released version.

isolationBoundaries:
- name: global
  istio:
  - revisions: stable

For instance, if isolation boundaries are being added in TSB 1.6.1, the default would looks something like this:

isolationBoundaries:
- name: global
  revisions:
  - name: stable
    istio:
      tsbVersion: 1.6.1
Field Description Validation Rule

name

string
REQUIRED
Name of the IsolationBoundary.

string = {
  min_len: 1
}

revisions

List of tetrateio.api.install.controlplane.v1alpha1.IstioRevision
REQUIRED
Configure multiple Istio Revisions under the IsolationBoundary. Once IstioIsolationBoundaries is enabled, for any IsolationBoundary configured - there must be atleast one IstioRevision.

repeated = {
  min_items: 1
}

Istio

Mesh and Kubernetes settings for Istio.

Field Description Validation Rule

tsbVersion

string
OPTIONAL
Specifies the tsb release version. This is used by the tsb control plane operator in determining the xcp version, which would eventually decide Istio version.

If not provided explicitly, this defaults to the current tsb version.

mountInternalWasmExtensions

bool
When this flag is set, the TSB internal WASM extensions will be mounted into the Sidecar, Ingress and Egress gateway pods automatically. These extensions will be loaded as local files instead of being downloaded from a remote OCI registry or HTTP endpoint. This is currently disabled by default, but in future releases this flag will be enabled by default.

kubeSpec

tetrateio.api.install.kubernetes.KubernetesIstioComponentSpec
Configure Kubernetes specific settings.

traceSamplingRate

double
The percentage of traces Envoy will sample.

defaultWorkloadCertTTL

google.protobuf.Duration
The default TTL of issued workload certificates. This sets both the default client-side CSR TTL and the default server-side issued certificate TTL.

maxWorkloadCertTTL

google.protobuf.Duration
The maximum TTL that can be set in issued workload certificates.

trustDomain

string
The trust domain corresponds to the trust root of a system. Refer to SPIFFE-ID. If omitted, TSB will configure the trust domain as CLUSTER_NAME.tsb.local, where CLUSTER_NAME is the name of the cluster object in TSB for this control plane.

baseOverlays

List of istio.operator.v1alpha1.K8sObjectOverlay
The overlays applied to the Istio base component. See https://istio.io/latest/docs/reference/config/istio.operator.v1alpha1/#IstioComponentSetSpec. When this is specified, the overlay in kubeSpec.overlays are ignored.

pilotOverlays

List of istio.operator.v1alpha1.K8sObjectOverlay
The overlays applied to the Istio pilot component. See https://istio.io/latest/docs/reference/config/istio.operator.v1alpha1/#IstioComponentSetSpec. When this is specified, the overlay in kubeSpec.overlays are ignored.

cniOverlays

List of istio.operator.v1alpha1.K8sObjectOverlay
The overlays applied to the Istio CNI component. See https://istio.io/latest/docs/reference/config/istio.operator.v1alpha1/#IstioComponentSetSpec. When this is specified, the overlay in kubeSpec.overlays are ignored.

logLevels

map<string, string>
Specifies the global logging level settings for the Istio control plane components.

IstioRevision

Istio control plane settings for a specific revision.

Field Description Validation Rule

name

string
REQUIRED
Name of the IstioRevision. Must be unique at cluster level, across Isolation Boundaries. The IstioRevision name is used to deploy revisioned Istio control-plane components.

Notice that the value constraints here are stricter than the ones in Istio. Apparently, Istio validation rules allow values that lead to internal failures at runtime, e.g. values with capital letters or values longer than 56 characters. Stricter validation rules here are meant to prevent those hidden pitfalls.

string = {
  min_len: 1
  max_len: 56
  pattern: ^[a-z0-9](?:[-a-z0-9]*[a-z0-9])?$
}

istio

tetrateio.api.install.controlplane.v1alpha1.Istio
REQUIRED
Istio overlay configuration for the revision. Revision specific Istio configs will be overlayed over the common Istio configs configured in the ControlPlaneSpec.

message = {
  required: true
}

disable

bool
OPTIONAL
If set to true, Istio control plane deployment with this revision will be cleaned up from the cluster. This field can be used to clean up revisioned control plane deployment while retaining the configurations in the CR. After cleanup, it can be again set to false to re-deploy revisioned control plane. By default the value is set to false.

NGAC

Kubernetes settings for the NGAC component.

Field Description Validation Rule

enabled

bool
NGAC is an experimental component. If enabled is false, this component will not be installed.

kubeSpec

tetrateio.api.install.kubernetes.KubernetesComponentSpec
Configure Kubernetes specific settings

logLevels

map<string, string>
The log level configuration by scopes. Supported log level: “none”, “error”, “info”, “debug”.

Oap

Kubernetes settings for the OAP (SkyWalking) component.

Field Description Validation Rule

streamingLogEnabled

bool
Feature flag to determine whether on-demand streaming logs should be enabled.

onDemandEnvoyMetricsEnabled

bool
Feature flag to determine whether on-demand envoy metrics should be enabled. If enabled, the envoy proxy will provide a set of metrics that can be queried using the metrics service. OAP will provide a query API that can be used to collect envoy proxy metrics for specific pods. This is only for temporary and real-time queries that can be used, for example, for application troubleshooting use cases. These metrics are not persisted.

storageIndexMergingEnabled

bool
Feature flag to determine whether metrics/meter and records should be shard into multi-physical indices, or instead if they should be merged into a single physical index. By default “false”, metric/meter and records are sharded into multi-physical indices. Instead of sharding, if enabled by setting it to “true”, metrics/meter and records will be merged into one physical index template metrics-all and records-all. This feature flag must be set on all clusters and have the same value as the management plane’s one, otherwise control plane observability data could be written to the wrong or not existing index. In this storage mode, user can adjust each concrete index should have to scale out by setting storageSpecificIndexSettings field in the management plane install manifest.

kubeSpec

tetrateio.api.install.kubernetes.KubernetesComponentSpec
Configure Kubernetes specific settings

logLevel

string
Specifies the log level for OAP component. Supported log level: “all”, “debug”, “info”, “warn”, “error”, “fatal”, “off” and “trace”.

Onboarding

Settings for the Workload Onboarding component.

Field Description Validation Rule

operator

tetrateio.api.install.controlplane.v1alpha1.OnboardingOperator
Configure Workload Onboarding Operator component.

repository

tetrateio.api.install.controlplane.v1alpha1.OnboardingRepository
Configure Workload Onboarding Repository component.

plane

tetrateio.api.install.controlplane.v1alpha1.OnboardingPlane
Configure Workload Onboarding Plane component.

OnboardingOperator

Kubernetes settings for the Workload Onboarding Operator component.

Field Description Validation Rule

kubeSpec

tetrateio.api.install.kubernetes.KubernetesComponentSpec
Configure Kubernetes specific settings.

logLevels

map<string, string>
The log level configuration by scopes. Supported log level: “none”, “error”, “warn”, “info”, “debug”.

OnboardingPlane

Configure Workload Onboarding Plane component.

Field Description Validation Rule

instance

tetrateio.api.onboarding.config.install.v1alpha1.OnboardingPlaneInstance
Kubernetes settings for the Workload Onboarding Plane Instance component.

OnboardingRepository

Kubernetes settings for the Workload Onboarding Repository component.

Field Description Validation Rule

kubeSpec

tetrateio.api.install.kubernetes.KubernetesComponentSpec
Configure Kubernetes specific settings.

OpenTelemetryCollector

Kubernetes settings for the OpenTelemetryCollector component.

Field Description Validation Rule

kubeSpec

tetrateio.api.install.kubernetes.KubernetesComponentSpec
Configure Kubernetes specific settings

logLevel

string
Specifies the log level for OTEL collector component. Supported log level: “debug”, “info”, “warn”, “error”, “dpanic”, “panic”, and “fatal”.

RateLimitServer

Configuration settings for the RateLimit Server

Field Description Validation Rule

backend

tetrateio.api.install.controlplane.v1alpha1.RateLimitServer.Backend
REQUIRED
Configure Database backend settings. This field must be configured by the user.

message = {
  required: true
}

domain

string
The domain field allows ratelimits to be namespaced to a certain domain. To support common ratelimits across multiple clusters set this string to a common value, across them. This assumes that the same backend (uri) is being used. By default the domain is set to the name of the control plane cluster.

kubeSpec

tetrateio.api.install.kubernetes.KubernetesComponentSpec
Configure Kubernetes specific settings.

Backend

External Backend Database types. This points to the backend used by the ratelimit server as a key/value store.

Field Description Validation Rule

redis

tetrateio.api.install.controlplane.v1alpha1.RateLimitServer.Backend.RedisSettings oneof backend_specifier
Settings for redis database backend.

RedisSettings

Configuration for the External Redis Backend Database

Field Description Validation Rule

uri

string
REQUIRED
The Redis Database URI. The value of the URI decides the scope for ratelimiting across multiple clusters.

string = {
  min_bytes: 1
}

Route53Controller

Kubernetes settings for the Route53 Integration Controller component.

Field Description Validation Rule

kubeSpec

tetrateio.api.install.kubernetes.KubernetesComponentSpec
Configure Kubernetes specific settings. Note: Route53 controller requires Service Account for IAM to be created before installing the controller. Therefore the Service Account will not be managed by this kubeSpec and all Service Account configuration in this kubeSpec will be ignored.

Satellite

Kubernetes settings for the Satellite (SkyWalking-Satellite) component.

Field Description Validation Rule

enabled

bool
Satellite is an optional component. If enabled is false, this component will not be installed.

kubeSpec

tetrateio.api.install.kubernetes.KubernetesComponentSpec
Configure Kubernetes specific settings

logLevel

string
Specifies the log level for the component. Supported log level: “panic”, “fatal”, “info”, “warn”, “error”, “debug” and “trace”.

WASMFetcher

Settings for the WASM Fetcher component.

Field Description Validation Rule

cacheDisableInsecureRegistries

bool
Denies insecure registries to be used for fetching WASM modules. Defaults to false.

cacheExpiration

google.protobuf.Duration
WASM Module cache expiration time. Defaults to 24h.

cacheMaxRetries

int32
Maximum number of retries when fetching WASM modules from the OCI registry. Defaults to 5.

cachePurgeInterval

google.protobuf.Duration
WASM cache purge interval to periodically clean up the stale WASM modules. Defaults to 1h.

cacheRequestTimeout

google.protobuf.Duration
Specifies the timeout used when retrieving the WASM plugin from the OCI registry. Defaults to 15s.

kubeSpec

tetrateio.api.install.kubernetes.KubernetesComponentSpec
Configure Kubernetes specific settings

logLevels

map<string, string>
The log level configuration by scopes. Supported log levels: “none”, “error”, “info”, “debug”.

XCP

Kubernetes settings for the XCP component.

Field Description Validation Rule

centralAuthMode

tetrateio.api.install.controlplane.v1alpha1.XCP.CentralAuthMode
Authentication mode for connections from XCP Edges to XCP Central. If not set will default to mutual TLS.

configProtection

tetrateio.api.install.common.ConfigProtection
ConfigProtection contains settings for enabling/disabling config protection over XCP created resources. Config protections are disabled by default.

kubeSpec

tetrateio.api.install.kubernetes.KubernetesComponentSpec
Configure Kubernetes specific settings

isolationBoundaries

List of tetrateio.api.install.controlplane.v1alpha1.IsolationBoundary
Configures Isolated Istio environments along with Istio revisions for each environment. IsolationBoundaries can be empty when the feature flag IstioIsolationBoundaries is disabled. Once enabled, isolation boundaries can be configured.

enableHttpMeshInternalIdentityPropagation

google.protobuf.BoolValue
Enables HTTP mesh internal service identity propagation across gateway hops, utilizing the propagated identity for evaluating TSB RBAC rules. Users should enable this feature when they want to create RBAC rules around request’s origin client identity for east west traffic. The most common case for this would be when using authorization features such as ALLOW/DENY rules mode and ServiceSecuritySettings in cross-cluster environment. This feature is disabled by default.

centralProvidedCaCert

google.protobuf.BoolValue
If true, obtain the CA cert for Istio from XCP central. To enable it, the XCP Central needs to be configured with certIssuer.clusterIntermediateCASettings: {}.

logLevels

map<string, string>
Loglevel for XCP. Supported log level: “none”, “fatal”, “error”, “warn”, “info”, “debug”.

EKSSettings

Settings specific to Elastic Kubernetes Service (EKS).

Field Description Validation Rule

useNlbByDefault

bool
When true, gateways will be configured to use NLBs with cross zone load balancing enabled when the load balancer type is not configured. When false, no additional annotations will be added.

ProviderSettings

Configure Kubernetes provider specific settings.

For example to configure EKS to use network load balancers (NLB) by default:

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
  name: controlplane
  namespace: istio-system
spec:
  providerSettings:
    eks:
      useNlbByDefault: true

To configure Route53 the only option that you must specify is the Service Account name to use for IAM role. You should create the Service Account before enabling the Route53 integration controller. You can do that using eksctl. Example:

   SA_NAME=route53-controller
   CP_NAMESPACE=istio-system
   eksctl create iamserviceaccount \
   --cluster $EKS_CLUSTER_NAME \
   --name $SA_NAME \
   --namespace $CP_NAMESPACE \
   --attach-policy-arn $POLICY_ARN \
   --approve

where:

  • $EKS_CLUSTER_NAME is the name of the EKS cluster.
  • $SA_NAME is the name of the Service Account to create.
  • $CP_NAMESPACE is the namespace where the Control Plane is installed. Usually istio-system.
  • $POLICY_ARN is the ARN of the policy to attach to the Service Account - the policy should allow the Service Account to manage Route53 resources.

More details can be found in the Publishing a Service docs

After creating the Service Account you can enable the Route53 integration controller using the following configuration:

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
  name: controlplane
  namespace: istio-system
spec:
  providerSettings:
    route53:
      serviceAccountName: $SA_NAME
Field Description Validation Rule

eks

tetrateio.api.install.controlplane.v1alpha1.EKSSettings
Settings specific to EKS.

route53

tetrateio.api.install.controlplane.v1alpha1.Route53Settings
Settings specific to Route53.

Route53Settings

Settings for integration with Route53 service.

Field Description Validation Rule

serviceAccountName

string
REQUIRED
Service account name to use for IAM role. Required.

namespaceSelector

tetrateio.api.install.controlplane.v1alpha1.Route53Settings.NamespaceSelector
Specifies the namespace to watch.

policy

tetrateio.api.install.controlplane.v1alpha1.Route53Settings.Policy
Specifies the policy to use when managing DNS records. Default: SYNC.

domainFilter

List of string
List of domains to limit possible target zones by a domain suffix. Default is empty list with means consider all resources as DNS target.

interval

google.protobuf.Duration
Duration of interval between individual synchronizations. Default: 60s.

ttl

int64
Default TTL (in seconds) value for DNS records. Default: 300.

evaluateTargetHealth

google.protobuf.BoolValue
Control whether to evaluate the health of a DNS target. Default: true.

filterSettings

tetrateio.api.install.controlplane.v1alpha1.Route53Settings.FilterSettings
Filter target settings. It filters out (removes) targets that matches any of the filters. Optional.

FilterSettings

Filter settings for route53 controller.

Field Description Validation Rule

annotationFilter

string
Filter out (remove) targets that matches annotation using label selector semantics. Optional. NOTE: The annotation value currently cannot be longer thant 63 characters.

labelFilter

string
Filter out (remove) targets that matches label selector. Optional.

excludeDomain

List of string
Exclude subdomains. Optional.

zoneType

tetrateio.api.install.controlplane.v1alpha1.Route53Settings.FilterSettings.AWSZoneType
Filter out (removes) zones of this type. Default: none, options: none, public, private.

zoneTagFilter

List of string
When using the AWS provider, filter for zones with this tag. Optional, format: key=value.

zoneIdFilter

List of string
When using the AWS provider, filter for zones with this ID. Optional.

NamespaceSelector

NamespaceSelector specifies which namespaces controller will watch.

Field Description Validation Rule

namespace

string oneof namespace_selector
Specifies the namespace to watch for resources. Mutually exclusive with ignore_namespaces. If not specified (""), all namespaces will be watched which is the default.

ignoreNamespaces

string oneof namespace_selector
Comma separated list of namespaces to ignore when watching for DNS endpoints. When using this option remember to include the name of the namespace in which Control Plane is installed. If Management Plane is installed in the same cluster include the namespace name in this option as well. Mutually exclusive with namespace. Default: the namespace where the controller is running, usually istio-system.

ElasticSearchSettings

Configure an Elasticsearch connection.

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
  name: controlplane
  namespace: istio-system
spec:
  telemetryStore:
    elastic:
      host: elastic
      port: 5678
      protocol: https
      selfSigned: true
      version: 7
Field Description Validation Rule

host

string
REQUIRED
Elasticsearch host address (can be hostname or IP address).

string = {
  address: true
}

port

int32
REQUIRED
Port Elasticsearch is listening on.

int32 = {
  lte: 65535
  gte: 1
}

protocol

tetrateio.api.install.controlplane.v1alpha1.ElasticSearchSettings.Protocol
Protocol to communicate with Elasticsearch, defaults to https.

selfSigned

bool
Use Self-Signed certificates. The Self-signed CA bundle and key must be in a secret called es-certs.

version

int32
DEPRECATED: Major version of the Elasticsearch cluster. Currently supported Elasticsearch major versions are 6, 7, and 8.

int32 = {
  lte: 8
  gte: 6
}

ManagementPlaneSettings

Configure the management plane connection.

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
  name: controlplane
  namespace: istio-system
spec:
  managementPlane:
    host: tsb.tetrate.io
    port: 8443
    selfSigned: true
    clusterName: control-plane-cluster
Field Description Validation Rule

host

string
REQUIRED
Management plane host address (can be hostname or IPv4/IPv6 address).

port

int32
REQUIRED
Port management plane is listening on.

int32 = {
  lte: 65535
  gte: 1
}

selfSigned

bool
Management plane uses a self signed or private TLS certificate. If true, the CA bundle used to verify the MP’s TLS certificate must be in a secret mp-certs under the key ca.crt.

clusterName

string
REQUIRED
The name of the Cluster object that was created in the Management Plane representing this Control Plane cluster.

string = {
  min_len: 1
}

MeshExpansionSettings

Configure mesh expansion to connect workloads external to Kubernetes to the mesh.

To enable mesh expansion set it to an empty object:

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
  name: controlplane
  namespace: istio-system
spec:
  meshExpansion: {}

If external workloads are unable to communicate with the default mesh expansion gateway via external IPs or hostnames, then you must specify the gateway that enables them to do so. This custom gateway must be configured to forward this communication to the VM gateway service:

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
  name: controlplane
  namespace: istio-system
spec:
  meshExpansion:
    customGateway:
      host: customgateway.tetrate.io
      port: 15443

To automate onboarding of workloads from auto-scaling groups of VMs, you need to enable the Workload Onboarding Plane.

Workload Onboarding Agent, a component that you install next to the workload, will connect to the Workload Onboarding Plane to authenticate itself, ask permission to join the mesh, register the workload into the mesh and retrieve boot configuration required to start Istio Sidecar.

All communication between the Workload Onboarding Agent and the Workload Onboarding Plane must occur over TLS.

Therefore, to enable Workload Onboarding Plane you must provide a TLS certificate for the endpoint that exposes Workload Onboarding API to Workload Onboarding Agents.

Make sure that TLS certificate is signed by the certificate authority known to Workload Onboarding Agents.

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
  name: controlplane
  namespace: istio-system
spec:
  meshExpansion:
    onboarding:
      endpoint:
        hosts:
        - onboarding.example.org
        secretName: onboarding-tls-cert
      tokenIssuer:
        jwt:
          expiration: 1h
      localRepository: {}

To onboard workloads from custom on-premise environments, you can leverage support for OIDC ID Tokens.

If workloads in your custom environment can authenticate themselves by means of an OIDC ID Token, you can define a list of JWT issuers permitted by the Workload Onboarding Plane.

For example,

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
  name: controlplane
  namespace: istio-system
spec:
  meshExpansion:
    onboarding:
      endpoint:
        hosts:
        - onboarding.example.org
        secretName: onboarding-tls-cert
      localRepository: {}
      workloads:
        authentication:
          jwt:
            issuers:
            - issuer: "https://mycompany.corp"
              jwksUri: "https://mycompany.corp/jwks.json"
              shortName: "mycorp"
              tokenFields:
                attributes:
                  jsonPath: .custom_attributes

To ensure there will be no traffic loss when an onboarded workload gets shutdown, you can configure the time period to delay the shutdown for after deregistering the workload from the mesh, which will give enough time to reconfigure all affected mesh nodes to not load balance requests to the deregistered workload before it becomes unavailable.

For example,

apiVersion: install.tetrate.io/v1alpha1
kind: ControlPlane
metadata:
  name: controlplane
  namespace: istio-system
spec:
  meshExpansion:
    onboarding:
      endpoint:
        hosts:
        - onboarding.example.org
        secretName: onboarding-tls-cert
      localRepository: {}
      workloads:
        deregistration:
          propagationDelay: 15s
Field Description Validation Rule

customGateway

tetrateio.api.install.controlplane.v1alpha1.MeshExpansionSettings.Gateway
A custom mesh expansion gateway. This is required when the workload can’t access the default gateway directly via the external IP or hostname.

onboarding

tetrateio.api.install.controlplane.v1alpha1.MeshExpansionSettings.OnboardingPlane
Configuration of the Workload Onboarding Plane.

Gateway

A custom mesh expansion gateway. This is required when the workload can’t access the default gateway directly via the external IP or hostname.

Field Description Validation Rule

host

string
REQUIRED
Mesh expansion gateway host address (can be hostname or IP address).

string = {
  address: true
}

port

int32
REQUIRED
Port mesh expansion gateway is listening on.

int32 = {
  lte: 65535
  gte: 1
}

OnboardingPlane

Configuration of the Workload Onboarding Plane.

Field Description Validation Rule

uid

string
Unique identifier of this particular installation of the Workload Onboarding Plane.

Is used in the workload authentication flow to prevent replay attacks that abuse compromised workload credentials intended for a different installation of the Workload Onboarding Plane.

Defaults to an auto-generated UUID.

string = {
  min_len: 1
}

endpoint

tetrateio.api.install.controlplane.v1alpha1.MeshExpansionSettings.OnboardingPlane.Endpoint
REQUIRED
Configuration of the endpoint exposing Workload Onboarding API to Workload Onboarding Agents.

message = {
  required: true
}

tokenIssuer

tetrateio.api.install.controlplane.v1alpha1.MeshExpansionSettings.OnboardingPlane.TokenIssuer
Configuration of the built-in Workload Onboarding Token Issuer.

message = {
  required: true
}

localRepository

tetrateio.api.install.controlplane.v1alpha1.MeshExpansionSettings.OnboardingPlane.LocalRepository
Configuration of the local repository with DEB and RPM packages of the Workload Onboarding Agent and Istio Sidecar.

Local repository is disabled by default. To enable it, set this field to an empty value, i.e. localRepository: {}.

workloads

tetrateio.api.onboarding.config.install.v1alpha1.WorkloadConfiguration
Configuration of the workload handling.

Endpoint

Configuration of the endpoint exposing Workload Onboarding API to Workload Onboarding Agents.

Field Description Validation Rule

hosts

List of string
REQUIRED
List of hosts included in the TLS certificate.

repeated = {
  min_items: 1
  items: {string:{address:true}}
}

secretName

string
REQUIRED
Name of the secret that holds TLS certificate chain and private key.

string = {
  min_len: 1
}

TokenIssuer

Configuration of the built-in Workload Onboarding Token Issuer.

Field Description Validation Rule

jwt

tetrateio.api.install.controlplane.v1alpha1.MeshExpansionSettings.OnboardingPlane.TokenIssuer.JwtTokenIssuer oneof token_issuer
Configuration of the built-in JWT Token Issuer.

JwtTokenIssuer

Configuration of the built-in JWT Token Issuer.

Field Description Validation Rule

expiration

google.protobuf.Duration
Expiration is the duration issued tokens are valid for. Defaults to 1h.

duration = {
  required: true
  gt: {nanos:0}
}

CentralAuthMode

Authentication mode for connections from XCP Edges to XCP Central

Field Number Description

UNKNOWN

0

Default when unset, do not use

MUTUAL_TLS

1

GRPC stream is encrypted with mutual TLS

JWT

2

XCP Edges present a JWT bearer token in the GRPC headers

AWSZoneType

AWS Route53 Zone type filters.

Field Number Description

NONE

0

No filter.

PUBLIC

1

Filter public zones.

PRIVATE

2

Filter private zones.

Policy

Policy that defines how DNS records are managed.

Field Number Description

SYNC

0

Allow full synchronization.

UPSERT_ONLY

1

Don't allow delete DNS records.

CREATE_ONLY

2

Allow only creating DNS records.

Protocol

The list of supported protocols to communicate with Elasticsearch.

Field Number Description

https

0

http

1