The Pipeline plugin lets you create or select a Cluster to run the steps of the pipeline on, install secrets to the cluster, create deployments, or create storage buckets.

Creating and using clusters

The cluster definition of a Banzai Cloud CI pipeline can either create a new cluster, or select an existing one to use.

To select an existing cluster, you only have to specify its name:

cluster:
  name: my_cluster

You can use one of the following actions:

action Description If exists If not
CreateCluster Creates a new cluster unconditionally Fails Succeeds
EnsureCluster (default) Uses existing or creates new cluster Succeeds Succeeds
GetCluster Uses existing cluster Succeeds Fails

To make this step able to create the cluster, you will have to:

  • specify a Pipeline secret for the cloud provider,
  • select the cloud provider and the location,
  • specify the provider specific details of the cluster (default values provided for EKS and GKE).

Example configuration for a simple GKE cluster:

cluster:
  action: EnsureCluster  # default
  secret:
    name: my_gke_secret
  name: my_cluster
  cloud: google
  location: us-central1-a
  properties:
    gke:
      nodePools:
        pool1:
          count: 1
          instanceType: n1-standard-1
      nodeversion: "1.10"
      master:
        version: "1.10"

To create an Amazon EKS cluster, you can start with an example like this:

cluster:
  action: EnsureCluster
  secret:
    name: my_aws_secret
  name: my_cluster
  cloud: amazon
  location: us-west-2
  properties:
    eks:
      version: "1.10"
      nodePools:
        pool1:
          instanceType: t2.medium
          spotPrice: "0.2"
          autoscaling: false
          count: 1

This will create a single node cluster, which can be used for testing, because the following defaults are applied:

cluster:
  postHooks:
    InstallMonitoring:
      enable: true
  properties:
    eks:
      version: "1.10"
      nodePools:
        pool1:
          instanceType: t2.medium
          spotPrice: "0.2"
          autoscaling: false
          minCount: 1
          maxCount: 1
          count: 1
          image: ami-0ea01e1d1dea65b5c

Similar results can be achieved on the other providers starting from the following snippets:

cluster:
  secret:
    name: my_oracle_secret
  name: my_cluster
  cloud: oracle
  location: eu-frankfurt-1
  properties:
    oke:
      nodePools:
        pool1:
          autoscaling: false
          count: 1
          version: v1.10.3
          image: Oracle-Linux-7.5
          shape: VM.Standard2.1
          labels:
            type: general
cluster:
  action: EnsureCluster
  secret:
    name: my_azure_secret
  name: my_cluster
  cloud: azure
  location: westeurope
  properties:
    aks:
      resourceGroup: name_of_resource_group
      kubernetesVersion: "1.10.8"
      nodePools:
        pool1:
          instanceType: Standard_B2ms
          autoscaling: false
          count: 1
cluster:
  secret:
    name: my_alibaba_secret
  name: my_cluster
  cloud: alibaba
  location: eu-west-1
  properties:
    acsk:
      regionId: eu-central-1
      zoneId: eu-central-1a
      masterInstanceType: ecs.sn1ne.large
      masterSystemDiskCategory: cloud_efficiency
      nodePools:
        pool1:
          count: 1
          image: centos_7
          instanceType: ecs.sn1ne.large
          systemDiskCategory: cloud_efficiency

Pipeline client as steps

To call other functions of the Pipeline, you can explicitly use the banzaicloud/ci-pipeline-client:0.7 image (plugin) in your pipeline steps.

Deploying Helm charts

To deploy a helm chart to the cluster running your CI/CD pipeline, you can use the CreateDeployment action.

You can either deploy a chart in the workspace:

cluster:
  name: my_cluster

pipeline:
  deploy_chart:
    image: banzaicloud/ci-pipeline-client:0.7
    action: CreateDeployment
    deployment:
      name: 'nodejs-mongodb'
      reuse: true
      releaseName: 'nodejs-mongodb'
      package: '@./nodejs-mongodb-1.0.0.tgz'
      values:
        application:
          deployment:
            image:
              repository: banzaicloud/nodejs-mongodb

The chart to deploy is specified in the package parameter, which is a filename in the workspace, prefixed with an @ sign. You can use the reuse parameter to update a release when it already existed instead of trying to create it.

Alternatively you can deploy charts from configured repositories:

pipeline:
  deploy_wordpress:
    image: banzaicloud/ci-pipeline-client:0.7
    action: CreateDeployment
    deployment:
      name: stable/wordpress
      releaseName: wp
      values:
        server:
          persistentVolume:
            enabled: false

Installing secrets

To install a secret from Pipeline to your cluster, you can use the InstallSecret action.

If you want to install the secret in the same format as it's stored in, then use a step like this one:

pipeline:
  install_secret:
    image: banzaicloud/ci-pipeline-client:0.7
    action: InstallSecret
    clusterSecret:
      sourceSecretName: pipeline-secret-name
      name: installed-secret-name
      namespace: default
      merge: true

This will install all fields of the Pipeline secret named like the value of the source_secret_name parameter to the cluster. The name parameter specifies the name of the secret in the cluster. A true merge parameter means that the fields will be added to a possibly existing cluster secret, overriding the existing values.

You can also specify the layout of the installed secret by listing the fields to install. If you specify a single field name, all the secret components will be serialized to a JSON value in the named field.

pipeline:
  install_secret:
    image: banzaicloud/ci-pipeline-client:0.7
    action: InstallSecret
    clusterSecret:
      sourceSecretName: pipeline-secret-name
      name: installed-secret-name
      namespace: default
      spec:
        - name: fieldname.json

You can also transform the keys to different scalar keys:

pipeline:
  install_secret:
    image: banzaicloud/ci-pipeline-client:0.7
    action: InstallSecret
    clusterSecret:
      sourceSecretName: my-foobar-secret
      name: foobar-secret
      namespace: default
      spec:
        - name: FOOBAR_USERNAME
          source: username
        - name:  FOOBAR_PASSWORD
          source: password

Please note that you don't have to install a secret to the cluster if you only need it for a step of your CI/CD pipeline, you can use the secretFrom parameter for any step.

Create object store bucket

You can create a storage bucket with the CreateBucket or EnsureBucket action. The latter one succeeds even if the bucket already exists.

The following examples will work with the different providers:

pipeline:
  ensure_bucket:
    image: banzaicloud/ci-pipeline-client:0.7
    action: EnsureBucket
    secret:
      name: my_aws_secret
    bucket:
      properties:
        amazon:
          location: us-west-1
pipeline:
  ensure_bucket:
    image: banzaicloud/ci-pipeline-client:0.7
    action: EnsureBucket
    secret:
      name: my_google_secret
    bucket:
      properties:
        google:
          location: us-central1-a
pipeline:
  ensure_bucket:
    image: banzaicloud/ci-pipeline-client:0.7
    action: EnsureBucket
    secret:
      name: my_azure_secret
    bucket:
      properties:
        azure:
          location: westeurope
          storageAccount: storage_account
          resourceGroup: name_of_resource_group

Updating cluster

You can change the specification of an existing cluster with the UpdateCluster action. For example to change the node count of a node pool of the GKE cluster running your pipeline define a step like the following:

pipeline:
  update_cluster:
    image: banzaicloud/ci-pipeline-client:0.7
    action: UpdateCluster
    properties:
      gke:
        nodePools:
          mypool:
            count: 4