Keda ScaledObject As Code Using CDK8S

Keda ScaledObject As Code Using CDK8S

ยท

5 min read

Play this article

Abstract

  • keda is Kubernetes Event-driven Autoscaling and it's been used wisely now. In this blog, it provides the way to create Keda scaledobject CRD as code using CDK8S typescript.

  • With importing Keda CRDs and using CDK8S you can create Keda scaledobjects using your familiar programming languages such as typescript as scale.

Table Of Contents


๐Ÿš€ Pre-requisite

  • Install typescript, node, and cdk8s as well as projen (optional) which is a tool for managing project configuration as code.

  • Getting started with cdk8s

๐Ÿš€ Overview of Keda

  • KEDA works alongside standard Kubernetes components like the Horizontal Pod Autoscaler and can extend functionality without overwriting or duplication.

  • KEDA supports multiple triggers within an Scaledobject. Each trigger is exposed split as a metric and the HPA Controller does a MAX between all the metrics.

๐Ÿš€ Import Keda CRDs

  • Keda does not provide its CRDs separately so we can find the manifest in GH release section. Here I import the current latest version of keda v2.8.0 and output the imports folder in src/imports

      โšก $ cdk8s import https://github.com/kedacore/keda/releases/download/v2.8.0/keda-2.8.0.yaml --output src/imports/
      ------------------------------------------------------------------------------------------------
      A new version 2.0.88 of cdk8s-cli is available (current 2.0.13).
      Run "npm install -g cdk8s-cli" to install the latest version on your system.
      For additional installation methods, see https://cdk8s.io/docs/latest/getting-started
      ------------------------------------------------------------------------------------------------
      Importing resources, this may take a few moments...
      keda.sh
        keda.sh/clustertriggerauthentication
        keda.sh/scaledjob
        keda.sh/scaledobject
        keda.sh/triggerauthentication
    
  • Import result

      โšก $ tree src/imports/
      src/imports/
      โ””โ”€โ”€ keda.sh.ts
    
      0 directories, 1 file
    

๐Ÿš€ Write code

  • Overview of keda scaledObjects in this post

  • It's much more convenient to use visual code writing KEDA scaledobject in typescript language. We can read the document and find all references of construct, objects and properties of KEDA CRDs

  • This blog provides the use case of creating scaledObject (SO) for Apache airflow worker component. It contains 3 triggers (Scalers) in the SO

1. Cron - Scale applications based on a cron schedule.

  • Airflow server scheduled pipelines and worker components are scaled out at that time, but it takes time to start nodes, join node to cluster and Pod ready (about 2-3mins) so we use cron to pre-scale workers

2. PostgreSQL - Scale applications based on a PostgreSQL query.

  • This scaler based on the output of the query command to scaleout works, here we count the number of running/queued airflow task instances belonging to the scheduled pipeline (eg. running reports). Divide the count number by Airflow worker concurrency (eg. 16 as default)

  • targetQueryValue: '1.5' - The result of above calculation divide to this target value to decide how many pods will be scaled out

3. CPU - Scale applications based on cpu metrics

  • This is optional as it ensures provisioning workers when CPU Utilization is higher than 80%

  • PostgreSQL Scaler requires TriggerAuthentication to provide a password of airflow user in order to query the database. The credential is get from K8S secret airflow-secret within the airflow namespace

        const pgAuth = new TriggerAuthentication(this, 'KedaPostgresAuthentication', {
          metadata: {
            name: 'keda-airflow-postgresql-auth',
            namespace: 'airflow',
          },
          spec: {
            secretTargetRef: [{
              parameter: 'password',
              name: 'airflow-secret',
              key: 'postgresql-password',
            }],
          },
        });
    
  • Some SO specs need to know

    • pollingInterval: This is the interval to check each trigger on. By default KEDA will check each trigger source on every ScaledObject every 30 seconds. So to reduce the query connections/workload to airflow database we need to care about this value.

    • cooldownPeriod: The period to wait after the last trigger is reported active before scaling the resource back to 0.

๐Ÿš€ Build keda scaledobjects from code

  • Source code:

      โšก $ tree src/
      src/
      โ”œโ”€โ”€ imports
      โ”‚   โ””โ”€โ”€ keda.sh.ts
      โ”œโ”€โ”€ keda-airflow.ts
      โ””โ”€โ”€ main.ts
    
      1 directory, 3 files
    
  • Build resource

      โšก $ npx projen build
      ๐Ÿ‘พ build ยป default | ts-node --project tsconfig.dev.json .projenrc.ts
      ๐Ÿ‘พ build ยป compile | tsc --build
      ๐Ÿ‘พ build ยป post-compile ยป synth | cdk8s synth
      No manifests synthesized
      ๐Ÿ‘พ build ยป test | jest --passWithNoTests --all --updateSnapshot
      No tests found, exiting with code 0
      ----------|---------|----------|---------|---------|-------------------
      File      | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
      ----------|---------|----------|---------|---------|-------------------
      All files |       0 |        0 |       0 |       0 |
      ----------|---------|----------|---------|---------|-------------------
      ๐Ÿ‘พ build ยป test ยป eslint | eslint --ext .ts,.tsx --fix --no-error-on-unmatched-pattern src test build-tools projenrc .projenrc.ts
    
  • Manifest yaml file

      โšก $ tree dist/
      dist/
      โ””โ”€โ”€ keda
          โ””โ”€โ”€ airflow-keda-so.yaml
    
      1 directory, 1 file
    

๐Ÿš€ Apply and test

  • Apply manifest and check the result

      # k apply -f dist/keda/airflow-keda-so.yaml
    
      # k get so -n airflow
      NAME               SCALETARGETKIND       SCALETARGETNAME   MIN   MAX   TRIGGERS   AUTHENTICATION                 READY   ACTIVE   FALLBACK   AGE
      airflow-worker-1   apps/v1.StatefulSet   airflow-worker    2     12    cron       keda-airflow-postgresql-auth   True    True     False      2d21h
    
      # k get hpa -n airflow
      NAME                        REFERENCE                    TARGETS                                  MINPODS   MAXPODS   REPLICAS   AGE
      keda-hpa-airflow-worker-1   StatefulSet/airflow-worker   500m/1 (avg), 500m/1 (avg) + 2 more...   2         12        2          2d21h
    

๐Ÿš€ Conclusion

  • Within the Scaledobject class, you can jump to the definition to understand the meaning of each property and also know which required/optional attributes.

  • We can create a custom construct and based on that provision multiple KEDA scaledobjects with customised specs/meta such as min/max/desired replicas, triggers, trigger authentication, etc.