Scroll Top

Kustomized Helm: Kubernetes Resource Management Made Easy

DevOps & Automation

Kubernetes, often abbreviated as K8s, is a powerful container orchestration system that has emerged as the go-to platform for containerized microservice workloads. K8s offers a rich set of features for deploying applications, services, and configurations. However, as applications grow in complexity, handling configuration files can become overwhelming.

The two of the most popular tools which are used to simplify the management of configurations in K8s are Helm and Kustomize. Both are having its own set of pros and cons.

In this blog, we will discuss those details and explore how they complement each other.

Helm

Helm is often referred to as the “package manager for K8s” which simplifies the installation and management of K8s applications using the concept of templating. It also introduces the concept of charts, which are pre-configured templates for K8s resources as a single unit. You can also define reusable, parameterized templates and share them with your team, significantly simplifying the process of deploying applications with consistent configurations. You can also manage release versioning with rollbacks.

example: Consider you have an application that consists of a deployment, a service, and an ingress. With Helm, you can create a chart containing all these resources under templates and parameterize certain things that you may want to change based on the values you pass while you are installing like environment-based values.

Kustomize

Kustomize is a native K8s tool for customizing K8s objects. While Helm works better in the templating, Kustomize works in the concept of overlays. Kustomize introduces a template-free way to customize application configuration without modifying the original resource files using a declarative approach. Kustomize employs overlays to apply specific changes for different environments, making it ideal for managing configurations across multiple environments of your application.

example: Consider the same application above and if we want to change, add, or remove some configuration. we can use Kustomize to set up the base resources and patch the configuration on top of the base resources based on different overlays. Kustomize also benefits in moving the repeated parts of the config across resources to the overlays layer to easily manage those configs.

The Powerful Synergy

While Helm and Kustomize are powerful on their own, combining their strengths, we can create an efficient, and scalable approach to K8s resource management. Let’s discuss how to take leverage of this. There are multiple ways we can combine these two tools, we are going to discuss two of those options.

Sample application

A simple nginx web server with a k8s deployment and a service associated with that.

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: "{{ .Values.image.repo }}:{{.Values.image.tag}}"
          imagePullPolicy: "{{ .Values.image.pullPolicy }}"
          ports:
            - containerPort: 80
---
# service.yaml
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    app: nginx
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

Helm files

Helm chart setup with container image configurations templated and passed as values files while installing the chart.

# Chart.yaml
apiVersion: v2
name: kustomized-helm
description: Sample project with Kustomise + Helm
type: application
version: 0.1.0
appVersion: "0.0.1"
---
# values.yaml
image:
  repo: nginx
  tag: 1.14.2
  pullPolicy: Always

Kustomize files

Kustomize overlays set up with the number of replicas, common labels, name suffixes, and the container security configuration patched based on the environment.

# overlays/dev/kustomization.yaml
bases:
  - ../../base
nameSuffix: "-dev"
commonLabels:
  env: dev
patches:
- path: dev.yaml
  target:
    kind: Deployment
---
# overlays/dev/dev.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: placeholder
spec:
  replicas: 1

---
# overlays/prod/kustomization.yaml
bases:
  - ../../base
nameSuffix: "-prod"
commonLabels:
  env: prod
patches:
- path: prod.yaml
  target:
    kind: Deployment
---
# overlays/prod/prod.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: placeholder
spec:
  replicas: 5
  securityContext:
    runAsUser: 1000
    runAsGroup: 3000
    fsGroup: 2000
  template:
    spec:
      containers:
        - name: nginx
          securityContext:
            allowPrivilegeEscalation: false

1. Template First Approach

In the template first approach, we will first use the helm to generate the template output from the charts and then use kustomize to set up the overlays based on the environment.

├── base #empty base, helm output will be updated here
│ └── kustomization.yaml
├── Chart.yaml
├── kustomization.yaml
├── overlays
│ ├── dev
│ │ ├── dev.yaml
│ │ └── kustomization.yaml
│ └── prod
│ ├── kustomization.yaml
│ └── prod.yaml
├── templates #application
│ ├── deployment.yaml
│ └── service.yaml
└── values.yaml

# base/kustomization.yaml
resources:
  - out.yaml
  • Generating the chart output by helm template and pipe the output to the base/out.yaml.
helm template . > base/out.yaml
  • Next by using the generated output from the helm as the base, running kustomize to patch based on the environment.
kustomize build overlays/dev (or) kustomize build overlays/prod

# pipe it to kubectl for apply
| kubectl apply -f -

pros:

  • Can fully utilize helm features.

cons:

  • No release versioning and rollbacks.

2 . Overlay First Approach

├── base #application in kustomize base
│ ├── deployment.yaml
│ ├── kustomization.yaml
│ └── service.yaml
├── Chart.yaml
├── overlays
│ ├── dev
│ │ ├── dev.yaml
│ │ └── kustomization.yaml
│ └── prod
│ ├── kustomization.yaml
│ └── prod.yaml
├── templates #empty helm templates, kustomize outpu will be updated here
└── values.yaml
# base/kustomization.yaml
resources:
  - deployment.yaml
  - service.yaml
  • Generate the kustomize output based on the environment and pipe the output to the templates directory.
kustomize build overlays/dev > templates/out.yaml

(or)

kustomize build overlays/prod > templates/out.yaml
  • Use the generated output to run the helm templates.
helm template . (or) helm upgrade webserver . -n webserver --install

pros:

  • Helm’s packaging and versioning
  • Release versioning and rollbacks

cons:

  • Limited helm benefits (kustomize conflicts with the helm templating)

Summary

There are so many use cases and possibilities when we combine these two with some limitations. But it really helps when you will change the configurations too often or if you have different environment-based changes or if you want to add cross-cutting fields or restrict resources to a namespace.

I hope this blog post is useful.

References

https://jfrog.com/blog/power-up-helm-charts-using-kustomize-to-manage-kubernetes-deployments/

Ibrahim, Mohamed

+ posts