Kubernetes is the default orchestration engine for containers. Its options for controlling and managing pods and containers include:
Each of these features has its own purpose, with the common function to ensure that pods run continuously. In failure scenarios, these controllers either restart or reschedule pods to ensure the services in the pods continue running.
As the Kubernetes documentation explains, a Kubernetes Job creates one or more pods and ensures that a specified number of the pods terminates when the task (Job) completes.
Just like in a typical operating system, the ability to perform automated, scheduled jobs without user interaction is important in the Kubernetes world. But Kubernetes Jobs do more than just run automated jobs, and there are multiple ways to utilize them through:
- Work queues (this is beyond the scope of this article)
Sounds simple right? Well, maybe. Anyone who works on containers and microservice applications knows that some require services to be transient so that they can do specific tasks for applications or within the Kubernetes clusters.
In this article, I will go into why Kubernetes Jobs are important, how to create Jobs and CronJobs, and when to use them for applications running on the Kubernetes cluster.
Differences between Kubernetes Jobs and CronJobs
More on Kubernetes
- What is Kubernetes?
- eBook: Storage Patterns for Kubernetes
- Test drive OpenShift hands-on
- eBook: Getting started with Kubernetes
- An introduction to enterprise Kubernetes
- How to explain Kubernetes in plain terms
- eBook: Running Kubernetes on your Raspberry Pi homelab
- Kubernetes cheat sheet
- eBook: A guide to Kubernetes for SREs and sysadmins
- Latest Kubernetes articles
Kubernetes Jobs are used to create transient pods that perform specific tasks they are assigned to. CronJobs do the same thing, but they run tasks based on a defined schedule.
Jobs play an important role in Kubernetes, especially for running batch processes or important ad-hoc operations. Jobs differ from other Kubernetes controllers in that they run tasks until completion, rather than managing the desired state such as in Deployments, ReplicaSets, and StatefulSets.
How to create Kubernetes Jobs and CronJobs
With that background in hand, you can start creating Jobs and CronJobs.
To do this exercise, you need to have the following:
- A working Kubernetes cluster; you can install it with either:
- The kubectl Kubernetes command line
Here is the Minikube deployment I used for this demonstration:
$ minikube version minikube version: v1.8.1 $ kubectl cluster-info Kubernetes master is running at https://172.17.0.59:8443 KubeDNS is running at https://172.17.0.59:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy $ kubectl get nodes NAME STATUS ROLES AGE VERSION minikube Ready master 88s v1.17.3
Just like anything else in the Kubernetes world, you can create Kubernetes Jobs with a definition file. Create a file called
sample-jobs.yaml using your favorite editor.
Here is a snippet of the file that you can use to create an example Kubernetes Job:
apiVersion: batch/v1 ## The version of the Kubernetes API kind: Job ## The type of object for jobs metadata: name: job-test spec: ## What state you desire for the object template: metadata: name: job-test spec: containers: - name: job image: busybox ## Image used command: ["echo", "job-test"] ## Command used to create logs for verification later restartPolicy: OnFailure ## Restart Policy in case container failed
Next, apply the Jobs in the cluster:
$ kubectl apply -f sample-jobs.yaml
Wait a few minutes for the pods to be created. You can view the pod creation's status:
$ kubectl get pod –watch
After a few seconds, you should see your pod created successfully:
$ kubectl get pods NAME READY STATUS RESTARTS AGE job-test 0/1 Completed 0 11s
Once the pods are created, verify the Job's logs:
$ kubectl logs job-test job-test
You have created your first Kubernetes Job, and you can explore details about it:
$ kubectl describe job job-test
Clean up the Jobs:
$ kubectl delete jobs job-test
You can use CronJobs for cluster tasks that need to be executed on a predefined schedule. As the documentation explains, they are useful for periodic and recurring tasks, like running backups, sending emails, or scheduling individual tasks for a specific time, such as when your cluster is likely to be idle.
As with Jobs, you can create CronJobs via a definition file. Following is a snippet of the CronJob file
cron-test.yaml. Use this file to create an example CronJob:
apiVersion: batch/v1beta1 ## The version of the Kubernetes API kind: CronJob ## The type of object for Cron jobs metadata: name: cron-test spec: schedule: "*/1 * * * *" ## Defined schedule using the *nix style cron syntax jobTemplate: spec: template: spec: containers: - name: cron-test image: busybox ## Image used args: - /bin/sh - -c - date; echo Hello this is Cron test restartPolicy: OnFailure ## Restart Policy in case container failed
Apply the CronJob to your cluster:
$ kubectl apply -f cron-test.yaml cronjob.batch/cron-test created
Verify that the CronJob was created with the schedule in the definition file:
$ kubectl get cronjob cron-test NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE cron-test */1 * * * * False 0 <none> 10s
After a few seconds, you can find the pods that the last scheduled job created and view the standard output of one of the pods:
$ kubectl logs cron-test-1604870760 Sun Nov 8 21:26:09 UTC 2020 Hello from the Kubernetes cluster
You have created a Kubernetes CronJob that creates an object once per execution based on the schedule
schedule: "*/1 * * * *". Sometimes the creation can be missed because of environmental issues in the cluster. Therefore, they need to be idempotent.
Other things to know
Unlike deployments and services in Kubernetes, you can't change the same Job configuration file and reapply it at once. When you make changes in the Job configuration file, you must delete the previous Job from the cluster before you apply it.
Generally, creating a Job creates a single pod and performs the given task, as in the example above. But by using completions and parallelism, you can initiate several pods, one after the other.
Use your Jobs
You can use Kubernetes Jobs and CronJobs to manage your containerized applications. Jobs are important in Kubernetes application deployment patterns where you need a communication mechanism along with interactions between pods and the platforms. This may include cases where an application needs a "controller" or a "watcher" to complete tasks or needs to be scheduled to run periodically.