Containerization brings many advantages:
- It guarantees portability. This technology’s inherent isolation lets developers deploy their application code on various environments and operating systems without worrying about compatibility.
- It boosts scalability. Developers can deploy containers quickly and repeatedly. Unlike virtual machines, containers don’t require booting up an entire operating system, letting developers scale up or down quickly by adding or removing containers on single or multiple machines.
- It enhances fault tolerance. Because each container operates in isolation, one malfunctioning container doesn’t affect others on the same machine. Using the agility containerization offers, developers can quickly swap out faulty containers for working ones.
Kubernetes offers several notable benefits:
- Self-healing — Kubernetes actively monitors the health of each container within its cluster of servers (nodes). If a container fails or becomes unresponsive, Kubernetes can restart or terminate it based on the specified configuration.
- Automatic bin packing — Kubernetes efficiently allocates CPU and memory resources to each container based on configurations set by developers. It strategically places containers on servers (nodes) to maximize the utilization of underlying resources.
- Storage orchestration — Kubernetes supports persistent storage, integrating seamlessly with both on-premises and cloud storage solutions.
- Load balancing — Kubernetes actively observes the traffic directed toward each managed container. Kubernetes balances and distributes traffic across containers based on the set configuration, ensuring application stability.
- Automated rollouts and rollbacks — Managing container states in Kubernetes is straightforward. Kubernetes easily deploys new features or updates container images across hundreds or thousands of containers. Moreover, it provides the flexibility to roll back deployments using various deployment strategies.
- Secret and configuration management — Kubernetes securely handles secrets and sensitive information, including passwords, OAuth tokens, and SSH keys. When secrets update, there’s no need to rebuild container images. Kubernetes ensures this update happens discreetly without exposing secrets in its configuration.
How does Kubernetes Work?
At a high level, deploying Kubernetes means deploying a cluster. A Kubernetes cluster consists of at least one control plane and a set of servers, referred to as “worker nodes”. It’s on these worker nodes that containerized applications run, consuming most of the cluster’s computational resources. Every Kubernetes cluster should have at least one worker node. The control plane is also hosted on a server. However, its components differ from those of a worker node. In a production environment, it is best practice to host the control plane on multiple machines to provide fault tolerance and high availability.
Moreover, in Kubernetes, the smallest deployable unit is a Pod. A Pod contains one or multiple containers that are tightly coupled with each other. Pods are deployed in worker nodes, and it’s crucial to recognize that Pods should remain stateless. The Kubernetes API can terminate, update, or instantiate a Pod at any moment, and given that a Pod is ephemeral, it’s vital to ensure that critical data isn’t stored solely within it. Instead, any persistent data should be stored in external storage systems, ensuring that it remains intact even if a Pod is replaced or removed. This approach not only safeguards data but also ensures smooth scaling, updates, and recovery in Kubernetes environments.
The illustration below basically shows the architecture of a working Kubernetes cluster:
Components of the Control Plane
The control plane acts as the brain of the Kubernetes cluster. With its components, it makes crucial decisions and governs all activities within the cluster.
These are the fundamental components of a Kubernetes control plane:
- kube-apiserver — This component exposes the Kubernetes API and acts as the front-end of the Kubernetes cluster. Users can query and modify the state of API objects in Kubernetes (like Pods, Namespaces, ConfigMaps, and Events) through the Kubernetes API, typically accessed using the
kubectlcommand-line interface or other command-line tools. - etcd — This is a key-value store that holds all the Kubernetes cluster data. It serves as a repository, preserving snapshots of the cluster’s states and vital details.
- kube-scheduler — This component is responsible for deploying newly created Pods to worker nodes. It evaluates complex criteria to determine the placement of Pods. These criteria encompass individual and collective resource demands, hardware/software/policy constraints, affinity and anti-affinity rules, data proximity, and inter-workload interference.
- kube-controller-manager — This component operates as the control loop of the Kubernetes cluster, running its controller processes. It is the brain behind the orchestration; It constantly observes the cluster’s present state and interacts with the apiserver to achieve the targeted state.
- cloud-controller-manager — This optional component links cloud-specific control logic to the Kubernetes cluster. It facilitates communication between the cloud provider’s API and the Kubernetes cluster. If the cluster is running on-premises, the cluster does not have a cloud-controller-manager.
Components of a Worker Node
As stated above, worker nodes shoulder the majority of the cluster’s compute capacity. These nodes can be physical or virtual machines that communicate with the control plane. Pods are deployed in these worker nodes, which run the application code.
These are the core components of a Kubernetes worker node:
- kubelet — This is an agent installed in each and every worker node. It is the component that communicates with the control plane’s apiserver. It manages all the containers that are deployed in a node and ensures that they are encapsulated inside a Pod. Containers not deployed through Kubernetes are not managed by the kubelet.
- kube-proxy — This component ensures the communication between Pods. It is responsible for making network rules on nodes. These rules allow network communication to Pods from sessions inside or outside of the cluster.
Fundamental terminologies
Containers – a lightweight, standalone, and executable package that can hold a workload.
Workload – an application. This can be a single component or multiple components that work together. An example of a simple workload is a ping workload that runs every hour. An example of a complex workload is a web server that needs multiple containers working together.
Orchestration – In Kubernetes, this refers to the automatic creation, deletion, modification, and querying of containers.
Virtual Machine (VM) – An abstract software of a virtualized or emulated Operating System. This can be packaged inside a container or run as a standalone program.
Docker – A suite of software development tools for creating, sharing, and running containers.
Resources – Refers to the available CPU, Memory, Storage, Network, and anything else that a Workload would need to run.
Server – An abstract machine that hosts and runs software. A server can be hosted and run on multiple physical machines acting as one or a single machine.
What is Kubernetes?
Kubernetes is a platform for orchestrating, managing, and operating containerized workloads. This simply means Kubernetes can automatically create and manage containers.
Kubernetes is portable, extensible, and open-source.
Google created Kubernetes for their internal use. It was designed for the large scale as its primary use was running Google’s production workloads. Google open-sourced Kubernetes to the public in 2014.
Origin of Kubernetes name
Kubernetes came from the Greek word for Helmsman or Pilot. This can be seen in the logo.
Kubernetes is also abbreviated as K8s. This naming was used as there are 8 letters between “K” and “s”.
Deployment without Kubernetes
Without Kubernetes, native applications running on physical servers can use up a large amount of resources such as memory and disk space. There is no generic way of limiting an application’s access to these resources, as it would be running bare metal on the machine.
Without Kubernetes, a possible solution would be to run each application on different servers, but this is expensive in both hardware and maintenance.
Deployment with Virtualization
Without Kubernetes, a Virtual Machine can be used to isolate the applications from each other. Configuration of these VMs will allow for resources to be statically set during the creation of the VM.
However, VM is a complete machine running its own Operating System. The host machine will still be spending resources in running the VM, which will, in turn, spend those resources to run its OS and the application within the VM. As such, it will be inherently CPU intensive.
Deployment with Containers via Kubernetes
Kubernetes is the platform that manages the deployment of containers.
Internally, Kubernetes uses containers for handling the isolation between applications on one or multiple servers. Kubernetes is also responsible for configuring the resources allowed per container. This allows each container to have a quota of resources that they can use.
What are Containers?
Containers are similar to VMs. However, unlike a VM, the container will be running on the host’s OS. This removes the CPU needed for running an OS inside a VM, making it inherently faster than VMs.
Like a VM, a container will have its own resources and run applications isolated from other containers.
- Quick creation and deployment compared to VMs.
- The same environment can be consistent across development, testing, and production.
- Rollbacks for container image build and deployment.
- Separation of concerns between Developers and Operation of application.
- Allows for microservices architecture in software development.
- Decoupling of applications from infrastructure. This allows for OS and Cloud portability.
- OS portability. This means that containers can run on most modern OS like Ubuntu, RHEL, and others.
- Cloud portability. This means that different cloud providers can run containers.
- Resource isolation can be done between each running container.
- Observability. The containers can be queried by the end user or by an application via an API.
Kubernetes and Docker
Docker is actually multiple tools and products used to create, share and run a container. In relation to Kubernetes, Docker is primarily used for creating container images.
Starting in Kubernetes v1.20, Docker’s container runtime will not be used in favor of Kubernetes’ own container runtime interface. But even when not using Docker’s container runtime, Kubernetes can still use the container images created in Docker.
In practical terms, you use Docker to build container images. Kubernetes will take those images and run them without any other changes.
Benefits of Using Kubernetes
Kubernetes was built with the following in mind:
- Resilience. If a container goes offline, a new container with the same configuration will automatically deploy, ensuring little to no downtime.
- Scaling. If a service or container needs more resources, Kubernetes can provide more resources to the container (vertical scaling) or by creating more instances of the container to handle load balancing (horizontal scaling).
- Failover. If a container fails, Kubernetes have mechanisms to redirect traffic to a different and ready container.
In addition, Kubernetes has the following features that are useful for large-scale workloads:
- Service discovery. Kubernetes provides a container or end users to find containers that provide a service.
- Storage orchestration. Kubernetes allows for the configuration of storage with different storage providers.
- Automated rollouts. When deploying a new version of a container, Kubernetes will automatically handle the deployment to all the Clusters.
- Automated rollbacks. If after deploying a new version of a container and an error occurs, Kubernetes can rollback to the previous version.
- Automatic bin packing. Given the resources available to Kubernetes among different clusters, the platform can determine which cluster to use to be efficient in resource usage.
- Self-healing. When a Kubernetes component fails or has a discrepancy with the configuration, Kubernetes can repair itself to provide the expected configuration.
- Secret and configuration management. Kubernetes provides an API for keeping secrets (such as passwords and API keys) and configuring Kubernetes components.
What are Kubernetes Components?
There are two main components to Kubernetes: Cluster and Control Plane.
When deploying Kubernetes, at least one cluster will be created. A cluster contains the following components:
● Nodes.
○ Worker machines that runs containerized applications.
○ Every cluster will have at least one node.
○ Managed by the Control Plane.
● Pods.
○ Components of an application workload.
○ Managed by the Control Plane.
A cluster can be configured to run multiple nodes. To provide fault-tolerance and high availability, the Control Plane can be set up to run on multiple computers.
Control Plane Components
The Control Plane is responsible for the following:
● control the entire cluster.
● ensure that the cluster is running within the expected configuration.
● handle cluster events.
Each Control Plane component can run on the same or separate machines.
The following are Control Plane components:
● kube-apiserver
● etcd
● kube-scheduler
● kube-controller-manager
● cloud-controller-manager (optional)
kube-apiserver
- This is an API server that exposes Kubernetes API to the end user and workloads.
- This component can be scaled horizontally by deploying more instances (load balancing).
etcd
- This is a storage component that uses a Key-Value database built for Kubernetes. It stores the cluster’s data and can be backed up.
kube-scheduler
- This component watches for newly created Pods and selects which node to run them based on configurable criteria. These criteria includes:
● individual and collective resource requirements
● hardware/software/policy constraints
● affinity and anti-affinity specifications
● data locality
● inter-workload interference
● deadlines
kube-controller-manager
- This component handles the running controller processes. Examples of controllers managed by this component:
● Node controller. Monitors and responds to nodes going offline.
● Job controller. Watches for Jobs tasks. Also creates the Pods to run those tasks.
● EndpointSlice controller. Provide connectivity between Services and Pods.
● ServiceAccount controller. Create default Service Accounts for new namespaces.
cloud-controller-manager
- This optional component is used to embed cloud-specific control logic and to allow a cloud provider to control the cluster. To improve performance and help tolerate failures, this component can be scaled horizontally by running more than one copy.
- This component is only required if running Kubernetes on a compatible cloud service.
- These are the controllers that can have cloud dependencies:
● Node controller
● Route controller
● Service controller
Node Components
The Node components are responsible for providing Kubernetes runtime environment and maintaining the running pods in the cluster. Node components run on every node.
All nodes in a cluster will be running these components:
● kubelet
● kube-proxy
kubelet
- This component is responsible for ensuring that containers are running in a Pod. It uses PodSpecs for its configuration.
kube-proxy
- This component is part of Kubernetes Service and acts as a network proxy. It maintains network rules and allows for network communications between Pods inside and outside of the cluster.
- This component uses the host OS’ packet filtering layer if available. Otherwise, this component forwards the traffic.
Container runtime
This is the software responsible for running the containers.
containerd and CRI-O are examples of supported container runtime software. Other runtime software can be used if it supports Kubernetes CRI (Container Runtime Interface).
Kubernetes Add-Ons
Kubernetes is extensible with add-ons. Here are examples of available add-ons:
● Service Discovery.
● Visualization and Control
○ Dashboard (Web UI)
○ Weave Scope
● Container Resource Monitoring
● Cluster-level Logging
Infrastructure
○ KubeVirt
○ Node problem detector
● Networking and Network Policy
○ ACI
○ Antrea
○ Calico
○ Canal
○ Cilium
○ CNI-Genie
○ Contiv
○ Contrail
○ Flannel
○ Knitter
○ Multus
○ OVN-Kubernetes
○ Nodus
○ NSX-T
○ Nuage
○ Romana
○ Weave Net
Kubernetes Objects represent the desired state of the cluster and are created by configuration. These objects can be created, modified, or deleted via the Kubernetes API.
Kubernetes Objects describes the following:
● what containerized applications are running and on which nodes.
● resources available to these applications.
● the policies on how these applications behave such as restart policies, upgrades, and
fault-tolerance.
Objects can be selected using any of the following:
● Label selectors
● Namespaces
● Field Selectors
Object Names and IDs
- Each object in the cluster has a unique Name for that resource.
- Each object has a UID that is unique across the whole cluster.
Names
- Names are client-provided string that refer to an object. It follows a resource URL format such as: /api/v1/pods/unnamed.
- For each kind of object, one name can be assigned to that object. If an object is deleted, the name can be assigned to the new object.
- Names must be unique across all API versions of the same resource.
- These are the commonly used name constraints:
1. DNS Subdomain Names
● contains no more than 253 characters
● contains only lower case alphanumeric characters, -, or .
● starts and ends with an alphanumeric character
2. RFC 1123 Label Names
● contains at most 63 characters
● contain only lower case alphanumeric characters, or –
● starts and ends with an alphanumeric character
3. RFC 1035 Label Names
● contains at most 63 characters
● contain only lower case alphanumeric characters, or –
● starts with an alphabetic character
● ends with an alphanumeric character
Path Segment Names
- Some resource types require their names to be encoded as a path segment. The name may not be “.” or “..” and the name may not contain “/” or “%”.
Kubernetes Object Management
There are three techniques in managing Kubernetes Objects:
1. Imperative commands
2. Imperative object configuration
3. Declarative object configuration
Imperative commands
- Imperative commands are used to operate directly on live objects in a cluster. Commands can be issued via kubectl or any equivalent Kubernetes client.
- Advantages
● Commands can be issued via one action word (for example: create, deploy).
● Cluster changes can be done in a single step.
● Simplest to use among the three
- Disadvantages
● does not easily integrate with a review process
● does not easily allow for an audit trail
● does not provide a record of the changes except for its effect
● does not allow for an organization to have a common template. - Example of Imperative command:
$ kubectl create deployment nginx –image nginx
Imperative object configuration
- Imperative object configuration uses a YAML or JSON file. The configurations are described in he file.
- Advantages
● Configuration can be stored in a source control system.
● Can be integrated with a review process.
● Allows for an audit trail.
● Allows for an organization to have a common template
- Disadvantages
● Requires basic understanding of the object schema.
● Requires writing of configuration file.
● Works best on files, not directories.
● Needs to be sync with any Imperative commands. - Example of Imperative object configuration:
| apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment spec: selector: matchLabels: app: nginx replicas: 2 template: metadata: labels: app: nginx spec: containers: – name: nginx image: nginx:1.14.2 ports: – containerPort: 80 |
- The above .yaml file can saved into disk and applied with the following command: $ kubectl apply -f [yaml file]
Declarative object configuration
- Declarative object configuration refers to a collection of files that describes the object. The configuration files do not define the operations to be taken. Instead, the create, update, and delete operations are automatically detected per-object by kubectl.
- Advantages
● Can be used with directories, where different operations might be needed for different
objects.
● Changes made directly to live objects are retained. Optionally, these changes can be
merged back into the file.
- Disadvantages
● The most complicated to use
● Harder to debug
● There is a potential for complicated merge and patch operations. - Command to process declarative object configuration: $ kubectl apply -f configs/
Labels and Selectors
- Labels can be attached to objects. A single label is a key/value pair. Multiple labels can be defined per object. Each key in an object must be unique.
- Labels are used to identify attributes of objects that are meaningful and relevant to users. Unlike names and UIDs, labels are not unique. Labels can be added at creation time or during run time.
- Labels allow for efficient queries and watches. It also allow for the organization and selection of subsets of objects.
- The following are rules for the naming of label keys [3]:
● must be 63 characters or less (can be empty),
● unless empty, must begin and end with an alphanumeric character ([a-z0-9A-Z]),
● could contain dashes (-), underscores (_), dots (.), and alphanumerics between.
● “kubernetes.io/” and “k8s.io/” prefixes are reserved for Kubernetes core components.
Label selector
- Label selectors are used to identify a set of objects. There are two types of selectors:
● equality-based.
● set-based. - Multiple label selectors can be used using the comma separator, which acts as a logical AND operator.
- Equality-base selector uses =, ==, and !=. Examples:
● environment = production
● tier != frontend
● environment=production,tier!=frontend - Set-based selector uses in, notin, and exists. Examples:
● environment in (production, qa)
● tier notin (frontend, backend)
● partition
● !partition
● partition,environment notin (qa) - Both types of selectors can be combined. For example:
● partition in (customerA, customerB),environment!=qa
Namespaces
- Namespaces are used to isolate groups of resources within a single cluster. Namespaces can also be used to divide cluster resources between multiple users (via a resource quota).
- A resource’s name can be unique inside a namespace, allowing for the same name to be used in other namespaces.
- Some cluster-wide objects are not applicable for namespacing (such as StorageClass, Nodes, PersistentVolumes, etc).
- Not all resources can be placed in a namespace. Examples of these are Nodes and persistentVolumes.
- Examples of objects that can be namespaced are Deployments and Services.
- A Kubernetes cluster will have four initial namespaces available:
● default – Allows for the creation of a new cluster without first creating a namespace.
● kube-node-lease – Used to hold Lease objects for each node. Node leases allows the
kubelet component to send heartbeats to the control plane to detect node failures.
● kube-public – Namespace that is readable by all clients, including unauthenticated
guests.
● kube-system – Created by the Kubernetes system for its objects - Example of accessing an object with a namespace
- $ kubectl get namespace
- $ kubectl get pods –namespace=[namespace]
- $ kubectl config set-context –current –namespace=[namespace]
- Note that DNS entries for Service components uses the form “<service-name>.<namespace>.svc.cluster.local”. If a container only contains “<service-name>”, it will resolve to the local namespace.
- Annotations
- Annotations are used to attach arbitrary custom metadata to objects. Like Labels, Annotations are key/value maps.
- Usually used by tools and libraries for their custom operations.
- Example:
- Field Selectors
- Field Selectors are used to select Kubernetes objects based on the values of one or more resource fields.
- Example:
- ● metadata.name=my-service
- ● metadata.namespace!=default
- ● status.phase=Pending
- Example of commands using field selectors:
$ kubectl get pods –field-selector status.phase=Running
$ kubectl get services –all-namespaces –field-selector metadata.namespace!=default
$ kubectl get pods –field-selector=status.phase!=Running,spec.restartPolicy=Always
$ kubectl get statefulsets,services –all-namespaces –field-selector metadata.namespace!=default - Owners and Dependents
- Objects can have an owner. These owner references describe the relationship between objects in Kubernetes. For example, a ReplicaSet is the owner of a set of Pods.
- Objects with an owner are dependent on the state of their Owner.
- Owner references allow Kubernetes components to avoid interfering with objects they don’t control.
- When a resource is deleted, the API server signals the managing controller to process any finalizer rules. This can prevent deletion of critical resources such as PersistentVolume still in use by a Pod.
- Finalizers
- Finalizers are configurable rules that tells Kubernetes to wait until specific conditions are met before fully deleting resources that are marked for deletion.
- Finalizers can be used to control garbage collection
- When an object is deleted, it is not deleted immediately, but instead, is marked for deletion. This is to allow for Owners to decide on a deletion action before it is executed or canceled.
What is a Pod?
A Pod is a Kubernetes Object that has one or more containers running inside it. A Pod will ensure that the containers inside it share the same resources and local network. Each Pod gets its own IP address.
If you use a Deployment to run your app, that Deployment can create and destroy Pods dynamically. Pods are created and destroyed to match the desired state of your cluster.
Pods are ephemeral resources, meaning that they can be deleted and recreated as needed by Kubernetes. As such, you shouldn’t expect an individual Pod to be reliable and durable. This is by design.
What are Kubernetes Services?
In Kubernetes, a Service is a method for exposing a network application that is running as one or more Pods in your Cluster.
Since Pods are ephemeral, we need to have a mechanism for Pods inside the Cluster to find other Pods. This is when we use Kubernetes Services.
The Service API is an abstraction to help you expose a group of Pods over a network. Each Service object defines a logical set of endpoints along with a policy to make these Pods available.
The Pods that a Service will use can be retrieved with a selector.
Kubernetes Service Types
In Kubernetes, you might need to expose a service via an external IP address. This is usually done for frontend and API endpoints.
These are the four types of Services:
- ClusterIP
- NodePort
- LoadBalancer
- ExternalName
ClusterIP
By default, this is the Service Type used by Kubernetes. ClusterIP will allow the Service to be reachable from within the Cluster. To expose this Service to outside the Cluster (for example, by allowing the public Internet to access the Service), an Ingress or Gateway is used.
ClusterIP works by assigning an IP address from the Cluster’s reserved pool of IP addresses.
NodePort
This exposes the Service on each Node’s IP at a static port.
NodePort works by having the Kubernetes Control Plane allocate a port from a predefined range (default: 30000 – 32767). The Node in which the Service resides in will proxy a port from that range, allowing access to the Service.
LoadBalancer
This exposes the Service using an external load balancer. Kubernetes doesn’t come with a built-in load balancer but can integrate with most load balancers (like Nginx) and any supported Cloud provider.
ExternalName
This maps the Service to the value of the “externalName” field. The mapping uses the cluster’s DNS server to return a CNAME record with the value.
Defining a Service
ClusterIP
By default, ClusterIP is used for a Service. Other Service Types are based on this configuration. Here is an example of a Service manifest:
| apiVersion: v1kind: Servicemetadata: name: my-servicespec: selector: app.kubernetes.io/name: MyApp ports: – protocol: TCP port: 80 targetPort: 9376 |
After saving this yaml to disk, you can apply the yaml with the following command:
$ kubectl apply -f [filename.yaml]
This manifest creates a Service named “my-service”. By default, this is a ClusterIP service type. The Service targets TCP port 9376 on any pod with the app.kubernetes.io/name: MyApp label.
After creation, Kubernetes will assign the cluster’s IP address to this Service via Kubernetes virtual IPs and Service Proxies.
You can define ports with names instead of hardcoding them in the Service. Here is an example:
| apiVersion: v1kind: Podmetadata: name: nginx labels:app.kubernetes.io/name: proxyspec: containers:– name: nginx image: nginx:stable ports: – containerPort: 80 name: http-web-svc |
| —apiVersion: v1kind: Servicemetadata: name: nginx-servicespec: selector: app.kubernetes.io/name: proxy ports: – name: name-of-service-port protocol: TCP port: 80 targetPort: http-web-svc |
You can also define multiple ports for a Service. Here is an example:
| apiVersion: v1kind: Servicemetadata: name: my-servicespec: selector: app.kubernetes.io/name: MyApp ports:– name: httpprotocol: TCPport: 80targetPort: 9376– name: httpsprotocol: TCPport: 443targetPort: 9377 |
NodePort
This is an example of a NodePort:
| apiVersion: v1kind: Servicemetadata: name: my-servicespec: type: NodePort selector: app.kubernetes.io/name: MyAppports: – protocol: TCP port: 80 targetPort: 9376 # optional. without this line, Kubernetes will use the default port range nodePort: 30007 |
Note that it is the same as a ClusterIP except we explicitly declare .spec.type as NodePort and added an optional .spec.ports.nodePort value.
LoadBalancer
This is an example of a LoadBalancer:
| apiVersion: v1kind: Servicemetadata: name: my-servicespec: selector: app.kubernetes.io/name: MyApp ports:– protocol: TCPport: 80targetPort: 9376 clusterIP: 10.0.171.201type: LoadBalancerstatus: loadBalancer:ingress:– ip: 192.0.2.127 |
Traffic from the load balancer will be directed to the Pod with the app.kubernetes.io/name: MyApp label.
If a cloud-controller-manager component is used in the Cluster, it will configure the external load balancer to forward traffic to the assigned node port. This can be omitted if the Cloud provider supports this.
ExternalName
This is an example of a ExternalName Service:
| apiVersion: v1kind: Servicemetadata: name: my-service namespace: prod spec: type: ExternalName externalName: my.database.example.com |
When a request for my-service.prod.svc.cluster.local is sent to the cluster DNS Service, this configuration will returns a CNAME record with the externalName value of “my.database.example.com”.
Headless Service
If you plan to use a different or customized service discovery mechanism other than Kubernetes’ default implementation, you can run a Headless Service.
A Headless Service doesn’t use load-balancing or a single Service IP. A cluster IP is not allocated, kube-proxy will not handle these services, and Kubernetes will not provide a proxy.
A Headless Service is created when the value for .spec.clusterIP is set to “None”.
The redirection happens at the DNS level instead of forwarding or proxying.
A workload is an application that can have one or more components running on Kubernetes. A Pod represents a set of running containers in the cluster.
Kubernetes allows for declarative configuration of workloads and its components. This will allow the Control Plane to manage the creation, deletion, and modification of these Pods.
The built-in APIs for managing workload resources are:
- Deployments – The most commonly used workload to run an application in a Kubernetes Cluster. Deployments are used for managing workloads that are stateless, where any Pod in the configuration can be replaced if needed. Under the hood, Deployments creates and manages ReplicaSet with a declarative configuration based on the deployment configuration.
- ReplicaSet – A ReplicaSet contains pods that are identical to each Its purpose is to ensure that a specific number of Pods are always available at a given time. Deployments are recommended over ReplicaSet.
- StatefulSets – This is used for managing workloads that are stateful. This workload configuration manages the deployment and scaling of a set of unique and persistent Pods. The uniqueness is done by providing each Pod with a sticky identity.
- DaemonSet – This defines Pods that provides functionality that will ensure that all Nodes or a subset of Nodes in the Cluster will run a copy of a Pod. This concept follows how a system daemon is traditionally used in a Unix server. For example, a driver that allows access to a storage system can be implemented as a DaemonSet.
- Jobs – These workloads are designed for one-off tasks. Jobs will continue to retry execution until a specified number of them successfully terminate. When this number is reached, the Job will be marked The simplest Job uses a single Pod. Jobs can run multiple Pods in parallel.
- CronJobs – These workloads are the same as Jobs but are expected to run to completion and then stop until the next scheduled run.
Defining a Kubernetes Workload Resource
Deployments
Example of a Deployment:
| apiVersion: apps/v1kind: Deploymentmetadata:name: nginx-deploymentlabels: app: nginxspec:replicas: 3selector:matchLabels:app: nginxtemplate: metadata:labels: app: nginx spec: containers:– name: nginximage: nginx:1.14.2ports:– containerPort: 80 |
Deployment is commonly abbreviated to deploy, including in kubectl. For the following examples, you can use deploy instead of deployment.
(If you want to test using the above example, use nginx-deployment for [deployment name]) Command to check Deployment:
$ kubectl get deployments
$ kubectl get deploy
Command to get details of Deployment:
$ kubectl describe deployments
$ kubectl describe deployment [deployment name]
Command to see Deployment rollout status:
$ kubectl rollout status deployment/[deployment name]
Command to check history and revisions to the Deployment:
$ kubectl rollout history deployment/[deployment name]
To see the details of each revision:
$ kubectl rollout history deployment/[deployment name] –revision=[index]
$ kubectl rollout history deployment/nginx-deployment –revision=2
To undo the current rollout and rollback to the previous revision:
$ kubectl rollout undo deployment/[deployment name]
To undo the current rollout and rollback to a specific revision:
$ kubectl rollout undo deployment/[deployment name] –to-revision=[index]
$ kubectl rollout undo deployment/nginx-deployment –to-revision=2
To pause the current rollout:
$ kubectl rollout pause deployment/[deployment name]
To resume the paused rollout:
$ kubectl rollout resume deployment/[deployment name]
Command to scale a deployment:
$ kubectl scale deployment/[deployment name] –replicas=[value]
Command to autoscale if horizontal Pod autoscaling is enabled in cluster:
$ kubectl autoscale deployment/[deployment name] –min=[value] –max=[value]
–cpu-percent=[value]
$ kubectl autoscale deployment/nginx-deployment –min=10 –max=15 –cpu-percent=80
Command to update the Deployment:
$ kubectl set image deployment.v1.apps/nginx-deployment [image name]=[new value] or
$ kubectl set image deployment/nginx-deployment [image name]=[new value]
$ kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.16.1 or
$ kubectl set image deployment/nginx-deployment nginx=nginx:1.16.1
Command to edit the Deployment:
$ kubectl edit deployment/nginx-deployment
ReplicaSet
Example of a Replicaset:
| apiVersion: apps/v1kind: ReplicaSetmetadata:name: frontendlabels: app: gremlin tier: frontendspec:replicas: 10selector: matchLabels: tier: frontendtemplate: metadata:labels:tier: frontendspec: containers:– name: php-redisimage: gcr.io/google_samples/gb-frontend:v3 |
Replicaset is commonly abbreviated to rs, including in kubectl. For the following examples, you can use rs instead of replicaset.
Command to get current ReplicaSets deployed:
$ kubectl get replicaset
$ kubectl get rs
Command to check the state of the ReplicaSet:
$ kubectl describe replicaset/[ReplicaSet name]
$ kubectl describe replicaset/frontend
Command to autoscale the ReplicaSet:
$ kubectl autoscale replicaset [ReplicaSet name] –max=[value] –min=[value]
–cpu-percent=[value]
$ kubectl autoscale replicaset frontend –max=10 –min=3 –cpu-percent=50
StatefulSets
Example YAML configuration of StatefulSets:
| apiVersion: v1kind: Servicemetadata:name: nginxlabels:app: nginxspec: ports: – port: 80 name: web clusterIP: None selector:app: nginx—apiVersion: apps/v1kind: StatefulSetmetadata: name: webspec: selector:matchLabels:app: nginx # has to match .spec.template.metadata.labels serviceName: “nginx” replicas: 3 # by default is minReadySeconds: 10 # by default is 0 template:metadata:labels: app: nginx # has to match .spec.selector.matchLabels spec: terminationGracePeriodSeconds: 10 containers:– name: nginximage: registry.k8s.io/nginx-slim:0.8 ports:-containerPort: 80– name: webvolumeMounts:– name: wwwmountPath: /usr/share/nginx/htmlvolumeClaimTemplates: – metadata: name:wwwspec:accessModes: [ “ReadWriteOnce” ]storageClass.Name: “my-storage-class”resources: requests:storage: 1Gi |
StatefulSet is commonly abbreviated to sts, including in kubectl.
Command to get current StatefulSet deployed:
$ kubectl get statefulset
$ kubectl get sts
Command to check the state of the StatefulSet:
$ kubectl describe statefulset/[StatefulSet name]
$ kubectl describe statefulset/web
Command to scale the StatefulSet:
$ kubectl scale statefulset [StatefulSet name] –replicas=[value]
$ kubectl scale statefulset web –replicas=5
DaemonSet*
Example of DaemonSet:
| apiVersion: apps/v1kind: DaemonSetmetadata: name: fluentd-elasticsearch namespace: kube-system labels: k8s-app: fluentd-loggingspec:selector: matchLabels:name: fluentd-elasticsearchtemplate: metadata:labels:name: fluentd-elasticsearchspec:tolerations:# these tolerations are to have the daemonset runnable on control plane nodes# remove them if your control plane nodes should not run pods-key: node-role.kubernetes.io/control-planeoperator: Existseffect: NoSchedule– key: node-role.kubernetes.io/masteroperator: Existseffect: NoSchedulecontainers:– name: fluentd-elasticsearchimage: quay.io/fluentd_elasticsearch/fluentd:v2.5.2resources: limits: memory: 200Mirequests: cpu: 100mmemory: 200MivolumeMounts:– name: varlogmountPath: /var/logterminationGracePeriodSeconds: 30volumes:– name: varloghostPath:path: /var/log |
DaemonSet is commonly abbreviated to ds, including in kubectl.
Command to get current DaemonSet deployed:
$ kubectl get daemonset
$ kubectl get ds
Command to check the state of the DaemonSet:
$ kubectl describe daemonset/[StatefulSet name]
Jobs
Example of Job:
| apiVersion: batch/v1kind: Jobmetadata: name: pispec:template: spec: containers: – name: pi image: perl:5.34.0 command: [“perl”, “-Mbignum=bpi”, “-wle”, “print bpi(2000)”]restartPolicy: NeverbackoffLimit: 4 |
Command to check Jobs:
$ kubectl get job
Command to check specific Job:
$ kubectl get job [Job name]
$ kubectl get job pi
Command to get details of specific Jobs:
$ kubectl describe job [Job name]
Command to view the logs of a Job:
$ kubectl logs jobs/[Job name]
Command to suspend an active job:
$ kubectl patch job/[Job name] –type=strategic –patch ‘{“spec”:{“suspend”:true}}’
Command to resume an active job:
$ kubectl patch job/[Job name] –type=strategic –patch ‘{“spec”:{“suspend”:false}}’
CronJob
Example of a CronJob:
| apiVersion: batch/v1kind: CronJobmetadata: name: hellospec: schedule: “* * * * *” jobTemplate:spec: template: spec: containers:– name: helloimage: busybox:1.28imagePullPolicy: IfNotPresentcommand:– /bin/sh– c-date; echo Hello from the Kubernetes cluster– restartPolicy: OnFailure |
The .spec.schedule field uses the Cron syntax:
# +————- minute (0 – 59)
# ¦ +————- hour (0 – 23)
# ¦ ¦ +————- day of the month (1 – 31)
# ¦ ¦ ¦ +————- month (1 – 12)
# ¦ ¦ ¦ ¦ +————- day of the week (0 – 6) (Sunday to Saturday;
# ¦ ¦ ¦ ¦ ¦ 7 is also Sunday on some systems)
# ¦ ¦ ¦ ¦ ¦ OR sun, mon, tue, wed, thu, fri, sat # ¦ ¦ ¦ ¦ ¦
# * * * * *
For example, 0 0 13 * 5 states that the task must be started every Friday at midnight, as well as on the 13th of each month at midnight.
Other than the standard syntax, some macros like @monthly can also be used:
| Entry | Description | Equivalent to |
| @yearly (or @annually) | Run once a year at midnight of 1 January | 0 0 1 1 * |
| @monthly | Run once a month at midnight of the first day of the month | 0 0 1 * * |
| @weekly | Run once a week at midnight on Sunday morning | 0 0 * * 0 |
| @daily (or @midnight) | Run once a day at midnight | 0 0 * * * |
| @hourly | Run once an hour at the beginning of the hour | 0 * * * * |
Automatic Cleanup for Finished Jobs
After a Job is finished, Kubernetes will not immediately delete the Job workload. Instead, a TTL-after-finished (time to live) controller will activate. This allows the client via the Kubernetes API to check within the TTL if a Job has succeeded or not.
This is done by modifying the .spec.ttlSecondsAfterFinished field of a Job.
After a Job is finished with either Complete or Failed status condition, the timer will trigger. When the TTL-after-finished seconds are reached, the Job will become eligible for cascading removal. The Job will still follow an object lifecycle, including waiting for finalizers.
The TTL-after-finished controller is only supported for Jobs.
When the .spec.ttlSecondsAfterFinished field is modified after the timer has expired, Kubernetes will not guarantee that the Job will extend. This is true even if TTL returns a successful API response.
This feature is stable starting from Kubernetes v1.23.
Example yaml of using TTL-after-finished
| apiVersion: batch/v1kind: Jobmetadata: name: pi-with-ttlspec:ttlSecondsAfterFinished: 100template: spec: containers:– name: piimage: perl:5.34.0command: [“perl”, “-Mbignum=bpi”, “-wle”, “print bpi(2000)”]restartPolicy: Never |
ReplicationController
The ReplicationController ensures that a user-specified number of pod replicas are always up. This ensures that those pods are always running and available.
The ReplicationController will check the number of running pods it is maintaining. If there are too few, it will start more pods. If there are too many, it will delete existing pods. This is done automatically if a pod fails, deleted, terminated, or created.
A ReplicationController is required even if the workload requires a single pod. The ReplicationController supervises multiple pods across multiple nodes.
The ReplicationController is designed to facilitate rolling updates by replacing pods one by one. Example of ReplicationController to run 5 copies of the nginx web server:
| apiVersion: v1kind: ReplicationControllermetadata: name: nginxspec: replicas: 5selector:app: nginxtemplate:metadata:name: nginx labels:app: nginxspec:containers:– name: nginx image: nginx ports:– containerPort: 80 |
ReplicationController is commonly abbreviated to rc, including in kubectl.
Kubectl command to check the status of ReplicationController
$ kubectl describe replicationcontrollers/nginx
$ kubectl describe rc/nginx