Resolving Pending Zookeeper Pods Issue in Strimzi's Kafka Deployment on AWS EKS

Описание: Learn how to address the `Pending` state of Zookeeper pods when deploying Strimzi's Kafka on AWS EKS using the `kafka-persistent-single.yaml` configuration. Follow our troubleshooting guide to ensure successful deployment!
---
This video is based on the question https://stackoverflow.com/q/75487295/ asked by the user 'Mahendran' ( https://stackoverflow.com/u/1761065/ ) and on the answer https://stackoverflow.com/a/75487538/ provided by the user 'Jakub' ( https://stackoverflow.com/u/8607826/ ) at 'Stack Overflow' website. Thanks to these great users and Stackexchange community for their contributions.

Visit these links for original content and any more details, such as alternate solutions, latest updates/developments on topic, comments, revision history etc. For example, the original title of the Question was: Zookeeper Pods are in pending state when strimzi's sample kafka-persistent-single.yaml is deployed in AWS EKS

Also, Content (except music) licensed under CC BY-SA https://meta.stackexchange.com/help/l...
The original Question post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license, and the original Answer post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license.

If anything seems off to you, please feel free to write me at vlogize [AT] gmail [DOT] com.
---
Troubleshooting Zookeeper Pods in Pending State on AWS EKS

Deploying applications on Kubernetes can occasionally lead to unexpected hurdles, especially when utilizing services like Strimzi to manage Kafka clusters. A common issue that users may face is encountering Zookeeper pods stuck in a pending state during deployment, particularly when using the kafka-persistent-single.yaml configuration in AWS EKS. In this blog, we will explore the underlying causes of the problem and provide effective solutions to get your Zookeeper pods running smoothly.

Understanding the Problem

What’s Happening?

When deploying the kafka-persistent-single.yaml, users have reported that the Zookeeper pods remain in a pending state. Looking at the Kubernetes events can provide some insights into potential issues:

Pending Pods: The events log shows messages like:

running PreBind plugin "VolumeBinding": binding volumes: provisioning failed for PVC "data-my-cluster-zookeeper-0"

waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator

These errors indicate that the Persistent Volume Claims (PVC) required for Zookeeper's operation are not being satisfied.

Analyzing the Cause

Key Factors to Consider

Persistent Storage Requirement: The configuration file kafka-persistent-single.yaml requires persistent storage, which means a Persistent Volume (PV) needs to be created and bound to the PVC. If this step fails, your pods will remain in the pending state.

Storage Class and Provisioner: The deployment relies on the AWS EBS (Elastic Block Store) provisioner (ebs.csi.aws.com). If there’s a misconfiguration in the storage class or if the provisioner encounters issues, the storage simply won’t be available.

AWS Account Limits: Sometimes, the problems may stem from limits within your AWS account. Check if you have reached your limits for EBS volumes or other related resources.

Why Does kafka-ephemeral-single.yaml Work?

In contrast, the kafka-ephemeral-single.yaml file uses emptyDir as its storage mechanism. This does not require persistent disks and can run without provisioning, but it is important to note that this option is not ideal for production environments due to its ephemeral nature.

Steps to Resolve the Issue

Check Your Storage Class

Review the Storage Class: Make sure that the storage class named gp2 is correctly configured and available in your cluster. You can verify this by running:

[[See Video to Reveal this Text or Code Snippet]]

Provisioner Status: Check the status of the EBS CSI provisioner:

[[See Video to Reveal this Text or Code Snippet]]

Look for any errors indicating that the provisioner has failed to create PVs.

Adjust Persistent Volume Claims

Examine PVCs: Use the command below to check the status of your PVCs and see if any are stuck in a pending state:

[[See Video to Reveal this Text or Code Snippet]]

Manual Creation: If necessary, you can manually create a Persistent Volume that matches your PVC requirements if the dynamic provisioning is not working as expected.

Monitor AWS Resource Limits

Check Limits: Log into your AWS management console and inspect the EBS and EC2 limits. If you've reached any limits, you can request an increase.

Resource Cleanup: Ensure that unused EBS volumes or resources are deleted to make room for new ones.

Conclusion

Successfully deploying a Kafka cluster using Strimzi on AWS EKS requires ensuring that all components work together seamlessly. When your Zookeeper pods are in a pending state, start by checking your Persistent Volume Claims, sto