Failed to create pod sandbox #3374
Description
Hello AWS Support Team,
I am currently experiencing an issue in my EKS cluster where only one node is failing to create pods due to the following error message:
Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "d44bf49e347ef9258e2b67c340bf506c4b3d80f8d3848d60a0a1735de4d7744c": plugin type="aws-cni" name="aws-cni" failed (add): add cmd: failed to assign an IP address to container
This problem has persisted for the last three days and is isolated to a single node with the hostname:
ip-10-20-0-5.eu-central-1.compute.internal
All other nodes in the cluster are functioning normally and are able to schedule pods without any issues.
I have checked the available IP addresses in the VPC subnets using the following command:
aws ec2 describe-subnets --filters "Name=vpc-id,Values=vpc-03759931b3e63f45c" | jq '.Subnets[] | .SubnetId + "=" + "\(.AvailableIpAddressCount)"' The output is:
"subnet-0d50e0a1241cc349c=245"
"subnet-030689ff9bd91a0a5=16"
"subnet-00dd0c66633b6c817=67"
"subnet-05cf1d1286395378a=248"
"subnet-0643962c19e8e3107=244"
"subnet-007eebd2ec697c7ec=127"
and ip-10-20-0-5.eu-central-1.compute.internal is c5.xlarge
As you can see, there are sufficient IP addresses available, so I believe IP exhaustion is unlikely the root cause.
My aws-vpc-cni plugin version is: v1.19.3-eksbuild.1.
Please let me know if you require any additional information or logs to help troubleshoot this issue.
Thank you for your assistance.