[e2e test flake] unexpected error trying to use websockets for pod exec

[alimaazamat]

Which jobs are flaking?

capi-e2e-mink8s-release-1-10

Which tests are flaking?

Different tests fail but commonly is periodic-cluster-api-e2e-mink8s-release-1-10.Overall
Here is the repeated example error in build log:

I0804 05:14:56.605011 15 exec_util.go:201] unexpected error trying to use websockets for pod exec: <nil>
  I0804 05:14:56.605096 15 utils.go:355] Waiting for responses: map[netserver-0:{}]
  I0804 05:14:58.609058 15 exec_util.go:63] ExecWithOptions {Command:[/bin/sh -c curl -g -q -s 'http://192.168.2.4:9080/dial?request=hostname&protocol=http&host=172.18.0.11&port=30340&tries=1'] Namespace:dualstack-1871 PodName:test-container-pod ContainerName:webserver Stdin: CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
  I0804 05:14:58.609138 15 exec_util.go:68] ExecWithOptions: Clientset creation
  I0804 05:14:58.609228 15 exec_util.go:84] ExecWithOptions: execute(https://172.18.0.5:6443/api/v1/namespaces/dualstack-1871/pods/test-container-pod/exec?command=%2Fbin%2Fsh&command=-c&command=curl+-g+-q+-s+%27http%3A%2F%2F192.168.2.4%3A9080%2Fdial%3Frequest%3Dhostname%26protocol%3Dhttp%26host%3D172.18.0.11%26port%3D30340%26tries%3D1%27&container=webserver&stderr=true&stdout=true)
  I0804 05:14:58.738590 15 exec_util.go:201] unexpected error trying to use websockets for pod exec: <nil>
  I0804 05:14:58.738702 15 utils.go:355] Waiting for responses: map[]
  I0804 05:14:58.738737 15 utils.go:359] reached 172.18.0.11 after 1/34 tries
  I0804 05:14:58.738884 15 helper.go:125] Waiting up to 7m0s for all (but 0) nodes to be ready
  STEP: Destroying namespace "dualstack-1871" for this suite. @ 08/04/25 05:14:58.742
• [26.775 seconds]

Since when has it been flaking?

08/02/2025 and before that during the IPv6 dualstack conformance test failures

Testgrid link

https://prow.k8s.io/view/gs/kubernetes-ci-logs/logs/periodic-cluster-api-e2e-mink8s-release-1-10/1952228381764358144
https://prow.k8s.io/view/gs/kubernetes-ci-logs/logs/periodic-cluster-api-e2e-mink8s-release-1-10/1952046934256521216
https://prow.k8s.io/view/gs/kubernetes-ci-logs/logs/periodic-cluster-api-e2e-mink8s-release-1-10/1951200849795485696
https://prow.k8s.io/view/gs/kubernetes-ci-logs/logs/periodic-cluster-api-e2e-mink8s-release-1-10/1951684290777452544

Reason for failure (if possible)

No response

Anything else we need to know?

No response

Label(s) to be applied

/kind flake
One or more /area label. See https://github.com/kubernetes-sigs/cluster-api/labels?q=area for the list of labels.

[k8s-ci-robot]

This issue is currently awaiting triage.

If CAPI contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[sbueringer]

3 of the 4 linked jobs seem to show a different error. The 4th is the dual stack error we just started to skip (xref #12496)

Can you please point me to a job that fails because of the quoted log?

[alimaazamat]

3 of the 4 linked jobs seem to show a different error. The 4th is the dual stack error we just started to skip (xref #12496)

Can you please point me to a job that fails because of the quoted log?

All of the jobs fail on different errors but I just noticed a commonality where the build logs all show "unexpected error trying to use websockets for pod exec: ". In the failures some of them I've noticed the flakes were known issues but don't always have the "unexpected error trying to use websockets for pod exec: " in the build log.
For example for the first linked testL https://prow.k8s.io/view/gs/kubernetes-ci-logs/logs/periodic-cluster-api-e2e-mink8s-release-1-10/1952228381764358144 which is the changed ResourceVersion issue #12334 not all of the test flakes with that failure have the "unexpected error trying to use websockets for pod exec: " in the build log.
It just raised an internal alarm in me that this same error in the build logs shows up between different job failures.

Thought to be safe to report this as a seperate issue, however, if this feels like it is reporting a symptom rather than the actual root issue then I can just mentally note this for CI Release Team and close this issue. Let me know your thoughts!

[sbueringer]

Okay, thx. Got it. If I see correctly these logs are part of the output of the dual stack tests. Could it be that this log always occurs? (i.e. also if the entire job is successful)

[alimaazamat]

Yeah I see that too: https://prow.k8s.io/view/gs/kubernetes-ci-logs/logs/periodic-cluster-api-e2e-mink8s-release-1-10/1952711822549192704
Looking into k/k code here https://github.com/kubernetes/kubernetes/blob/master/test/e2e/framework/pod/exec_util.go#L201
I think the issue is that it is logging even when err is nil so seems to be a simple logging bug. Will create issue in k/k

[sbueringer]

Nice catch! Thank you

[alimaazamat]

kubernetes/kubernetes#133387

[sbueringer]

Should we close or retitle the issue here? (not sure if it then still makes sense as one issue or if it should be multiple)

[alimaazamat]

I think we can close the issue, everything will just be correctly documented on kubernetes/kubernetes#133387