Sidecar Containers

[Joseph-Irving]

Enhancement Description

• One-line enhancement description (can be used as a release note): Kubernetes introduces the built-in support for sidecar containers pattern.
• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/753-sidecar-containers/README.md
• Discussion Link: Sidecar working group and KEP draft
• Primary contact (assignee): @SergeyKanzhelev
• Responsible SIGs: node
• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): 1.28
  • Beta release target (x.y): 1.29
  • Stable release target (x.y): 1.33
• Alpha
  • KEP (k/enhancements) update PR(s):
    • Sidecar containers KEP #3761
    • move sidecar to milestone 1.28 #3968
  • Code (k/k) update PR(s):
    • Add SidecarContainers feature kubernetes#116429
  • Docs (k/website) update PR(s):
    • Add initial (alpha) documentation about sidecar containers website#41802
• Beta
  • KEP (k/enhancements) update PR(s): KEP-753: add PRR questionnaire answers for beta #4255
  • Code (k/k) update PR(s):
    • sidecars: terminate sidecars after main containers kubernetes#120620
    • Promote SidecarContainers feature to beta kubernetes#121579
    • Add cel new validator into Kubernetes kubernetes#121307
  • Docs (k/website) update(s): Document SidecarContainers graduation to beta website#43471
• Stable
  • KEP (k/enhancements) update PR(s):
    • KEP 753: SidecarContainers is GA #5081
    • KEP-753: SidecarContainers update oom score adj logic #4932
  • Code (k/k) update PR(s): Promote SidecarContainers feature to GA kubernetes#129731
  • Docs (k/website) update(s): update SidecarContainers to stable in v1.33 website#49685

/sig node

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

[Joseph-Irving]

@enisoc @dchen1107 @fejta @thockin @kow3ns @derekwaynecarr, opened this tracking issue so that we can discuss.

[kow3ns]

/assign

[Joseph-Irving]

@derekwaynecarr I've done some scoping out of the kubelet changes required for next week's sig-node meeting, I believe that changes are only needed in the kuberuntime package, specifically kuberuntime_manager.go in and kuberuntime_container.go.

In kuberuntime_manager.go you could modify computePodActions to implement the shutdown triggering (kill sidecars when all non-sidecars have permanently exited), and starting up the sidecars first.

In kuberuntime_container.go you could modify killContainersWithSyncResult for terminating the sidecars last and sending the preStop hooks (the preStop hooks bit was a bit debatable, it wasn't settled whether that should be done or not. @thockin had a good point about why you might not want to encourage that behaviour, see comment).

Let me know if you want me to investigate any further.

[resouer]

@kow3ns The discussion makes more sense to me if maybe we can define a full description of containers sequence in Pod spec (sig-app), and how to handle the sequence in kubelet for start, restart and cascading consideration (sig-node). Let's catch the Feb 5 sig-node meeting to give more inputs.

cc @Joseph-Irving

[luksa]

The proposal says that sidecars only run after the init containers run. But what if the use-case requires the sidecar to run while/before the init containers run. For example, if you'd like route the pod's traffic through a proxy running as a sidecar (as in Istio), you probably want that proxy to be in place while the init containers run in case the init container itself does network calls.

[Joseph-Irving]

@luksa I think there's the possibility of looking at having sidecars that run in init phase at some point but currently the proposal is not going to cover that use case. There is currently no way to have concurrent containers running in the init phase so that would be potentially a much larger/messier change than what is being suggested here.