Kubernetes and etcd

Hyades runs as a cluster of machines, each running multiple containers. We would like to be able to easily create and destroy containers in a resource-efficient way without caring about the underlying infrastructure, as well as expand the network when necessary and handle machine failures. Kubernetes is used to manage this.

Kubernetes uses etcd to coordinate data across machines. Etcd operates on the Raft protocol, which we're happy with because it's considered a safer and better approach to data coherency than Paxos.

[insert more stuff about why Raft is awesome here]

Some Kubernetes terminology: A "node" is Kubernetes' abstraction of a machine (since it could also be either a physical or virtual machine) and a "pod" is Kubernetes' abstraction of a group of containers.

[Presumably, for user applications, we're only going to run one container per pod?]

In Hyades, we have a supervisor node that manages cluster administration, and multiple worker nodes each running end-user containers. The supervisor node contains, for example, kube-controller-manager (manages a number of "controllers" that supervise nodes and manage resource allocation) and kube-scheduler (assigns pods to nodes). Cluster administration operations are exposed to us through kube-apiserver.

With this architecture, if a node goes down, the supervisor node will find out about it and re-create its pods on other nodes.

[This system is even resistant to the supervisor node going down momentarily, since the worker nodes do not need the supervisor node for normal operation; the supervisor node is only required for pod creation and destruction?]

Kubernetes allows us to associate multiple pods to a single service. This is useful if, for example, we want to load-balance a computationally expensive operation to multiple machines. Each service is uniquely identified by an address which containers can access through kube-proxy.