document local minikube. Fixes tilt-dev/tilt#3193

Author: nicks

README.md

@@ -1,2 +1,68 @@
 # minikube-local
+
 The best way to set minikube up for local development
+
+[![Build Status](https://circleci.com/gh/tilt-dev/minikube-local/tree/master.svg?style=shield)](https://circleci.com/gh/tilt-dev/minikube-local)
+
+When using Tilt with a [Minikube](https://minikube.sigs.k8s.io/docs/) cluster, 
+we recommend using a local registry for faster image pushing and pulling.
+
+This repo documents the best way to set it up.
+
+## Why use Minikube with a local registry?
+
+Minikube offers many different ways to get your app into the cluster.
+
+Using a local registry is the best method for iterative app development.
+
+- Unlike with a remote registry, the image stays local to your machine, with no
+  network traffic to wait on or credentials to setup.
+
+- Unlike with an in-cluster builder, you can reset the cluster without deleting
+  the image cache.
+
+- Unlike with loading into the container runtime, docker will skip pushing any
+  layers that already exist in the registry.
+
+Over all these approaches, a local registry has good speed, incremental caching,
+and few footguns. But setting it up is awkward and fiddly. This script makes it
+easy.
+
+## How to Try It
+
+1) Install [Minikube](https://minikube.sigs.k8s.io/docs/)
+
+2) Copy the [minikube-with-registry.sh](minikube-with-registry.sh) script somewhere on your path.
+
+3) Create a cluster with `minikube-with-registry.sh`. Currently it creates the registry at port 5000.
+
+```
+minikube-with-registry.sh
+```
+
+4) Try pushing an image.
+
+```
+docker tag alpine localhost:5000/alpine
+docker push localhost:5000/alpine
+```
+
+You can now use the image name `localhost:5000/alpine` in any resources you deploy to the cluster.
+
+[Tilt](https://tilt.dev) will automatically detect the local registry created by this script.
+
+## Thanks to
+
+High five to [MicroK8s](https://github.com/ubuntu/microk8s) for the initial local registry feature
+that inspired a lot of this work.
+
+The Kind team ran with this, writing up documentation and hooks for how to [set up a local registry](https://kind.sigs.k8s.io/docs/user/local-registry/) with Kind.
+
+This repo adapts the Kind team's approach and applies the local registry configmap, so that tools
+like Tilt can discover the local-registry. This protocol is a [Kubernetes Enhancement Proposal](https://github.com/kubernetes/enhancements/issues/1755).
+
+## License
+
+Copyright 2020 Windmill Engineering
+
+Licensed under [the Apache License, Version 2.0](LICENSE)

minikube-with-registry.sh

@@ -0,0 +1,60 @@
+#!/bin/sh
+#
+# Adapted from:
+# https://github.com/kubernetes-sigs/kind/commits/master/site/static/examples/kind-with-registry.sh
+#
+# Copyright 2020 The Kubernetes Project
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -o errexit
+
+# desired profile name; default is ""
+MINIKUBE_PROFILE_NAME="${MINIKUBE_PROFILE_NAME:-minikube}"
+
+reg_name='minikube-registry'
+reg_port='5000'
+
+# create registry container unless it already exists
+running="$(docker inspect -f '{{.State.Running}}' "${reg_name}" 2>/dev/null || true)"
+if [ "${running}" != 'true' ]; then
+  docker run \
+    -d --restart=always -p "${reg_port}:5000" --name "${reg_name}" \
+    registry:2
+fi
+
+reg_host="$(docker inspect -f '{{.NetworkSettings.IPAddress}}' "${reg_name}")"
+echo "Registry Host: ${reg_host}"
+
+# create a cluster
+minikube start -p "$MINIKUBE_PROFILE_NAME" --driver=docker --container-runtime=containerd
+
+# patch the container runtime
+# this is the most annoying sed expression i've ever had to write
+minikube ssh sudo sed "\-i" "s,\\\[plugins.cri.registry.mirrors\\\],[plugins.cri.registry.mirrors]\\\n\ \ \ \ \ \ \ \ [plugins.cri.registry.mirrors.\\\"localhost:${reg_port}\\\"]\\\n\ \ \ \ \ \ \ \ \ \ endpoint\ =\ [\\\"http://${reg_host}:${reg_port}\\\"]," /etc/containerd/config.toml
+
+# restart the container runtime
+minikube ssh sudo systemctl restart containerd
+
+# document the registry
+cat <<EOF | kubectl apply -f -
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: local-registry-hosting
+  namespace: kube-public
+data:
+  localRegistryHosting.v1: |
+    host: "localhost:${reg_port}"
+    help: "https://github.com/tilt-dev/minikube-local"
+EOF

test/pod.yaml

@@ -0,0 +1,13 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: minikube-test
+  labels:
+    app: minikube-test
+spec:
+  containers:
+  - name: minikube-test
+    image: localhost:5000/busybox
+    command: ["sh", "-c", "busybox httpd -f -p 8000"]
+    ports:
+      - containerPort: 8000

test/test.sh

@@ -0,0 +1,14 @@
+#!/bin/bash
+#
+# Make sure the local registry works as expected.
+
+set -ex
+
+cd $(dirname $0)
+
+docker pull busybox
+docker tag busybox localhost:5000/busybox
+docker push localhost:5000/busybox
+kubectl delete -f pod.yaml --ignore-not-found
+kubectl create -f pod.yaml
+kubectl wait --for=condition=ready pod/minikube-test --timeout=60s