markdown 通过LXC / LXD运行Minikube

Posted 2021-05-05

# Running Minikube via LXD

I make some assumptions, and make no claims in how well supported this is or ever will be. I wanted to avoid using VMs
because i've been working in containers for the last half decade. It made sense to just skip the middle man and use a
machine type container system to run my minikube workloads.

#### Why not juju?

Simply put, Juju does a fantastic job; but to stay objective I wanted to achieve minikube in LXD
as a functional alternative to `juju deploy kubernetes-core`, or using KVM/VirtualBox in this solution.

## Prereqs

You'll need to install some things to make this work. I'm going to presume you're on an Ubuntu LTS installation (16.04 plz)

```
sudo apt-get install -y lxd
sudo lxd init
# configure lxd with the prompts. This is mostly trivial, i did however skip ipv6 networking and opted for ipv4 only.
sudo snap install kubectl
```

Once you've got lxd installed and configured, you're ready to create the profile and launch your minikube "machine".

```
lxc profile create minikube
lxc profile edit minikube
```

Put the following contents in your minikube profile verbatim

```
name: minikube
config:
  linux.kernel_modules: ip_tables,ip6_tables,netlink_diag,nf_nat,overlay
  raw.lxc: |
    lxc.aa_profile=unconfined
    lxc.mount.auto=proc:rw sys:rw
    lxc.cap.drop=
  security.nesting: "true"
  security.privileged: "true"
description: Profile supporting minikube in containers
devices:
  aadisable:
    path: /sys/module/apparmor/parameters/enabled
    source: /dev/null
    type: disk
```

Now, launch your minikube container

```
lxc launch ubuntu:16.04 minikube
lxc profile apply minikube default,minikube
```

From here, you're ready to enter the container and setup the components

```
lxc exec minikube /bin/bash
```

## Inside the container

```
curl https://get.docker.com | bash
curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/

minikube start --apiserver-name minikube --vm-driver none
```

This will spin up the minikube instance. If you dont get any errors in `minikube log` you're nearly complete!

Install kubectl
```
curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin/kubectl
```
```
cd /root/.minikube
kubectl config --kubeconfig=minikube set-cluster minikube --server=https://kubernetes:8443 --certificate-authority=ca.crt --embed-certs=true
kubectl config --kubeconfig=minikube unset users
kubectl config --kubeconfig=minikube set-credentials minikube --client-key=client.key --client-certificate=client.crt --embed-certs=true
kubectl config --kubeconfig=minikube set-context default --cluster=minikube --user=minikube
kubectl config --kubeconfig=minikube use-context default
```

Awesome!  We have a portable kubeconfig now too. we're ready to exit the container

```
exit
```

### Back on our host

We'll need to do 2 final things to finish the setup. We need to grab that kubeconfig from the minikube container, and
we'll need to do an `/etc/hosts` poison to satisfy the x509 validation on the TLS certificates

To get the IP address of the container, you can re-exec into it, or run `lxc list` to  get the IP from the listing.

```
+----------+---------+--------------------------------+------+------------+-----------+
|   NAME   |  STATE  |              IPV4              | IPV6 |    TYPE    | SNAPSHOTS |
+----------+---------+--------------------------------+------+------------+-----------+
| minikube | RUNNING | 172.17.0.1 (docker0)           |      | PERSISTENT | 0         |
|          |         | 10.169.52.195 (eth0)           |      |            |           |
+----------+---------+--------------------------------+------+------------+-----------+
```


So we'll put that in our `/etc/hosts` file. Included snippet for clarity if you haven't poisoned your DNS before.

```$ cat /etc/hosts
127.0.0.1	localhost
127.0.1.1	bushido
10.169.52.195 kubernetes
```

Now grab the kubeconfig from the container that we generated and we're ready to go

```
lxc exec minikube cat /root/.minikube/minikube > kubeconfig
kubectl --kubeconfig kubeconfig  get no

NAME       STATUS    ROLES     AGE       VERSION
minikube   Ready     <none>    25m       v1.7.5
```

Viola! 

You can do everything you would do with k8s in a vm (barring some testing and limitations mind you, but it should be pretty close!)  

```
kubectl --kubeconfig kubeconfig proxy --port 8080 (maybe you should configure port)
```

now visit [https://localhost:8001/ui](https://localhost:8001/ui) in your browser and start deploying the world!


#### Disclaimer

This is not an official project, I'm happy to help get you started if you're interested in this or in making it an officially supported mechanism by the minikube project. However - with that being said this is the hackers warranty. You've set this up and if/when it breaks, there's no warranty and I'm not supporting this in an official project capacity. 

Best of luck and happy hacking!