My first MicroK8s cluster on Raspberry Pi

I have acquired some Raspberry Pis. One version 4, and two version 3. The following is the highlighted steps that I’ve taken to set them up, so I have a future cheat-sheet to look back on.

The rest of the hardware acquired is the cheapest network switch I could find (it was on sale), a few ethernet cords, a few USB cords, and a few 32GB microsd cards. Nothing special.

Installing Ubuntu

I’m currently working on a Windows 10 Pro machine. Once I insert the new microsd card into the reader, I see it on the list of drives.

SD Card New

I can right click and format the SD Card to make sure it’s ready.

SD Card format

I intend to be installing Ubuntu Server 19.10 because I’m fine with rolling a non-LTS on a playground server. Download is here.

Ubuntu Download

Once the image is downloaded, I write it to the microsd card using Win32 Disk Imager.

Disk Imager

Go ahead and insert the microsd card into the Raspberry Pi and plug it in.


Now there is one problem. I couldn’t get Ubuntu 19.10 on Raspberry Pi to recognize any keyboards, perhaps related to this bug report.

I’m not sure. I didn’t really spend a lot of time figuring it out.

Instead, I went and looked what new devices were on my network.

New Device

Using that address, I can SSH to it using WSL (Windows Subsystem for Linux). The first login has user “ubuntu” and password “ubuntu”, and then requests a password change for that user.

$ ssh ubuntu@
The authenticity of host ' (' can't be established.
ECDSA key fingerprint is SHA256:<redacted>.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '' (ECDSA) to the list of known hosts.
ubuntu@'s password:
You are required to change your password immediately (administrator enforced)
Welcome to Ubuntu 19.10 (GNU/Linux 5.3.0-1007-raspi2 aarch64)

 * Documentation:
 * Management:
 * Support:

  System information as of Sun Nov 17 18:31:48 UTC 2019

  System load:  2.13              Processes:           130
  Usage of /:   8.4% of 29.05GB   Users logged in:     0
  Memory usage: 40%               IP address for eth0:
  Swap usage:   0%

 * Kata Containers are now fully integrated in Charmed Kubernetes 1.16!
   Yes, charms take the Krazy out of K8s Kata Kluster Konstruction.

18 updates can be installed immediately.
11 of these updates are security updates.
To see these additional updates run: apt list --upgradable

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

WARNING: Your password has expired.
You must change your password now and login again!
Changing password for ubuntu.
Current password:

At this point, change the password and then mess with users and credentials however your preferences are.

Log back into the Raspberry Pi. I like to do a couple steps beginning with the traditional update all packages.

sudo apt update

Then upgrade

sudo apt upgrade

I like to change my hostname so I know which machine it is. “pi301” refers to it being a Raspberry Pi 3, and then “01” refers to a zero-based count of how many are on the network. This is the second Pi3 on the network.

sudo hostnamectl set-hostname pi301 --static

One more thing, is to enable cgroup memory.

To look at what’s enabled, run

cat /proc/cgroups

The output will look something like this, and the memory row will be “0” for disabled.

#subsys_name    hierarchy       num_cgroups     enabled
cpuset  7       1       1
cpu     5       1       1
cpuacct 5       1       1
blkio   10      1       1
memory  0       96      0
devices 8       87      1
freezer 4       2       1
net_cls 2       1       1
perf_event      9       1       1
net_prio        2       1       1
pids    6       94      1
rdma    3       1       1

Edit the boot firmware nobtcmd.txt (and not cmdline.txt as many places say)

sudo vi /boot/firmware/nobtcmd.txt

Prepend the following arguments to whatever is in there.

cgroup_enable=memory cgroup_memory=1

Optionally, go ahead and give your Raspberry Pi a reserved address on your network. This will vary based on what hardware and management software is running, but it might look something like this.

Reserved Address

Now, go ahead and reboot the system

sudo reboot

If everything goes well, you should be able to SSH into the machine at its new address within a minute or two.

Make a quick check on cgroups to make sure that memory is enabled.

cat /proc/cgroups

Installing MicroK8s

MicroK8s comes as an easy to use Snap package.

$ sudo snap install microk8s --classic
microk8s v1.16.2 from Canonical✓ installed

Letting user run MicroK8s without sudo is done by this command. The user “ubuntu” will be able to run the “microk8s” command with sudo on their next login. This is optional, and your preference.

sudo usermod -a -G microk8s ubuntu

Let’s check on MicroK8s and see what’s happening.

sudo microk8s.status

Sometimes, if the above command doesn’t respond. There might be something or other wrong. Checking on the kubelet with journalctl is a good place to start looking for info.

journalctl -f -u snap.microk8s.daemon-kubelet

An easy enough debugging step is to run a stop and start.

sudo microk8s.stop
sudo microk8s.start

Look at the cluster info and make sure it’s up:

$ sudo microk8s.kubectl cluster-info
Kubernetes master is running at

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Joining a cluster

Joining a cluster is easy. I happen to already have a MicroK8s master node running on the Raspberry Pi 4. So once I ssh into that, I can run a quick command to get enough info to join a worker node (the new Pi 3 setup above)

$ sudo microk8s.add-node
Join node with: microk8s.join< redacted >

If the node you are adding is not reachable through the default interface you can use one of the following:
 microk8s.join< redacted >
 microk8s.join< redacted >
 microk8s.join< redacted >

Run the command on the worker node to add.

sudo microk8s.join< redacted >

Dashboard and remote login

Whether you now have a cluster or a single Pi running MicroK8s, let’s add a dashboard to it.

Run this command on the master node to enable these features.

sudo microk8s.enable dashboard dns

Now check on those deployments, of which one seems to be having problems for me.

$ sudo microk8s.kubectl get deployments --all-namespaces
NAMESPACE     NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
kube-system   coredns                          1/1     1            1           13h
kube-system   dashboard-metrics-scraper        0/1     1            0           91m
kube-system   heapster-v1.5.2                  1/1     1            1           91m
kube-system   kubernetes-dashboard             1/1     1            1           91m
kube-system   monitoring-influxdb-grafana-v4   1/1     1            1           91m

So far throughout these steps, I’ve been SSH’d into one machine or another. But now I want to work on this cluster from my own machine without SSH.

Go ahead and install kubectl on your local machine, in whatever installation method appropriate.

In your home directory, there will be a hidden directory called “.kube” (or you might have to make it yourself). Inside this, create a file named “config” and paste the following in.

apiVersion: v1
- cluster:
    insecure-skip-tls-verify: true
  name: raspberrypi
- context:
    cluster: raspberrypi
    user: raspberrypiadmin
  name: raspberrypi
current-context: raspberrypi
kind: Config
preferences: {}
- name: raspberrypiadmin
    password: < redacted password >
    username: admin

To find the password, look on the master node.

ubuntu@pi400:/var/snap/microk8s/current/credentials$ ls
basic_auth.csv       certs-request-tokens.txt         client.config       cluster-tokens.txt.backup  known_tokens.csv  proxy.config
callback-tokens.txt  certs-request-tokens.txt.backup  cluster-tokens.txt  controller.config          kubelet.config    scheduler.config

There are a number of files that detail credential methods. The password method isn’t great, but I’m going to roll with it for the sake of getting up to speed quickly in a playground environment.

ubuntu@pi400:/var/snap/microk8s/current/credentials$ cat basic_auth.csv
< redacted password >,admin,admin,"system:masters"

On your working machine, save the file and run kubectl to see what the current context is.

$ kubectl config current-context

To test it, let’s see if a command works.

$ kubectl cluster-info
Kubernetes master is running at
Heapster is running at
CoreDNS is running at
Grafana is running at
InfluxDB is running at

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

On your working machine, run this command to open a proxy.

kubectl proxy --accept-hosts=.* --address=

Now open a browser and go to this address

Token Login

One method to login is by a token. The token may be found by running these two commands on the master node.

token=$(microk8s.kubectl -n kube-system get secret | grep default-token | cut -d " " -f1)
microk8s.kubectl -n kube-system describe secret $token

Once logged in, click on “Nodes” on the menu on the left. Your cluster may have more or less nodes depending on how many Raspberry Pis that you’ve setup.

Cluster Nodes


I have a working Raspberry Pi MicroK8s cluster. As shown in the picture below, it badly needs better organization and to be cleaned up. I’m not sure what my next steps will be. Better security and cable management are probably good places to start.

Cluster Picture