New Home Lab

Where better to start than how my lab is working.

My home lab is not one that has really been planned, rather I acquire/buy things as the opportunity arises. It really has 2 things:

• NAS
• Kubernetes

The NAS

My NAS is not really a NAS, rather it is an HP Microserver NL36 that runs NFS. I will phase this out eventually, but it is running nicely and I do not ask much of it. The disks are mirrored (using LVM mirror) and I replace them periodically to make sure the data is relatively safe. I think I am on the 3rd complete set of disks since I got it in 2012. I know that, one day, the host itself will die. At that point I will simply get a new box to host the data disks.

It is running Rocky Linux with KVM so I can run a couple of simple VMs:

• Pi-hole
• Bastion
• Tailscale
• Unifi controller

I also have a Raspberry Pi Zero that runs Pi-hole, so there is really nothing essential running except NFS.

Kubernetes

I run Kubernetes because I like it and it looks something like this:

As I want to play around with stuff in Kubernetes and not Kubernetes itself I decided to simply run K3s which makes it super simple.

Because I am realistic and have a family and a house to run, my cluster is really simple:

• A single server/master (running in a VM on the HP Microserver)
• Multiple agents on NUCs of various generations.

In terms of hardware that is it, I will probably add some ARM64 single board computers (SBC) when I get round to it, but I am not in a rush. The oDroid M1S looks pretty tempting.

Access is taken care of using MetalLB to provide Load Balancers, and I simply assign about 20 addresses for use on my Lab VLAN. I also run 2 Ingress Controllers:

• Traefik - deployed alongside K3s and I use this for internal access only.
• Nginx - My router forwards HTTP to this one so they are accessible externally.

I have 2 types of storage: Longhorn and NFS. I went this way because the NAS has TBs of space, but relatively low speed. The NUCs have all got SSDs in them, but they are physically small, so space is naturally limited.

Anything that will work on NFS gets provisioned using the NFS subdir external provisioner. Obviously anything that needs a lot of space too.

For when I need higher performance, I use Longhorn which shares the SSDs in the NUCs themselves. This gives more than enough performance than I expect without the workload being locked to a specific node.¹ Some things needing Longhorn’s performance have surprised me. Jellyfin’s media works fine on NFS, but ESPHome was a disaster.

I manage everything in the K3s cluster using FluxCD and a repo in Gitlab. I do not host that repo locally because, if I were to host code locally, it would be in the K3s cluster. There is a chicken-egg situation there, but I am happy with Gitlab.

So that is my home lab as of February 2024.

• Kubernetes
• Lab