What Can You Learn About Kubernetes in a Day (2023-01-19, to be precise)?

This is a summary of some work from Forskningsdag 2023-01-19.

Well firstly what isn't Kubernetes? Well what it isn't is a replacement for or an improvement on docker. In fact probably the majority of k8s (as we experts call it) deployments in the world are actually built entirely using docker containers - literally instantiations of docker images, retrieved from an image registry like docker.io. K8s is a framework (software ecosystem maybe) for deploying and managing those containers in various environments.

A Diversion

As a sidenote, I prepared for forskningsdag by bookmarking a link from Mastodon to a tutorial series on k8s from Medium. As I began to read through the series though, I became more and more frustrated at its vagueness and repetitiveness. In the end, I began to wonder if it had actually been written by ChatGPT. What do you think: https://medium.com/@Techie1/kubernetes-kubectl-commands-part-4-2cbc660656a9? 

Fortunately there are plenty of other tutorials out there - for example  https://kubernetes.io/docs/tutorials/kubernetes-basics/deploy-app/deploy-interactive/ . Most of these tutorials follow a similar path

1. Install a local k8s cluster
2. Deploy an app on it using a prebuilt docker image
3. Explore how to manage the app using k8s commands (typically via kubectl).

Now fortunately I already had a running k8s to use - I'd installed microk8s in order to deploy browsertrix-cloud. So it didn't take long to work through 2 and 3. I found myself irritated at having to remember command line tricks so I went looking for a dashboard (something like Portainer for docker) and quickly found the "standard" dashboard (https://upcloud.com/resources/tutorials/deploy-kubernetes-dashboard) although there are others.

And So What?

The problem is this: there are plenty of tutorials that will show you how to deploy some docker image or other in a k8s cluster and interact with it. But my use case as a developer is how to quickly deploy my custom code to a cluster for testing and perhaps production. Finding good tutorials on this kind of use case was harder and it took a fair amount of digging to finally find some of the necessary tools.

I started perhaps over-ambitiously with my favourite dockerised software NetarchiveSuite (https://github.com/netarchivesuite/netarchivesuite-docker-compose). And was quickly led to a tool called kompose which can convert a docker-compose.yaml file into a whole bunch of deployment and service yaml-files suitable for deploying in k8s. I ran kompose on my docker-compose.yml file

And threw the resulting files at my k8s cluster 

I'm not sure I entirely expected the whole thing to function first time, and lo and behold it didn't. The first problem I could see in the dashboard was simply that k8s could not find images for most of these containers. The issue I had was that my docker-compose file had many services like 

but for k8s deploy the service must also define an image e.g. 

So the first lesson is the kompose will always generate k8s deployments from your docker-compose config, but it's still up to you to write docker-compose config in such a way that the generated files are ready to use.

It's All About Image

At first I thought it would be as easy as 

1. Run "docker-compose push" to push my images to my local docker and
2. Add image tags for all the images to the docker-compose.yml .

That turns out not to be good enough because k8s assumes that all images are on docker.io unless an explicit alternative image registry is specified. Fortunately there is an image registry built-in with microk8s - it runs on localhost:32000. So the line above just needs to be changed to something like 

where ${REGISTRY_HOST} is defined in my .env file as localhost:32000. Except that the deployment files generated by kompose substitute the empty string for ${REGISTRY_HOST}  because it turns out that kompose ignores .env!!!

The workaround is to apply the environment variables first in a separate step: 

With this done I was actually able to deploy all my kubernetes pods, as they are called.

All my actual applications till died though and it was obvious from their logs that first of all none of them could see the NetarchiveSuite message broker. This points to an important difference between docker-compose and k8s. In docker-compose, a given assembly runs on its own custom network and all the applications can see each other if they are marked as linked in the docker-compose.yml file. You only have to specify explicitly which ports are to be visible externally.  But in k8s the applications may be running on different nodes and you have to specify all the ports that are required. So, for example, in the kompose-generated file mq-deployment.yaml I had to explicitly add

- containerPort: 7677

 or just add it as port in the docker-compose.yml. After similarly opening port 5432 for the database container I was finally able to start the NetarchiveSuite GUI application!

Getting Exposure

At this point I was running out of time. Using kubectl's port-forwarding 

I was able to expose the NetarchiveSuite GUI and load it in a browser. Unfortunately it seemed that the GUI wasn't talking to the other applications. There are clearly a bunch more ports I need to investigate, but at this point I ran out of time.

Take The Helm Number 1

Kompose generates 22 yaml files for this k8s deployment. That's a lot of files to maintain. There is a tool called Helm which seems to be widely used and which may be helpful here. It seems to be, so far as I can tell, a templating engine for k8s deployment - perhaps a bit like ansible for conventional deployment. In any case it's something to look into next time.