31 May 2025

Kubernetes notes !!

Heads up! This article is a work in progress.

Packages required:

nix-shell -p minikube kubectl kubernetes-helm openssl --command fish

Minikube Quick Start

Get a simple single node cluster running:

# start the cluster
minikube start

# access the dashboard
minikube dashboard

# test deployment
kubectl create deployment hello-node --image=registry.k8s.io/e2e-test-images/agnhost:2.39 -- /agnhost netexec --http-port=8080

# verify it exists: (CLI or dashboard)
kubectl get deployments
kubectl get pods

# stuff that happens:
kubectl get events

# cluster configuration:
kubectl config view

# application logs
kubectl get pods # to find id
kubectl logs hello-node-66d457cb86-d5266

# make the pod accessible using a service
kubectl expose deployment hello-node --type=LoadBalancer --port=8080
kubectl get services
minikube service hello-node # opens the browser to the load balancer

For minikube, there exist addons you can enable, for example metrics:

minikube addons list # view the whole list
minikube addons enable metrics-server # enable metrics
kubectl get pod,svc -n kube-system # view pod/service created
kubectl top pods # now you can query top processes
minikube addons disable metrics-server # disable after

Okay clean up now:

kubectl delete service hello-node
kubectl delete deployment hello-node
minikube stop
minikube delete

Deploy

For example, we want to deploy January:

# january.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: january-deployment
  labels:
    app: january
spec:
  replicas: 3
  selector:
    matchLabels:
      app: january
  template:
    metadata:
      labels:
        app: january
    spec:
      containers:
        - name: january
          image: ghcr.io/revoltchat/january:20250530-1
          ports:
            - containerPort: 14705

Then run:

kubectl apply -f january.yaml

Performing an update according to docs:

# e.g. update version of image
kubectl set image deployment/january-deployment january=ghcr.io/revoltchat/january:20250515-1

# ... you can also edit:
kubectl edit deployment/january-deployment
# (or re-apply the file!)

# monitor the rollout:
kubectl rollout status deployment/january-deployment

# get the replicasets:
kubectl get rs # the old ones will get deleted
               # but you can set a limit! https://stackoverflow.com/a/37258369

# undo!!!
kubectl rollout undo deployment/january-deployment

# describe
kubectl describe deployment january-deployment

Ingress!

Let’s put an ingress in front of it.

Using Caddy

Deploy Caddy ingress controller.

Run the following:

minikube tunnel
kubectl create namespace caddy-system
helm install \
  --namespace=caddy-system \
  --repo https://caddyserver.github.io/ingress/ \
  --atomic \
  mycaddy \
  caddy-ingress-controller

# it should provision an "external ip":
kubectl get svc -n caddy-system

Now let’s deploy an ingress:

# january-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: january
  annotations:
    spec.ingressClassName: caddy
spec:
  rules:
  - host: january.local
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: january
            port:
              number: 80
  tls:
    - secretName: mycerts
      hosts:
        - january.local

openssl req -x509 -newkey rsa:4096 -keyout tls.key -out tls.crt -sha256 -days 3650 -nodes -subj "/C=XX/ST=StateName/L=CityName/O=CompanyName/OU=CompanySectionName/CN=january.local"
kubectl create secret tls mycerts --key ./tls.key --cert ./tls.crt
kubectl apply -f january-ingress.yaml
curl -sk --resolve january.local:443:10.104.136.31 https://january.local

Misc. notes

MongoDB (https://artifacthub.io/packages/helm/bitnami/mongodb)

kubectl create namespace mongo-system
helm install among-us-db --namespace mongo-system oci://registry-1.docker.io/bitnamicharts/mongodb
# now available at among-us-db-mongodb.mongo-system.svc.cluster.local

# root pwd:
export MONGODB_ROOT_PASSWORD=$(kubectl get secret --namespace mongo-system among-us-db-mongodb -o jsonpath="{.data.mongodb-root-password}" | base64 -d)

# connect as admin:
kubectl run --namespace mongo-system among-us-db-mongodb-client --rm --tty -i --restart='Never' --env="MONGODB_ROOT_PASSWORD=$MONGODB_ROOT_PASSWORD" --image docker.io/bitnami/mongodb:8.0.9-debian-12-r4 --command -- bash
mongosh admin --host "among-us-db-mongodb" --authenticationDatabase admin -u $MONGODB_ROOT_USER -p $MONGODB_ROOT_PASSWORD

# external conn:
kubectl port-forward --namespace mongo-system svc/among-us-db-mongodb 27017:27017 &
mongosh --host 127.0.0.1 --authenticationDatabase admin -p $MONGODB_ROOT_PASSWORD

# ... tear it down:
helm uninstall among-us-db --namespace mongo-system

# try replicaset instead:
helm install among-us-db --namespace mongo-system --set architecture=replicaset oci://registry-1.docker.io/bitnamicharts/mongodb
# ... this time it'll give you a set of DNS names

# try scaling it up:
kubectl scale statefulsets --namespace mongo-system among-us-db-mongodb --replicas=3

# purge everything:
kubectl delete all -n mongo-system --all
kubectl delete namespace mongo-system