Setup a mTLS enabled Ingress in Kubernetes

Posted by on April 8, 2024 No comments

mTLS is a widely used for securing sensitive data the world over. It is dependent on that both sides of a communication trust each other before a transaction can happen. The trust is based on certificates and can be devided into two parts, one part where the clients needs to trust the servers presented certificate and another part where the server needs to trust the clients presented certificate. We are now going to see an example of how to set this up in Kubernetes

All certificates are going to be self-signed in this example, regular certificates from trusted sources like Thwate, GlobalSign and many others will also work.

For Kubernetes I will use Minikube with the Ingress addon:

minikube addons enable ingress

1. First we need a server certificate to present to any client wanting to connect

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout server.key -out server.crt -subj "/CN=test.localdev.me/O=test.localdev.me"

This should give you two files, a server.key and a server.crt file with the private key and the certificate to present to clients trying to connect.
NOTE: the “domain” test.localdev.me will normally return 127.0.0.1 automatically which makes it perfect to use in cases like this

2. Lets add the certificate to the cluster via a Secret and the special type tls

kubectl create secret tls server-certificate --key server.key --cert server.crt

3. Now we need the client key and certificate. We start by creating our own “CA Authority”

openssl req -x509 -sha256 -newkey rsa:4096 -keyout ca.key -out ca.crt -days 356 -nodes -subj "/CN=My CA"

4. Add the CA to the cluster as a Secret with the type ca-secret

kubectl create secret generic ca-secret --from-file=ca.crt=ca.crt

5. A CSR for our client cert

openssl req -new -newkey rsa:4096 -keyout client.key -out client.csr -nodes -subj "/CN=My Client"

6. Sign the CSR with our CA (same we put into the cluster)

openssl x509 -req -sha256 -days 365 -in client.csr -CA ca.crt -CAkey ca.key -set_serial 02 -out client.crt

We should now have a client.key and a client.crt ready to use

7. Now we need an application to call. We create one with the Deployment below:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: mywebserver
  name: mywebserver
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mywebserver
  template:
    metadata:
      labels:
        app: mywebserver
    spec:
      containers:
      - image: httpd
        name: httpd
        ports:
        - containerPort: 80

8. A Service to expose the application to the cluster

apiVersion: v1
kind: Service
metadata:
  labels:
    app: my-service
  name: my-service
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: mywebserver

9. A Ingress to handle the authentication

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream: "true"
    nginx.ingress.kubernetes.io/auth-tls-secret: default/ca-secret
    nginx.ingress.kubernetes.io/auth-tls-verify-client: "on"
    nginx.ingress.kubernetes.io/auth-tls-verify-depth: "1"
  name: mtls-ingress
  namespace: default
spec:
  ingressClassName: nginx
  rules:
  - host: test.localdev.me
    http:
      paths:
      - backend:
          service:
            name: my-service
            port:
              number: 80
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - test.localdev.me
    secretName: server-certificate

10. Time to test our mTLS setup
First we need to setup a port binding to port 443 on our local machine

sudo kubectl port-forward -n ingress-nginx service/ingress-nginx-controller 443:443

and now we can test with a call with our client certificate and key

curl -k -v https://test.localdev.me/ --key client.key --cert client.crt

If everything is working we should get “It works!” from the Web Server

Tested in Minikube 1.26.0 and with OpenSSL 1.1.1f on Ubuntu 20.08

‘kubectl’ on WSL2 gives you “The connection to the server localhost:8080 was refused – did you specify the right host or port?” when using ‘sudo’

Posted by on April 8, 2024 No comments

I ran into this problem trying to do a port-forward on port 443 in WSL2. Normally you don’t use ‘sudo’ for ‘kubectl’ commands but when it comes to port-forwarding ports under port 1024, 443 in this case, you need ‘sudo’ to get your OS to accept the binding of the port

The solution for me was to make sure the root user had access to the .kube/config file of my normal user. I did this via the environment variable KUBECONFIG:

1. Open a root shell

sudo su

2. Export the KUBECONFIG with a path to my regular users .kube/config

export KUBECONFIG=/home/niklas/.kube/config

3. Do the port-forward to port 443 (or anything else that needs root access)

kubectl port-forward -n ingress-nginx service/ingress-nginx-controller 443:443

I wasn’t looking for a permanent solution so this worked for the few tests I needed to do while setting up my Ingress for mTLS authentication

Tested on ‘kubectl’ 1.26, WSL2 with Ubuntu 20.08 and Kubernetes v1.26 on Minikube

Setup a simple Wiremock service in Kubernetes

Posted by on April 5, 2024 No comments

Wiremock is a great tool for many things and especially black box testing. I’m here going to show a small and simple way to setup a Wiremock implementation in Kubernetes

First we need a Deployment of the Wiremock application:

# Deploy wiremock to Kubernetes
apiVersion: apps/v1
kind: Deployment
metadata:
  name: wiremock
  labels:
    app: wiremock
spec:
  replicas: 1
  selector:
    matchLabels:
      app: wiremock
  template:
    metadata:
      labels:
        app: wiremock
    spec:
      containers:
        - name: wiremock
          image: rodolpheche/wiremock
          ports:
            - containerPort: 8080
          volumeMounts:
            - name: mappings
              mountPath: /home/wiremock/mappings
            - name: files
              mountPath: /home/wiremock/__files
      volumes:
        - name: mappings
          configMap:
            name: wiremock-mappings
        - name: files
          configMap:
            name: wiremock-files

Then we need a Service to expose Wiremock to the cluster

apiVersion: v1
kind: Service
metadata:
  name: wiremock
spec:
    selector:
        app: wiremock
    ports:
        - protocol: TCP
          port: 8080
          targetPort: 8080

Two ConfigMaps for configuration and files:

apiVersion: v1
kind: ConfigMap
metadata:
  name: wiremock-mappings
data:
    mappings.json: |
        {
            "request": {
                "method": "GET",
                "url": "/api/v1/hello"
            },
            "response": {
                "status": 200,
                "bodyFileName": "hello.json",
                "headers": {
                    "Content-Type": "application/json"
                }
            }
        }
        
---

apiVersion: v1
kind: ConfigMap
metadata:
  name: wiremock-files
data:
    hello.json: |
        {
        "Hello": "World"
        }

And finally a call to the mock to see that everything works as expected:

 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: check-wiremock
  labels:
    app: check-wiremock
spec:
    replicas: 1
    selector:
        matchLabels:
          app: check-wiremock
    template:
        metadata:
          labels:
            app: check-wiremock
        spec:
          containers:
            - name: check-wiremock
              image: cosmintitei/bash-curl
              imagePullPolicy: Always
              command: ["/bin/sh", "-c", "while true; do date; curl -sS wiremock:8080/api/v1/hello; sleep 10; done"]

You should now see the payload from the Wiremock application every ten second in the check-wiremock applications log

Tested in Minikube v1.29.0