Path based routing in a Kubernetes Ingress (Nginx)

Posted by on May 28, 2024 No comments

Here we want to route traffic to different applications within our cluster with the help of paths. One advantage of this approach is that we only need one server certificate, since all traffic is going to use the same host.

We will focus on the Ingress here and not the Service object (every application that exposes services will need a Service object as a bridge between the application and the Ingress)

Example:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-application-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  ingressClassName: nginx
  rules:
 - host: my.domain.com
   http:
     paths:
     - path: /app-a/(.*)
       pathType: Prefix
       backend:
         service:
           name: my-app-a-service
           port:
             number: 8080  
     - path: /app-b/(.*)
       pathType: Prefix
       backend:
         service:
           name: my-app-b-service
           port:
             number: 8080

With the example configuration above we see that the following url’s are valid:

my.domain.com/app-a/ # will hit the root of my-app-a-service at "/"
my.domain.com/app-a/actuator/health # my-app-a-service at "/actuator/health"
my.domain.com/app-b/service # will route to my-app-b-service at "/service"

How does it work?
First we look at the paths: “/app-a/(.*)”. The “(.*)” part is a regular expression that means “match all characters after the slash (“/”) and put it into a group” .

A little higher up in the configuration we find “nginx.ingress.kubernetes.io/rewrite-target: /$1” annotation. This tells the Ingress that we should extract the first group (“$1”) and forward it to the backend Service. This is the way we remove the first part of the path (/app-a/). We only use this part to separate to what service the call should go and do not want it to follow the call to the backend. Everything after the last slash (“/”) is forwarded to the application, both url and any query parameters.

A nifty solution when you don’t need every service to be a separate domain

Tested on VMWare Tanzu Kubernetes v1.22

WireMock: Verify payload sent to a mocked service in JUnit 5

Posted by on May 13, 2024 No comments

This was not totally logical to me so I’ll write the solution down here for myself and anyone else that might have the same problem as me 🙂

The solution is to use the WireMock.verify function to setup a payload assertion.

Example (pseudo code):

import com.github.tomakehurst.wiremock.client.WireMock;
import com.niklasottosson.myApplication;
import org.junit.jupiter.api.Test;

import static com.github.tomakehurst.wiremock.client.WireMock.equalToXml;
import static com.github.tomakehurst.wiremock.client.WireMock.postRequestedFor;
import static com.github.tomakehurst.wiremock.client.WireMock.urlEqualTo;


public class MyApplicationIntegrationTest {

  @Test
  public void happyCaseTest() {

    String expected = "Hello Test";
    String myPath = "/mymockservice"

    // 1. Setup WireMock
    WireMock.stubFor(post(urlEqualTo(myPath))
       .willReturn(
             aResponse()
                   .withStatus(200)
                   .withHeader("Content-Type", "text/xml")
                   .withBody("Hello from mock service")));        

   // 2. Run system under test
   myApplication.start();        

   // 3. Verify payload sent to mock service
   WireMock.verify(
       postRequestedFor(urlEqualTo(myPath))
            .withRequestBody(equalToXml(expected))
    );
  }
}

1. Setup a WireMock stub for receiving calls from myApplication on a specific path
2. Start system under test, myApplication in this case
3. Verify that a call has been made to the path AND with a request payload matching our “expected” result. If this validates the payload is as “expected” 🙂

So in conclusion, WireMock is doing the assertion here, not our testing framework

Tested with WireMock v.3.1.0

Validate subject information in a mTLS configured Ingress (Kubernetes)

Posted by on April 14, 2024 No comments

In large organisations you often need additional ways to validate a client certificate since most or all certificates use the same CA and you might want to have a more fine grained validation. To use this type of extra validation we also need to setup mTLS. This is a example of how to accomplish both mTLS and an extra layer of validation

All certificates are going to be self-signed in this example, regular certificates from trusted sources like Thwate, GlobalSign and many others will naturally also work.

For Kubernetes I will use Minikube with the Ingress addon:

minikube addons enable ingress

1. First we need a server certificate

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout server.key -out server.crt -subj "/CN=test.localdev.me/O=test.localdev.me"

This should give you two files, a server.key and a server.crt file with the private key and the certificate.

2. Lets add the certificate to the cluster via a Secret and the special type tls

kubectl create secret tls server-certificate --key server.key --cert server.crt

3. Now we need the client key and certificate. We start by creating our own “CA Authority”

openssl req -x509 -sha256 -newkey rsa:4096 -keyout ca.key -out ca.crt -days 356 -nodes -subj "/CN=My CA"

4. Add the CA to the cluster as a Secret with the type ca-secret

kubectl create secret generic ca-secret --from-file=ca.crt=ca.crt

5. A CSR for our client cert

openssl req -new -newkey rsa:4096 -keyout client.key -out client.csr -nodes -subj "/CN=MyClient"

6. Sign the CSR with our CA (same we put into the cluster)

openssl x509 -req -sha256 -days 365 -in client.csr -CA ca.crt -CAkey ca.key -set_serial 02 -out client.crt

We should now have a client.key and a client.crt ready to use

7. Another client certificate for testing the “match” function
CSR:

openssl req -new -newkey rsa:4096 -keyout client_2.key -out client_2.csr -nodes -subj "/CN=MyOtherClient"

Sign:

openssl x509 -req -sha256 -days 365 -in client_2.csr -CA ca.crt -CAkey ca.key -set_serial 02 -out client_2.crt

8. Now we need an application to call. We create one with the Deployment and Service below:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: mywebserver
  name: mywebserver
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mywebserver
  template:
    metadata:
      labels:
        app: mywebserver
    spec:
      containers:
      - image: httpd
        name: httpd
        ports:
        - containerPort: 80

---

apiVersion: v1
kind: Service
metadata:
  labels:
    app: my-service
  name: my-service
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: mywebserver

8. Now we need to configure the Ingress for mTLS and our extra layer of authentication:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream: "true"
    nginx.ingress.kubernetes.io/auth-tls-secret: default/ca-secret
    nginx.ingress.kubernetes.io/auth-tls-verify-client: "on"
    nginx.ingress.kubernetes.io/auth-tls-verify-depth: "1"
    nginx.ingress.kubernetes.io/auth-tls-match-cn: "CN=MyClient"
  name: mtls-ingress
  namespace: default
spec:
  ingressClassName: nginx
  rules:
  - host: test.localdev.me
    http:
      paths:
      - backend:
          service:
            name: my-service
            port:
              number: 80
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - test.localdev.me
    secretName: server-certificate

Here we added the nginx.ingress.kubernetes.io/auth-tls-match-cn for our extra validation. In this case we are looking for a “CN=MyClient” property in the Subject part of the client certificate. If the string is found we continue the communication between client and server, if not then the connection will be terminated with a HTTP 403 error

9. Time to test our mTLS setup with extra validation
First we need to setup a port binding to port 443 on our local machine

sudo kubectl port-forward -n ingress-nginx service/ingress-nginx-controller 443:443

and now we can test with a call with our client certificate and key

curl -k -v https://test.localdev.me/ --key client.key --cert client.crt

If everything is working we should get “It works!” from the Web Server

10. Now we are going to test the “match” function. Remember that both client.crt and client_2.crt uses the same CA so without the “auth-tls-match-cn” function they would both be accepted

curl -k -v https://test.localdev.me/ --key client_2.key --cert client_2.crt

This should fail and you should now get a HTTP 403 (Forbidden)

NOTE: A match with “CN=My Client” does not work! Spaces does not work when matching like this

Tested in Minikube 1.26.0 and with OpenSSL 1.1.1f on Ubuntu 20.08