Having played around for a couple of months, I have a various bunch of services running in my cluster, which all needs some form of authentication. Some doesn’t even support authentication in itself, but could use some form of login in front of it. I am using traefik for reverse proxy, and it’s always possible to configure something like basic authentication in traefik, but there’s more fun in implementing a proper single signon mechanism at home.

As before, the selection process was pretty simple. I wanted something that is flexible, that allows various authentication options, and can support my current and future authentication/authorization needs.

The decision process was swift – Keycloak is a decent production ready single signon solution that can do OIDC, SAML, OAUTH2 and more.

There is a helm chart for Keycloak from Bitnami, who usually provides decent free solutions with things like paid support as optional extras. I»m here for the ride and learning experience, so the free version is perfect for me.

Whenever I install something with helm, I like to have a thorough reading on what configuration options the helm chart provides:

helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update
helm show values bitnami/keycloak > values.yaml

This values.yaml file will show me the default configuration options and what I can tune for my installation.

To jump the gun a bit: After a whole lot of tinkering, I ended up with these changes:

production: true # To get it more production-ready.
...
proxyHeades: xforwarded # Add some info in headers
....
extraStartupArgs: "-Dkeycloak.provider.import=/opt/bitnami/keycloak/custom-providers" # To get support for custom providers
.....
initdbScripts: 
   custom_provider.sh: |
      #!/bin/bash
     cp /opt/bitnami/keycloak/extra-providers/* /opt/bitnami/keycloak/providers
....
extraEnvVars: 
  - name: CUSTOM_SPI_ENABLED
    value: "true"
...
replicaCount: 2 # Because I want to be able to restart Keycloak without downtime
....
extraVolumes: 
  - name: spi-jar
    configMap:
      name: trusted-device-jar

extraVolumeMounts: 
  - name: spi-jar
    mountPath: /opt/bitnami/keycloak/extra-providers

Then I added some settings to expose prometheus metrics, because I really love metrics. In fact, it will probably end up being another blog post, how to monitor your Kubernetes platform. But another day.

The extra provider support, I needed because Keycloak doesn’t support trusting a device, i.e. telling it that I don’t want 2FA anymore on a device because I trust it. I found a custom provider for that, which I wanted to support.

In addition to I created the trusted-device-jar configmap with the resulting JAR field after building the SPA.

kubectl create namespace keycloak
git clone https://github.com/wouterh-dev/keycloak-spi-trusted-device/
<..optionally install maven if not already installed..>
cd keycloak-spi-trusted-device
mvn package
kubectl create configmap trusted-device-jar --from-file=spi/target/my-spi.jar \
  -n keycloak

Then, installing the helm chart:

helm install -n keycloak keycloak bitnami/keycloak -f values.yaml

If everything is correct, you’ll now end up with 2 keycloak pods and one keycloak-postgresql pod, which will hold the configurations we create. Take a note of the information, you will need the initial admin password.

I also need a network policy, since I run a pretty tight standard network policy. Basically, I do my lazy allow all egress, allow incoming from traefik and allow all inside the namespace.

Then I need an ingressroute to push to traefik, which will also cause traefik to request a certificate:

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: keycloak-ingressroute
  namespace: traefik-external
  annotations:
    kubernetes.io/ingress.class: "traefik-external"
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`keycloak.engen.priv.no`)
      kind: Rule
      middlewares:
        - name: protohttps
      services:
        - name: keycloak
          namespace: keycloak
          port: 80
  tls:
    certResolver: letsencrypt

The protohttps is just to enable it to send a header to tell Keycloak that the request really came in via https. The more secure option would be to set up SSL also inside the platform, but as for now, I’m not going to care since this is my home platform, and if someone I don’t trust is logged in to my node, I have bigger problems.

kind: Middleware
metadata:
  creationTimestamp: "2025-04-23T16:00:44Z"
  generation: 1
  name: protohttps
  namespace: traefik-external
  resourceVersion: "13712657"
  uid: 85ba8b9e-49ce-4e9f-b428-eb9896bec229
spec:
  headers:
    customRequestHeaders:
      X-Forwarded-Proto: https

There is already an IPv4 loadbalancer for it, since I can only have one towards the external network on port 443, but I can do IPv6:

apiVersion: v1
kind: Service
metadata:
  annotations:
    external-dns.alpha.kubernetes.io/hostname: keycloak.engen.priv.no
    external-dns.alpha.kubernetes.io/ttl: "300"
    external-dns/external: "true"
    ipchanger.alpha.kubernetes.io/patch: "true"
    metallb.io/ip-allocated-from-pool: public-ipv6-pool
    metallb.universe.tf/address-pool: public-ipv6-pool
    unifi.engen.priv.no/firewall-group: externalweb
  name: traefik-keycloak
  namespace: traefik-external
  externalTrafficPolicy: Local
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv6
  ipFamilyPolicy: SingleStack
  ports:
  - name: web
    port: 80
    protocol: TCP
    targetPort: 80
  - name: websecure
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: traefik-external
  sessionAffinity: None
  type: LoadBalancer

This will create a DNS entry for it with externaldns and a firewall opening through my unifi-network-operator. I also add DNS towards my external IPV4 address:

apiVersion: v1
kind: Service
metadata:
  annotations:
    external-dns.alpha.kubernetes.io/hostname: ...... keycloak.engen.priv.no
    external-dns.alpha.kubernetes.io/ttl: "300"
    external-dns/external: "true"
  creationTimestamp: "2025-03-04T16:27:41Z"
  name: dns-service
  namespace: infrastructure
  resourceVersion: "20318526"
  uid: aa6e0196-f1ac-4217-8e89-b1abe1ac75d3
spec:
  externalName: <externalIP>
  sessionAffinity: None
  type: ExternalName

All correct, after a while I should be able to hit https://keycloak.engen.priv.no/ and log in with admin and the initial password installed by helm. And then, first thing you do is of course to change the password. Keeping this admin user in the master domain around for a while, or even permanently with an extremely secure password as a fallback login, might be a good idea.

Now, we’re about to start with the fun part – configure the single signon service. I’m not going to go into detail about what to click and where, that would be very boring and take for ages, so you’ll have to live with me describing my choices.

I decided to create my own realm – let’s call it MyRealm. I could have lived in the master realm, but online searches hinted that it might be a good idea to create a new one and leave master realm for Keycloak internal things.

Then, I created an admin user for MyRealm, with no rights in the master realm, which I will use for pretty much everything from here on out. This admin user can also double as your day-to-day-user, although the more security-minded might insist not using an admin user for daily use. I decided to go lazy here, and use my same user for both admin tasks and user tasks. It’s convenient, because it’s hard to juggle with two logged in sessions with different usernames.

Then, it’s time to do some planning. Or leave that for later, and just play with the setup, if you want to gain some experience to take better decisions. But you will eventually have to decide how to structure your roles and groups. Start simple and grow later I guess is a sensible approach.

I decided to create a group Everyone, and then a subgroup MyHome and another subgroup Guests. The guests subgroup might come in handy if I want to allow someone to auto-signup, giving them a minimal or even no access rights after singup, allowing me to assign only the rights I want to give them. Maybe I want to give someone access to my Nextcloud as a user, and only that, for example.

Under MyHome, I added basically Admins and Users, as my setup is extremely small. I don’t have that many people in my realm at all.

So I added myself both in Admins and Users, which we’ll fill with content in a little while.

You’ll want to put roles for your services into the respective groups, so that you don’t have to assign manually to everyone. I decided to create a realm role platform-admin which basically gives admin rights to the whole platform (but maybe not services), which I added to Admins.

Now, we’ve come to adding services, which will vary a lot depending on what you have running. Some services support OIDC natively, some support OAUTH2, and some doesn’t support any authentication. We can still put OIDC in the reverse proxy, letting that handle authentication even if the application doesn’t know about it. I might also do that for services that only have password authentication, giving me more security for those.

In OIDC, the things you connect to and protect with SSO is called clients. Every client will have different configuration, different roles, and different sets of properties that Keycloak should send to it. The authentication is handled by Keycloak, while authorization is up to the client, although it might be told about what roles and groups the user have so that it can do authorization based on that.

Traefik (reverse proxy)

I decided to call this client Kubernetes, as it is not only used for Traefik itself in my case.

There is a few options for middlewares to do OIDC or oauth in Traefik. One is oauth2-proxy, but I didn’t really like it a lot, the configuration for the services was a bit cumbersome and not exactly transparent. Another one is traefik-forward-auth but as far as I could see, it didn’t provide any options to do fine-grained authorization, which is an option I wanted.

I ended up downloading a plugin called traefikoidc to test out, and that one fit my needs so I stuck with that one. The thing I liked about it is the way I can send a rich set of information to the backend in headers to allow backends to take decisions on that. And in fact, one application, Kubernetes Dashboard, doesn’t support OIDC in itself but supported a token that traefik-oidc could forward. Another one, Grafana, can be set up to trust headers the proxy inserts, although it also supports OIDC in itself, so here you have options. I decided to go with headers, as I wanted the authentication in the ingressroute anyhow. It also saves the need of selecting OIDC login for the Grafana users, that will be done automatically.

Prometheus doesn’t really support authentication beyond basic authentication and client certificate, but I have actually left it open inside my platform, though of course protected by security policies (firewall rules), and connecting to the cluster IP service when Kubernetes components need to talk to it. For traffic from a browser, I have just added traefikoidc in Traefik, that Prometheus is absolutely unaware of.

For Portainer I am actually doing both, both OIDC in traefik and in Portainer, as they both support auto-login, so it’s not so bad UX-wise.

For WordPress (this blog), I am using one of numerous OIDC plugins that is provided for WordPress, but obviously I also need unauthenticated users, else noone could read my blog, so Traefik doesn’t do any access control.

For Nextcloud, I also keep it open in the ingress, as I want to be able to share externally. Nextcloud supports OIDC, so I have configured it directly there. But you also need to be able to authenticate with long lived tokens etc, so it will be an addition to the other mechanisms that Nextcloud have.

Each of these, if possible, I configure a client admin role, that I add as a subrole to the platform-admin role. That way, I could for example opt to give out fine grained access rights to users with limited specific needs. (And now I do pretend I actually have a lot of users. I don’t. But I might! One day!)

For each of these clients, I will need to configure, among other things:

• claims: What properties to send in tokens towards the backend services. THis is basically username, full name, email address and optionally roles and groups if the application has a need to act on that. This allows the backend application to auto-fill all of these, and especially name and email is highly useful. Some applications do have a need for sending me mails.
• Allowed redirect URLs. This is a security mechanism making sure you don’t leak information to other than indended sites, should they attempt something fishy.

Kubernetes Login

Kubernetes Login itself can also be protected with OIDC, and specifically for K3s, here is how to do it: https://geek-cookbook.funkypenguin.co.nz/kubernetes/oidc-authentication/k3s-keycloak/

In my case, I wanted an admin role and a readonly role. Since kubectl isn’t a web interface, and I run through SSH which doesn’t allow me to open a browser, I can configure kubelogin, the helper tool described, to give me an URL, which I then paste into a browser and do single signon authentication. My login session will poll to see if I am authenticated, and eventually log me in.

It’s of course good to keep a fallback login-mechanism, and the standard-installed client certificate is good to have in this case. But now, it can be protected more, making it only readable by root for example.

Login mechanism and flows.

The other important part of Keycloak is deciding how users log in? Should you require 2 factor authentication? Do you allow passwordless passkeys? Or passkeys only as 2nd factor? All of this can be achieved.

This is where I needed my custom plugin, which allows the 2 factor authentication to be stored and the device to be trusted. This way, I require 2FA, but I’m not evil (or a masochist), so I’ll allow trusting devices. I also allow passwordless passkeys, which makes simple fingerprint authentication working on a mobile phone.

There’s a ton of options, and you’ll probably want to customize the browser login flow to fit your needs. The standard one only requires username and password. There’s quite good documentation for this on Keycloak documentation site although it might take some trial and error.

Admin Cli

This is where admin cli comes in handy, which you can use from the kubernetes pod itself. Here, you can fix things like having locked yourself out. This happened to me on a couple of occations, so it’s good to be prepared for this to happen! You can actually fix most things here, including having lost access to your admin user.

Status? How far did I get?

I actually log in with Keycloak to all my home services, except for home assistant (so far) – and SSH to the node itself, I haven’t done that yet. It can probably be done.

Has it simplified my platform?

As a user, definitely. It now feels a lot more integrated.

As an admin? Well, I love having only one set of user credentials to manage. It finally actually feels like a home platform!