Having set up a number of services, and making sure everyone of them gets their own IPv6 address, there’s a whole lot of DNS records pointing to services running in Kubernetes. Today, I found a gem: external-dns. This service basically monitors my infrastructure for annotations that tells it to create a DNS record for it.

It supports a fair number of DNS services. Earlier, it had built-in support for all of them, but is now adding new services through plugin-ins (webhooks). My DNS provider, Linode, is built-in, which made it sligtly – though not much – easier.

I opted to install it through helm:

$ kubectl create namespace externaldns
$ helm repo add external-dns https://kubernetes-sigs.github.io/external-dns/

Then, I needed to configure a little bit before setting it up. This is usually a bit iterative process for me, especially if it’s things I’m unsure how to to. But in the end, the things I had to configure in values.yaml (the configuration file for the helm installation), was:

policy: sync
env: 
 - name: LINODE_TOKEN
   valueFrom:
     secretKeyRef:
       name: linodetoken
       key: token
provider:
   name: linode

The sync policy means I wante it to both change, delete and update records as I modify my kubernetes services.

To have it manage my Linode DNS, I first had to go into cloud.linode.com and create an API token. The API token needs full reads to DNS, but nothing else.

I store this in a secret:

kubectl create secret generic -n externaldns linodetoken --from-literal token=<APITOKEN>

This creates an opaque secret named linodetoken with one value (key: token), just as specified.

Then, to install external-dns configured for Linode, I simply could say:

helm install -n externaldns external-dns external-dns/external-dns --version 1.15.2

If all goes well, we now have a POD in the externaldns namespace:

$ kubectl get pods -n externaldns 
NAME                            READY   STATUS    RESTARTS   AGE
external-dns-54886c9bb9-65jnw   1/1     Running   0          1m

Having gotten this far, it is time to let it create my DNS records. It’s not happy with managing DNS records it hasn’t created itself. However, to minimize the time your DNS records is unconfigured, it’s best to add all annotations and configurations before deleting the old records from DNS. Per default, external-dns will run once a minute, but there’s other ways to configure it. For me, this was fine.

For IPV6, I only have one real IP address, on the outside of my external router, port forwarding in to my Kubernetes services. Since I don’t know anything about the external IP address in Kubernetes, I have to create a special service for it, an ExternalName service. For services where Kubernetes have the public IP addresses, this isn’t needed, and we’ll handle IPv6 addresses directly.

I decided to handle it in only one service, since I can specify several DNS names in the configuration:

$ cat externalnames.yaml 
kind: Service
apiVersion: v1
metadata:
  name: dns-service
  namespace: infrastructure
  annotations:
    external-dns.alpha.kubernetes.io/hostname:vegard.blog.engen.priv.no,nextcloud.engen.priv.no,portainer.engen.priv.no,....
spec:
  type: ExternalName
  externalName: <my ip address>>

In the annotation, I simply list all the hostnames I need to add. It doesn’t matter what type of service has this ip address, or if there’s one or many. In this case, this record is everything that manages these IPv4 addresses. There might be more elegant ways to do this, but this works for me. Whenever I create a new service accessible externally on IPv4, I just update this record, and external-dns will handle creating the record for me. This is already pretty neat.

Now, officially, I haven’t a static IPv4 address. It hasn’t changed so far, but it might. If it does, I only need to change this record, and all my IPv4 DNS records to my Kubernetes services will be updated within a minute or two. Pretty sure I can even automate it. Yes – I know dynamic dns services exists, but automating this, I have in effect created my own, in-Kubernetes dynamic DNS service. This is even neater.

Having gotten this far, it’s time to tackle IPv6. All my DNS services currently run through MetalLB load balancers, so I can do all of them the same way. Here’s the one for this blog:

$ cat lb-vegard.blog.engen.priv.no.yaml 
apiVersion: v1
kind: Service
metadata:
    name: traefik-vegardblog
    namespace: traefik
    annotations:
      metallb.universe.tf/address-pool: public-ipv6-pool
      external-dns.alpha.kubernetes.io/hostname: vegard.blog.engen.priv.no

spec:
  externalTrafficPolicy: Local
  type: LoadBalancer
  ipFamilyPolicy: SingleStack
  ipFamilies:
    - IPv6
  loadBalancerIP: 2a01:799:393:f10b:2::1
  ports:
    - name: web
      port: 80
    - name: websecure
      port: 443
  selector:
    app: traefik

Now, my IPv6 prefix is hopefully a little bit more static than my IPV4 address. But if it changes, I anyway have to update my load balancer service, and when doing so, it automatically updates my DNS record.

And if it’s dynamic? Pretty sure I can automate updating these services.

So, how would I update these? Well, let’s go and change my portainer.engen.priv.no load balancer (there’s TTL on DNS, so I don’t want to change entries used by other than me…). The portainer is currently defined as this:

$ cat lb-portainer.yaml 
apiVersion: v1
kind: Service
metadata:
    name: traefik-portainer
    namespace: traefik
    annotations:
      metallb.universe.tf/address-pool: public-ipv6-pool
      external-dns.alpha.kubernetes.io/hostname: portainer.engen.priv.no

spec:
  externalTrafficPolicy: Local
  type: LoadBalancer
  ipFamilyPolicy: SingleStack
  ipFamilies:
    - IPv6
  loadBalancerIP: 2a01:799:393:f10b:2::15
  ports:
    - name: web
      port: 80
    - name: websecure
      port: 443
  selector:
    app: traefik

To change this ip-adress:

$ kubectl patch -n traefik service traefik-portainer -p '{"spec": {"loadBalancerIP": "2a01:799:393:f10b:2::14"}}'
service/traefik-portainer patched

…and then comes external-dns and does it’s magic:

$ kubectl logs -n externaldns external-dns-54886c9bb9-65jnw
.....
time="2025-03-04T18:54:43Z" level=warning msg="Creating New Target" dnsName=portainer.engen.priv.no recordType=AAAA target="2a01:799:393:f10b:2::14" zoneID=43496 zoneName=engen.priv.no
time="2025-03-04T18:54:43Z" level=warning msg="Deleting Target" dnsName=portainer.engen.priv.no recordType=AAAA target="2a01:799:393:f10b:2::15" zoneID=43496 zoneName=engen.priv.no
time="2025-03-04T18:54:43Z" level=warning msg="Updating Existing Target" dnsName=aaaa-portainer.engen.priv.no recordType=TXT target="\"heritage=external-dns,external-dns/owner=default,external-dns/resource=service/traefik/traefik-portainer\"" zoneID=43496 zoneName=engen.priv.no
time="2025-03-04T18:54:43Z" level=info msg="Creating record." action=Create record=portainer type=AAAA zoneID=43496 zoneName=engen.priv.no
time="2025-03-04T18:54:43Z" level=info msg="Deleting record." action=Delete record=portainer type=AAAA zoneID=43496 zoneName=engen.priv.no
time="2025-03-04T18:54:43Z" level=info msg="Updating record." action=Update record=aaaa-portainer type=TXT zoneID=43496 zoneName=engen.priv.no

And there we go!

% host -t AAAA portainer.engen.priv.no                                 
portainer.engen.priv.no has IPv6 address 2a01:799:393:f10b:2::14

Now, of course there’s a firewall too. Maybe someone will write an external-fw Kubernetes provider one day….in the meantime, I have been lazy and just opened up port 80 and 443 towards my loadbalancer IPV6 range. Oh, right – if your IPV6 network change, remember to update that!

I found external-dns pretty neat, and it allows me to manage the DNS records close to the services they belong, making it much more likely that I’ll clean up, or remember to change them should I change anything. I promise, I’ll blog about it if I get around to automate updating my firewall too! I’ve already got a pretty good idea how to do it quick and dirty, at least.

Last, but not least: Remember that as you put these things into Kubernetes cluster, it becomes more important to think about the security of it, because suddenly it has access to update so much more. Make sure you secure your node properly too!