As promised in my last blog post, here is part two of my BGP series. I’ve decided to split it into two, one covering the VPN, and another one covering the BGP end of it. So this one isn’t actually about BGP, it is about IPSEC.
I am running services at home – but that comes with some caveats. I don’t have a 24/7 on-call team with enough people to cater for vacations, so when I’m out travelling, things can go down. For this reason mainly, I run things like my main email (I’ve still not sold my soul to Gmail or Outlook.com) in a VPS. This means I have some level of support from the provider, it will be restarted if it goes down and if there’s problems with the underlying service, they will fix it.
It being in the cloud doesn’t mean I can’t integrate it well with my on-prem services, though. VPN has existed for ages, and me wanting to play more with BGP gave me just the right incentive to set it up.
There’s plenty of VPN protocols and software out there. I wanted something that doesn’t necessarily do any NAT (which you by now should know I hate), and it should be well integrated into the systems I run.
I figured I want to pretend that my VPS is a full-blown site. It does, in fact, have a /56 IPV6 network dedicated (totally overkill, it’s only one server…), so I have something to play with. So I decided I wanted to use some of the built-in site-to-site VPN mechanisms in my Unifi router. That means openbgp or ipsec for now, although for example Wireguard would probably be possible to use also.
Ipsec is built into the Linux kernel, and I have a hang to selecting network mechanisms fully built into the OS without extra services running. So ipsec was my selection. It’s probably not the easiest choice, but I’m not here for easy, I am here for the play value.
I spent a lot of time trying to convince Unifi to do ipv6 over the IPSEC tunnel, but I have yet to find a way (and neither have I seen others who have found a way), so I concluded it wasn’t possible for now. So I’ll have to go with an IPv4 ipsec tunnel and solve IPv6 separately. There’s tunnel mechanisms for that, and as long as I am my own tunnel provider, I am fine – not happy, but fine – with that.
There’s lots of decicions to do once you start using IPSec. There’s encryption protocols, authentication mechanisms, IP addressing and a whole lot more. I will not go into the details of everything, but I will try to provide details for my choices.
As I said, IPSEC is built into the kernel. There’s still user-space tools for administration and setup, though, so I need to select software suite. Since I run Debian on my VPS, I decided to pick Libreswan, since that is the route Debian seems to go. There’s plenty of tutorials about configuring IPSEC. Also, since this was mainly a BGP exercise, I’ve simplified it by doing just a shared secret for the authentication mechanism. IPSEC has a lot more advanced and arguably more secure ways to authenticate the end systems than that.
But enough talk, on with the configuration. In IPSEC, we’re talking about left side and right side a lot. Left side is the local side, Right side is the remote side, so in my opinion it’d be just as easy to call it local and remote. Since I want to do dynamic routing with BGP, I’ll just allow everything to be routed over the tunnel, and just not install routes for it in the IPSEC setup. I’ll also configure a point-to-point-ipsec tunnel, where both endpoints have an ip address inside the same subnet. This makes things a little bit easier.
On Unifi, you can either use IP addresses for IDs, or you can create some labels. I decided to go with the labels route, as my home IP address is in theory dynamic (but it has yet to change). Just one less place to change things should it change. I decided to call my VPS for debian and my home for unifi.
This is the configuration I have on the Unifi side of things. My home network external IP address blacked out, as it’s not possible to actually change it in the interface, it is automatically filled in.
The connection parameters are IPSEC specific, and I’ll not really cover them here. IKEv2 is the newest protocol for setting up the security association for IPSEC, and the rest are pretty decent parameters – probably a bit on the low end when it comes to key length, but it’ll do for now. I chose to disable PFS, as it created issues earlier, but it’s more secure should you get it to work with keeping it.
I have chosen 192.168.229.0/24 for my tunnel network. I cold have gone smaller, probably, but these are private ip addresses and I have no shortage of those. On the Unifi end, 192.168.229.1 is the IP address. As it’s a point-to-point, everything you send to it is in effect sent to the other side, so I need not worry about the remote end in this setup.
On the VPS end, I’ll need to configure this with configuration files. In the installation, /etc/ipsec.conf is configured to read all configurations that it finds in /etc/ipsec.d, so I’ll make an /etc/ipsec.d/unifi.conf for the connection to my Unifi:
conn unifi-dynamic
authby=secret
auto=start
left=vps.engen.priv.no
leftid=@debian
right=homeip.engen.priv.no
rightid=@unifi
leftsubnet=0.0.0.0/0 # Route all traffic through the tunnel
rightsubnet=0.0.0.0/0 # Same on the remote side
ike=aes256-sha256;modp2048
phase2alg=aes256-sha256
ikelifetime=28800s
salifetime=3600s
type=tunnel
pfs=no
aggressive=no
dpddelay=30
dpdtimeout=120
dpdaction=restart
vti-interface=vti0
mark=0x02000000/0xfe000000
vti-routing=no
leftvti=192.168.229.2/24
This all matches the settings on Unifi. I have configured 192.168.229.2 as the IP address for the Debian side of the tunnel. vti-routing=no tells ipsec not to set up towards the right subnet, we’ll handle that with BGP. Routing 0.0.0.0/0 would be a bad idea anyhow. The 0.0.0.0/0’s and vti-routing=no corresponds to the «dynamic» checkbox in the Unifi end of things.
THen, we’ll need to configure the key, in /etc/ipsec.d/unifi.secrets:
@debian @unifi : PSK "********************************"
The secret here should of course be the same as on Unifi.
You’ll also need to ensure that IPSEC traffic flows through your firewall on your VPS, should you have one:
iptables -A INPUT -p esp -j ACCEPT
iptables -A INPUT -p udp -m udp --dport 4500 -j ACCEPT
iptables -A INPUT -p udp -m udp --dport 500 -j ACCEPT
On Unifi site of things, this will be handled automatically.
Now, we’re ready to bring up the tunnel, and if all is well you should be able to ping the Unifi end from you Debian machine:
$ ping 192.168.229.1
PING 192.168.229.1 (192.168.229.1) 56(84) bytes of data.
64 bytes from 192.168.229.1: icmp_seq=1 ttl=64 time=27.6 ms
64 bytes from 192.168.229.1: icmp_seq=2 ttl=64 time=27.6 ms
64 bytes from 192.168.229.1: icmp_seq=3 ttl=64 time=27.9 ms
Yay! Traffic!
As mentioned, for IPv6 I have resorted to a tunnel (GRE) for now, which I have configured in /etc/network/interfaces on the debian end:
auto tun-gre
iface tun-gre inet6 static
address fd46:c709:32c6:3::1
netmask 64
pre-up ip tunnel add tun-gre mode gre remote 192.168.229.1 local 192.168.229.2 ttl 255
I have chosen a ULA prefix here, because I’m dead sure I’m never going to route this subnet in itself on the Internet, so it’s one less place to change it if my ISP changes my IPv6 range.
On the Unfi end, I need a small shell-script. To make this persistent, I need to resort to tricks like unifios-utilities. You can of course run it on another server than your router, but running it on the router feels a whole lot more logical, so tricks it was.
The script:
#!/bin/sh
ip tunnel add tun-gre mode gre remote 192.168.229.2 local 192.168.229.1 ttl 255
ip link set tun-gre up
ip addr add fd46:c709:32c6:3::2/64 dev tun-gre
If I can now can ping fd46:c709:32c6:3::2 from the VPS, I have done everything correctly:
$ ping fd46:c709:32c6:3::2
PING fd46:c709:32c6:3::2(fd46:c709:32c6:3::2) 56 data bytes
64 bytes from fd46:c709:32c6:3::2: icmp_seq=1 ttl=64 time=28.2 ms
64 bytes from fd46:c709:32c6:3::2: icmp_seq=2 ttl=64 time=27.9 ms
64 bytes from fd46:c709:32c6:3::2: icmp_seq=3 ttl=64 time=27.6 ms
…and we’ve got IPv6!
Now, we’re all set for routing with BGP in my next post.