Tailscale: Connect to Postgres privately

August 17, 2021

Categories: Technical Tags: linux postgres networking

I have bought into the Private Network philosophy a while ago. But automating deployment in practice is still kind of a pain. With wireguard now baked into the kernel, the friction is greatly reduced. However things like key management still remains tedious. All considered, this space is still new, and higher level solutions are still fledgling. The make or break about competing with existing things like IPsec would be decided by polish, and today I think Tailscale has an edge.

Fair warning though, Tailscale uses userspace wireguard, which necessarily can't match kernel module in terms of throughput. I am trying out the free tier which offers 1 admin and 20 devices, other services might have better offerings.

Now, why Tailscale? Roughly, it's very easy to deploy anywhere and get going (golang static binaries). You can do social login, and dashboard is web based. Key issuing and revoking is easy, and can be automated over their API. Works over IPv6, otherwise can employ complex NAT punching techniques (STUN/TURN) to achieve flat network with p2p capacity, so port opening is not a necessity. However, as last resort it can relay traffic through their server so there is some guarantee of it always working (and traffic is end-to-end encrypted). There is a magicDNS feature (as of now, still in beta) that allows you to use built-in resolver without dependency. Can do complex routing and access control (though not in free tier). Has features like service discovery or file sharing and so on. Works on FreeBSD and iOS/android (though support is community/beta).

It's packaged in my distro, for different flavours of server distros they maintain native package. The software has two components, a daemon and a client. The daemon is best initiated with your init/supervisor, the RHEL package already comes with systemd unit files. And then the client is used to connect/disconnect or otherwise see network status. Creating a node out of a headless server automatically is done with auth keys, which can be different types such as one-off, reusable, ephemeral. And that's all it takes to have your network up and running. New nodes are given both an IPv6 and an IPv4 address (in CGNAT subnet e.g 100.64.0.0/10 perhaps so as not to conflict with any other subnets you may have).

I ran into a minor problem where I can ping between machines, but an HTTP server running on the RHEL server couldn't be reached from home. Doing tcpdump on the tailscale0 interface on the server showed packet reaching it, so I surmised it was the doing of firewalld. This fixed it:

sudo firewall-cmd --permanent --zone=trusted --change-interface=tailscale0
sudo firewall-cmd --permanent --zone=trusted --add-port=5432/tcp
sudo firewall-cmd --reload

Now I can have local services such as Postgres listening on some port only on tailscale0 interface, and access it from another machine over this private network securely, nifty. Tailscale assigned IP addresses are supposed to remain device unique, no matter how much you roam, this guarantee is useful for scripting. However even better can be using built-in DNS resolution feature (which automatically appends necessary nameserver to /etc/resolv.conf). So if the server is called server in tailscale (deduced from its hostname, but can be changed), and if you put this is in postgresql.conf:

  listen_addresses = 'localhost,server'

And this in pg_hba.conf:

  host    all             all             100.64.0.0/10           trust
  host    replication     all             100.64.0.0/10           trust

Then subsequently from home you can connect to remote server postgres instance securely like:

psql -h server -U postgres db

And voila,

SET
psql (13.4)
Type "help" for help.

[email protected]:db
= #

Networking doesn't get simpler than that. This simplifies backup and replication plans.