Rewrite PIA VPN as multi-container bridge architecture

Replace the single VPN container (veth pair, host-side auth scripts) with a multi-container setup on a shared bridge network: - Dedicated VPN container handles all PIA auth, WireGuard config, NAT, and optional port forwarding DNAT - Service containers default-route through VPN container (leak-proof by topology) - Host runs tinyproxy on bridge for PIA API bootstrap before WG is up - WG interface is still created in host netns and moved into VPN container namespace - Monthly renewal to ensure that connection stays up (PIA allows connections to last up to 2 months) - Drop OpenVPN support entirely
2026-02-22 23:10:25 -08:00
parent 412dd12b5a
commit f2f5761c83
16 changed files with 1049 additions and 890 deletions
--- a/common/network/pia-vpn/README.md
+++ b/common/network/pia-vpn/README.md
@@ -0,0 +1,89 @@
+# PIA VPN Multi-Container Module
+
+Routes service containers through a PIA WireGuard VPN using a shared bridge network.
+
+## Architecture
+
+```
+                  internet
+                     │
+              ┌──────┴──────┐
+              │    Host      │
+              │  tinyproxy   │  ← PIA API bootstrap proxy
+              │  10.100.0.1  │
+              └──────┬───────┘
+                     │ br-vpn (no IPMasquerade)
+        ┌────────────┼──────────────┐
+        │            │              │
+ ┌──────┴──────┐ ┌───┴────┐  ┌─────┴──────┐
+ │ VPN ctr     │ │ servarr│  │transmission│
+ │ 10.100.0.2  │ │  .11   │  │   .10      │
+ │ piaw (WG)   │ │        │  │            │
+ │ gateway+NAT │ └────────┘  └────────────┘
+ └─────────────┘
+```
+
+- **Host** creates the WG interface (encrypted UDP stays in host netns) and runs tinyproxy on the bridge so the VPN container can bootstrap PIA auth before WG is up.
+- **VPN container** authenticates with PIA via the proxy, configures WG, sets up NAT (masquerade bridge→WG) and optional port forwarding DNAT.
+- **Service containers** default-route through the VPN container. No WG interface = no internet if VPN is down = leak-proof by topology.
+- **Host** reaches containers directly on the bridge for nginx reverse proxying.
+
+## Key design decisions
+
+- **Bridge, not veth pairs**: All containers share one bridge (`br-vpn`), so the VPN container can act as a single gateway. The host does NOT masquerade bridge traffic — only the VPN container does (through WG).
+- **Port forwarding is implicit**: If any container sets `receiveForwardedPort`, the VPN container automatically handles PIA port forwarding and DNAT. No separate toggle needed.
+- **DNS through WG**: Service containers use the VPN container as their DNS server. The VPN container runs `systemd-resolved` listening on its bridge IP, forwarding queries through the WG tunnel.
+- **Monthly renewal**: `pia-vpn-setup` uses `Type=simple` + `Restart=always` + `RuntimeMaxSec=30d` to periodically re-authenticate with PIA and get a fresh port forwarding signature (signatures expire after ~2 months). Service containers are unaffected during renewal.
+
+## Files
+
+| File | Purpose |
+|---|---|
+| `default.nix` | Options, bridge, tinyproxy, host firewall, WG interface creation, assertions |
+| `vpn-container.nix` | VPN container: PIA auth, WG config, NAT, DNAT, port refresh timer |
+| `service-container.nix` | Generates service containers with static IP and gateway→VPN |
+| `scripts.nix` | Bash function library for PIA API calls and WG configuration |
+| `ca.rsa.4096.crt` | PIA CA certificate for API TLS verification |
+
+## Usage
+
+```nix
+pia-vpn = {
+  enable = true;
+  serverLocation = "swiss";
+
+  containers.my-service = {
+    ip = "10.100.0.10";
+    mounts."/data".hostPath = "/data";
+    config = { services.my-app.enable = true; };
+
+    # Optional: receive PIA's forwarded port (at most one container)
+    receiveForwardedPort = { port = 8080; protocol = "both"; };
+    onPortForwarded = ''
+      echo "PIA assigned port $PORT, forwarding to $TARGET_IP:8080"
+    '';
+  };
+};
+```
+
+## Debugging
+
+```bash
+# Check VPN container status
+machinectl shell pia-vpn
+systemctl status pia-vpn-setup
+journalctl -u pia-vpn-setup
+
+# Verify WG tunnel
+wg show
+
+# Check NAT/DNAT rules
+iptables -t nat -L -v
+iptables -L FORWARD -v
+
+# From a service container — verify VPN routing
+curl ifconfig.me
+
+# Port refresh logs
+journalctl -u pia-vpn-port-refresh
+```
--- a/common/network/pia-vpn/ca.rsa.4096.crt
+++ b/common/network/pia-vpn/ca.rsa.4096.crt
@@ -0,0 +1,43 @@
+-----BEGIN CERTIFICATE-----
+MIIHqzCCBZOgAwIBAgIJAJ0u+vODZJntMA0GCSqGSIb3DQEBDQUAMIHoMQswCQYD
+VQQGEwJVUzELMAkGA1UECBMCQ0ExEzARBgNVBAcTCkxvc0FuZ2VsZXMxIDAeBgNV
+BAoTF1ByaXZhdGUgSW50ZXJuZXQgQWNjZXNzMSAwHgYDVQQLExdQcml2YXRlIElu
+dGVybmV0IEFjY2VzczEgMB4GA1UEAxMXUHJpdmF0ZSBJbnRlcm5ldCBBY2Nlc3Mx
+IDAeBgNVBCkTF1ByaXZhdGUgSW50ZXJuZXQgQWNjZXNzMS8wLQYJKoZIhvcNAQkB
+FiBzZWN1cmVAcHJpdmF0ZWludGVybmV0YWNjZXNzLmNvbTAeFw0xNDA0MTcxNzQw
+MzNaFw0zNDA0MTIxNzQwMzNaMIHoMQswCQYDVQQGEwJVUzELMAkGA1UECBMCQ0Ex
+EzARBgNVBAcTCkxvc0FuZ2VsZXMxIDAeBgNVBAoTF1ByaXZhdGUgSW50ZXJuZXQg
+QWNjZXNzMSAwHgYDVQQLExdQcml2YXRlIEludGVybmV0IEFjY2VzczEgMB4GA1UE
+AxMXUHJpdmF0ZSBJbnRlcm5ldCBBY2Nlc3MxIDAeBgNVBCkTF1ByaXZhdGUgSW50
+ZXJuZXQgQWNjZXNzMS8wLQYJKoZIhvcNAQkBFiBzZWN1cmVAcHJpdmF0ZWludGVy
+bmV0YWNjZXNzLmNvbTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBALVk
+hjumaqBbL8aSgj6xbX1QPTfTd1qHsAZd2B97m8Vw31c/2yQgZNf5qZY0+jOIHULN
+De4R9TIvyBEbvnAg/OkPw8n/+ScgYOeH876VUXzjLDBnDb8DLr/+w9oVsuDeFJ9K
+V2UFM1OYX0SnkHnrYAN2QLF98ESK4NCSU01h5zkcgmQ+qKSfA9Ny0/UpsKPBFqsQ
+25NvjDWFhCpeqCHKUJ4Be27CDbSl7lAkBuHMPHJs8f8xPgAbHRXZOxVCpayZ2SND
+fCwsnGWpWFoMGvdMbygngCn6jA/W1VSFOlRlfLuuGe7QFfDwA0jaLCxuWt/BgZyl
+p7tAzYKR8lnWmtUCPm4+BtjyVDYtDCiGBD9Z4P13RFWvJHw5aapx/5W/CuvVyI7p
+Kwvc2IT+KPxCUhH1XI8ca5RN3C9NoPJJf6qpg4g0rJH3aaWkoMRrYvQ+5PXXYUzj
+tRHImghRGd/ydERYoAZXuGSbPkm9Y/p2X8unLcW+F0xpJD98+ZI+tzSsI99Zs5wi
+jSUGYr9/j18KHFTMQ8n+1jauc5bCCegN27dPeKXNSZ5riXFL2XX6BkY68y58UaNz
+meGMiUL9BOV1iV+PMb7B7PYs7oFLjAhh0EdyvfHkrh/ZV9BEhtFa7yXp8XR0J6vz
+1YV9R6DYJmLjOEbhU8N0gc3tZm4Qz39lIIG6w3FDAgMBAAGjggFUMIIBUDAdBgNV
+HQ4EFgQUrsRtyWJftjpdRM0+925Y6Cl08SUwggEfBgNVHSMEggEWMIIBEoAUrsRt
+yWJftjpdRM0+925Y6Cl08SWhge6kgeswgegxCzAJBgNVBAYTAlVTMQswCQYDVQQI
+EwJDQTETMBEGA1UEBxMKTG9zQW5nZWxlczEgMB4GA1UEChMXUHJpdmF0ZSBJbnRl
+cm5ldCBBY2Nlc3MxIDAeBgNVBAsTF1ByaXZhdGUgSW50ZXJuZXQgQWNjZXNzMSAw
+HgYDVQQDExdQcml2YXRlIEludGVybmV0IEFjY2VzczEgMB4GA1UEKRMXUHJpdmF0
+ZSBJbnRlcm5ldCBBY2Nlc3MxLzAtBgkqhkiG9w0BCQEWIHNlY3VyZUBwcml2YXRl
+aW50ZXJuZXRhY2Nlc3MuY29tggkAnS7684Nkme0wDAYDVR0TBAUwAwEB/zANBgkq
+hkiG9w0BAQ0FAAOCAgEAJsfhsPk3r8kLXLxY+v+vHzbr4ufNtqnL9/1Uuf8NrsCt
+pXAoyZ0YqfbkWx3NHTZ7OE9ZRhdMP/RqHQE1p4N4Sa1nZKhTKasV6KhHDqSCt/dv
+Em89xWm2MVA7nyzQxVlHa9AkcBaemcXEiyT19XdpiXOP4Vhs+J1R5m8zQOxZlV1G
+tF9vsXmJqWZpOVPmZ8f35BCsYPvv4yMewnrtAC8PFEK/bOPeYcKN50bol22QYaZu
+LfpkHfNiFTnfMh8sl/ablPyNY7DUNiP5DRcMdIwmfGQxR5WEQoHL3yPJ42LkB5zs
+6jIm26DGNXfwura/mi105+ENH1CaROtRYwkiHb08U6qLXXJz80mWJkT90nr8Asj3
+5xN2cUppg74nG3YVav/38P48T56hG1NHbYF5uOCske19F6wi9maUoto/3vEr0rnX
+JUp2KODmKdvBI7co245lHBABWikk8VfejQSlCtDBXn644ZMtAdoxKNfR2WTFVEwJ
+iyd1Fzx0yujuiXDROLhISLQDRjVVAvawrAtLZWYK31bY7KlezPlQnl/D9Asxe85l
+8jO5+0LdJ6VyOs/Hd4w52alDW/MFySDZSfQHMTIc30hLBJ8OnCEIvluVQQ2UQvoW
+no177N9L2Y+M9TcTA62ZyMXShHQGeh20rb4kK8f+iFX8NxtdHVSkxMEFSfDDyQ=
+-----END CERTIFICATE-----
--- a/common/network/pia-vpn/default.nix
+++ b/common/network/pia-vpn/default.nix
@@ -0,0 +1,279 @@
+{ config, lib, pkgs, ... }:
+
+# PIA VPN multi-container module.
+#
+# Architecture:
+#   Host creates WG interface, runs tinyproxy on bridge for PIA API bootstrap.
+#   VPN container does all PIA logic via proxy, configures WG, masquerades bridge→piaw.
+#   Service containers default route → VPN container (leak-proof by topology).
+#
+# Reference: https://www.wireguard.com/netns/#ordinary-containerization
+
+with lib;
+
+let
+  cfg = config.pia-vpn;
+
+  # Derive prefix length from subnet CIDR (e.g. "10.100.0.0/24" → "24")
+  subnetPrefixLen = last (splitString "/" cfg.subnet);
+
+  containerSubmodule = types.submodule ({ name, ... }: {
+    options = {
+      ip = mkOption {
+        type = types.str;
+        description = "Static IP address for this container on the VPN bridge";
+      };
+
+      config = mkOption {
+        type = types.anything;
+        default = { };
+        description = "NixOS configuration for this container";
+      };
+
+      mounts = mkOption {
+        type = types.attrsOf (types.submodule {
+          options = {
+            hostPath = mkOption {
+              type = types.str;
+              description = "Path on the host to bind mount";
+            };
+            isReadOnly = mkOption {
+              type = types.bool;
+              default = false;
+              description = "Whether the mount is read-only";
+            };
+          };
+        });
+        default = { };
+        description = "Bind mounts for the container";
+      };
+
+      receiveForwardedPort = mkOption {
+        type = types.nullOr (types.submodule {
+          options = {
+            port = mkOption {
+              type = types.nullOr types.port;
+              default = null;
+              description = ''
+                Target port to forward to. If null, forwards to the same PIA-assigned port.
+                PIA-assigned ports below 10000 are rejected to avoid accidentally
+                forwarding traffic to other services.
+              '';
+            };
+            protocol = mkOption {
+              type = types.enum [ "tcp" "udp" "both" ];
+              default = "both";
+              description = "Protocol(s) to forward";
+            };
+          };
+        });
+        default = null;
+        description = "Port forwarding configuration. At most one container may set this.";
+      };
+
+      onPortForwarded = mkOption {
+        type = types.nullOr types.lines;
+        default = null;
+        description = ''
+          Optional script run in the VPN container after port forwarding is established.
+          Available environment variables: $PORT (PIA-assigned port), $TARGET_IP (this container's IP).
+        '';
+      };
+    };
+  });
+
+  # NOTE: All derivations of cfg.containers are kept INSIDE config = mkIf ... { }
+  # to avoid infinite recursion. The module system's pushDownProperties eagerly
+  # evaluates let bindings and mkMerge contents, so any top-level let binding
+  # that touches cfg.containers would force config evaluation during structure
+  # discovery, creating a cycle.
+in
+{
+  imports = [
+    ./vpn-container.nix
+    ./service-container.nix
+  ];
+
+  options.pia-vpn = {
+    enable = mkEnableOption "PIA VPN multi-container setup";
+
+    serverLocation = mkOption {
+      type = types.str;
+      default = "swiss";
+      description = "PIA server region ID";
+    };
+
+    interfaceName = mkOption {
+      type = types.str;
+      default = "piaw";
+      description = "WireGuard interface name";
+    };
+
+    wireguardListenPort = mkOption {
+      type = types.port;
+      default = 51820;
+      description = "WireGuard listen port";
+    };
+
+    bridgeName = mkOption {
+      type = types.str;
+      default = "br-vpn";
+      description = "Bridge interface name for VPN containers";
+    };
+
+    subnet = mkOption {
+      type = types.str;
+      default = "10.100.0.0/24";
+      description = "Subnet CIDR for VPN bridge network";
+    };
+
+    hostAddress = mkOption {
+      type = types.str;
+      default = "10.100.0.1";
+      description = "Host IP on the VPN bridge";
+    };
+
+    vpnAddress = mkOption {
+      type = types.str;
+      default = "10.100.0.2";
+      description = "VPN container IP on the bridge";
+    };
+
+    proxyPort = mkOption {
+      type = types.port;
+      default = 8888;
+      description = "Tinyproxy port for PIA API bootstrap";
+    };
+
+    containers = mkOption {
+      type = types.attrsOf containerSubmodule;
+      default = { };
+      description = "Service containers that route through the VPN";
+    };
+
+    # Subnet prefix length derived from cfg.subnet (exposed for other submodules)
+    subnetPrefixLen = mkOption {
+      type = types.str;
+      default = subnetPrefixLen;
+      description = "Prefix length derived from subnet CIDR";
+      readOnly = true;
+    };
+
+    # Shared host entries for all containers (host + VPN + service containers)
+    containerHosts = mkOption {
+      type = types.attrsOf (types.listOf types.str);
+      internal = true;
+      readOnly = true;
+    };
+  };
+
+  config = mkIf cfg.enable {
+    assertions =
+      let
+        forwardingContainers = filterAttrs (_: c: c.receiveForwardedPort != null) cfg.containers;
+        containerIPs = mapAttrsToList (_: c: c.ip) cfg.containers;
+      in
+      [
+        {
+          assertion = length (attrNames forwardingContainers) <= 1;
+          message = "At most one pia-vpn container may set receiveForwardedPort";
+        }
+        {
+          assertion = length containerIPs == length (unique containerIPs);
+          message = "pia-vpn container IPs must be unique";
+        }
+      ];
+
+    # Enable systemd-networkd for bridge management
+    systemd.network.enable = true;
+
+    # TODO: re-enable once primary networking uses networkd
+    systemd.network.wait-online.enable = false;
+
+    # Tell NetworkManager to ignore VPN bridge and container interfaces
+    networking.networkmanager.unmanaged = mkIf config.networking.networkmanager.enable [
+      "interface-name:${cfg.bridgeName}"
+      "interface-name:ve-*"
+    ];
+
+    # Bridge network device
+    systemd.network.netdevs."20-${cfg.bridgeName}".netdevConfig = {
+      Kind = "bridge";
+      Name = cfg.bridgeName;
+    };
+
+    # Bridge network configuration — NO IPMasquerade (host must NOT be gateway)
+    systemd.network.networks."20-${cfg.bridgeName}" = {
+      matchConfig.Name = cfg.bridgeName;
+      networkConfig = {
+        Address = "${cfg.hostAddress}/${cfg.subnetPrefixLen}";
+        DHCPServer = false;
+      };
+      linkConfig.RequiredForOnline = "no";
+    };
+
+    # Allow wireguard traffic through rpfilter
+    networking.firewall.checkReversePath = "loose";
+
+    # Block bridge → outside forwarding (prevents host from being a gateway for containers)
+    networking.firewall.extraForwardRules = ''
+      iifname "${cfg.bridgeName}" oifname != "${cfg.bridgeName}" drop
+    '';
+
+    # Allow tinyproxy from bridge (tinyproxy itself restricts to VPN container IP)
+    networking.firewall.interfaces.${cfg.bridgeName}.allowedTCPPorts = [ cfg.proxyPort ];
+
+    # Tinyproxy — runs on bridge IP so VPN container can bootstrap PIA auth
+    services.tinyproxy = {
+      enable = true;
+      settings = {
+        Listen = cfg.hostAddress;
+        Port = cfg.proxyPort;
+      };
+    };
+    systemd.services.tinyproxy.before = [ "container@pia-vpn.service" ];
+
+    # WireGuard interface creation (host-side oneshot)
+    # Creates the interface in the host namespace so encrypted UDP stays in host netns.
+    # The container takes ownership of the interface on startup via `interfaces = [ ... ]`.
+    systemd.services.pia-vpn-wg-create = {
+      description = "Create PIA VPN WireGuard interface";
+
+      before = [ "container@pia-vpn.service" ];
+      requiredBy = [ "container@pia-vpn.service" ];
+      partOf = [ "container@pia-vpn.service" ];
+      wantedBy = [ "multi-user.target" ];
+
+      path = with pkgs; [ iproute2 ];
+
+      serviceConfig = {
+        Type = "oneshot";
+        RemainAfterExit = true;
+      };
+
+      script = ''
+        [[ -z $(ip link show dev ${cfg.interfaceName} 2>/dev/null) ]] || exit 0
+        ip link add ${cfg.interfaceName} type wireguard
+      '';
+
+      preStop = ''
+        ip link del ${cfg.interfaceName} 2>/dev/null || true
+      '';
+    };
+
+    # Host entries for container hostnames — NixOS only auto-creates these for
+    # hostAddress/localAddress containers, not hostBridge. Use the standard
+    # {name}.containers convention.
+    pia-vpn.containerHosts =
+      { ${cfg.vpnAddress} = [ "pia-vpn.containers" ]; }
+      // mapAttrs' (name: ctr: nameValuePair ctr.ip [ "${name}.containers" ]) cfg.containers;
+
+    networking.hosts = cfg.containerHosts;
+
+    # PIA login secret
+    age.secrets."pia-login.conf".file = ../../../secrets/pia-login.age;
+
+    # IP forwarding needed for bridge traffic between containers
+    networking.ip_forward = true;
+  };
+}
--- a/common/network/pia-vpn/scripts.nix
+++ b/common/network/pia-vpn/scripts.nix
@@ -0,0 +1,209 @@
+let
+  caPath = ./ca.rsa.4096.crt;
+in
+
+# Bash function library for PIA VPN WireGuard operations.
+  # All PIA API calls accept an optional $proxy variable:
+  #   proxy="http://10.100.0.1:8888" fetchPIAToken
+  # When $proxy is set, curl uses --proxy "$proxy"; otherwise direct connection.
+
+  # Reference materials:
+  #   https://serverlist.piaservers.net/vpninfo/servers/v6
+  #   https://github.com/pia-foss/manual-connections
+  #   https://github.com/thrnz/docker-wireguard-pia/blob/master/extra/wg-gen.sh
+  #   https://www.wireguard.com/netns/#ordinary-containerization
+
+{
+  scriptCommon = ''
+    proxy_args() {
+      if [[ -n "''${proxy:-}" ]]; then
+        echo "--proxy $proxy"
+      fi
+    }
+
+    fetchPIAToken() {
+      local PIA_USER PIA_PASS resp
+      echo "Reading PIA credentials..."
+      PIA_USER=$(sed '1q;d' /run/agenix/pia-login.conf)
+      PIA_PASS=$(sed '2q;d' /run/agenix/pia-login.conf)
+      echo "Requesting PIA authentication token..."
+      resp=$(curl -s $(proxy_args) -u "$PIA_USER:$PIA_PASS" \
+        "https://www.privateinternetaccess.com/gtoken/generateToken")
+      PIA_TOKEN=$(echo "$resp" | jq -r '.token')
+      if [[ -z "$PIA_TOKEN" || "$PIA_TOKEN" == "null" ]]; then
+        echo "ERROR: Failed to fetch PIA token: $resp" >&2
+        return 1
+      fi
+      echo "PIA token acquired"
+    }
+
+    choosePIAServer() {
+      local serverLocation=$1
+      local servers servers_json totalservers serverindex
+      servers=$(mktemp)
+      servers_json=$(mktemp)
+      echo "Fetching PIA server list..."
+      curl -s $(proxy_args) \
+        "https://serverlist.piaservers.net/vpninfo/servers/v6" > "$servers"
+      head -n 1 "$servers" | tr -d '\n' > "$servers_json"
+
+      totalservers=$(jq -r \
+        '.regions | .[] | select(.id=="'"$serverLocation"'") | .servers.wg | length' \
+        "$servers_json")
+      if ! [[ "$totalservers" =~ ^[0-9]+$ ]] || [ "$totalservers" -eq 0 ] 2>/dev/null; then
+        echo "ERROR: Location \"$serverLocation\" not found." >&2
+        rm -f "$servers_json" "$servers"
+        return 1
+      fi
+      echo "Found $totalservers WireGuard servers in region '$serverLocation'"
+      serverindex=$(( RANDOM % totalservers ))
+
+      WG_HOSTNAME=$(jq -r \
+        '.regions | .[] | select(.id=="'"$serverLocation"'") | .servers.wg | .['"$serverindex"'].cn' \
+        "$servers_json")
+      WG_SERVER_IP=$(jq -r \
+        '.regions | .[] | select(.id=="'"$serverLocation"'") | .servers.wg | .['"$serverindex"'].ip' \
+        "$servers_json")
+      WG_SERVER_PORT=$(jq -r '.groups.wg | .[0] | .ports | .[0]' "$servers_json")
+
+      rm -f "$servers_json" "$servers"
+      echo "Selected server $serverindex/$totalservers: $WG_HOSTNAME ($WG_SERVER_IP:$WG_SERVER_PORT)"
+    }
+
+    generateWireguardKey() {
+      PRIVATE_KEY=$(wg genkey)
+      PUBLIC_KEY=$(echo "$PRIVATE_KEY" | wg pubkey)
+      echo "Generated WireGuard keypair"
+    }
+
+    authorizeKeyWithPIAServer() {
+      local addKeyResponse
+      echo "Sending addKey request to $WG_HOSTNAME ($WG_SERVER_IP:$WG_SERVER_PORT)..."
+      addKeyResponse=$(curl -s -G $(proxy_args) \
+        --connect-to "$WG_HOSTNAME::$WG_SERVER_IP:" \
+        --cacert "${caPath}" \
+        --data-urlencode "pt=$PIA_TOKEN" \
+        --data-urlencode "pubkey=$PUBLIC_KEY" \
+        "https://$WG_HOSTNAME:$WG_SERVER_PORT/addKey")
+      local status
+      status=$(echo "$addKeyResponse" | jq -r '.status')
+      if [[ "$status" != "OK" ]]; then
+        echo "ERROR: addKey failed: $addKeyResponse" >&2
+        return 1
+      fi
+      MY_IP=$(echo "$addKeyResponse" | jq -r '.peer_ip')
+      WG_SERVER_PUBLIC_KEY=$(echo "$addKeyResponse" | jq -r '.server_key')
+      WG_SERVER_PORT=$(echo "$addKeyResponse" | jq -r '.server_port')
+      echo "Key authorized — assigned VPN IP: $MY_IP, server port: $WG_SERVER_PORT"
+    }
+
+    writeWireguardQuickFile() {
+      local wgFile=$1
+      local listenPort=$2
+      rm -f "$wgFile"
+      touch "$wgFile"
+      chmod 700 "$wgFile"
+      cat > "$wgFile" <<WGEOF
+    [Interface]
+    PrivateKey = $PRIVATE_KEY
+    ListenPort = $listenPort
+    [Peer]
+    PersistentKeepalive = 25
+    PublicKey = $WG_SERVER_PUBLIC_KEY
+    AllowedIPs = 0.0.0.0/0
+    Endpoint = $WG_SERVER_IP:$WG_SERVER_PORT
+    WGEOF
+      echo "Wrote WireGuard config to $wgFile (listen=$listenPort)"
+    }
+
+    writeChosenServerToFile() {
+      local serverFile=$1
+      jq -n \
+        --arg hostname "$WG_HOSTNAME" \
+        --arg ip "$WG_SERVER_IP" \
+        --arg port "$WG_SERVER_PORT" \
+        '{hostname: $hostname, ip: $ip, port: $port}' > "$serverFile"
+      chmod 700 "$serverFile"
+      echo "Wrote server info to $serverFile"
+    }
+
+    loadChosenServerFromFile() {
+      local serverFile=$1
+      WG_HOSTNAME=$(jq -r '.hostname' "$serverFile")
+      WG_SERVER_IP=$(jq -r '.ip' "$serverFile")
+      WG_SERVER_PORT=$(jq -r '.port' "$serverFile")
+      echo "Loaded server info from $serverFile: $WG_HOSTNAME ($WG_SERVER_IP:$WG_SERVER_PORT)"
+    }
+
+    # Reset WG interface and tear down NAT/forwarding rules.
+    # Called on startup (clear stale state) and on exit via trap.
+    cleanupVpn() {
+      local interfaceName=$1
+      wg set "$interfaceName" listen-port 0 2>/dev/null || true
+      ip -4 address flush dev "$interfaceName" 2>/dev/null || true
+      ip route del default dev "$interfaceName" 2>/dev/null || true
+      iptables -t nat -F 2>/dev/null || true
+      iptables -F FORWARD 2>/dev/null || true
+    }
+
+    connectToServer() {
+      local wgFile=$1
+      local interfaceName=$2
+
+      echo "Applying WireGuard config to $interfaceName..."
+      wg setconf "$interfaceName" "$wgFile"
+      ip -4 address add "$MY_IP" dev "$interfaceName"
+      ip link set mtu 1420 up dev "$interfaceName"
+      echo "WireGuard interface $interfaceName is up with IP $MY_IP"
+    }
+
+    reservePortForward() {
+      local payload_and_signature
+      echo "Requesting port forward signature from $WG_HOSTNAME..."
+      payload_and_signature=$(curl -s -m 5 \
+        --connect-to "$WG_HOSTNAME::$WG_SERVER_IP:" \
+        --cacert "${caPath}" \
+        -G --data-urlencode "token=$PIA_TOKEN" \
+        "https://$WG_HOSTNAME:19999/getSignature")
+      local status
+      status=$(echo "$payload_and_signature" | jq -r '.status')
+      if [[ "$status" != "OK" ]]; then
+        echo "ERROR: getSignature failed: $payload_and_signature" >&2
+        return 1
+      fi
+      PORT_SIGNATURE=$(echo "$payload_and_signature" | jq -r '.signature')
+      PORT_PAYLOAD=$(echo "$payload_and_signature" | jq -r '.payload')
+      PORT=$(echo "$PORT_PAYLOAD" | base64 -d | jq -r '.port')
+      echo "Port forward reserved: port $PORT"
+    }
+
+    writePortRenewalFile() {
+      local portRenewalFile=$1
+      jq -n \
+        --arg signature "$PORT_SIGNATURE" \
+        --arg payload "$PORT_PAYLOAD" \
+        '{signature: $signature, payload: $payload}' > "$portRenewalFile"
+      chmod 700 "$portRenewalFile"
+      echo "Wrote port renewal data to $portRenewalFile"
+    }
+
+    readPortRenewalFile() {
+      local portRenewalFile=$1
+      PORT_SIGNATURE=$(jq -r '.signature' "$portRenewalFile")
+      PORT_PAYLOAD=$(jq -r '.payload' "$portRenewalFile")
+      echo "Loaded port renewal data from $portRenewalFile"
+    }
+
+    refreshPIAPort() {
+      local bindPortResponse
+      echo "Refreshing port forward binding with $WG_HOSTNAME..."
+      bindPortResponse=$(curl -Gs -m 5 \
+        --connect-to "$WG_HOSTNAME::$WG_SERVER_IP:" \
+        --cacert "${caPath}" \
+        --data-urlencode "payload=$PORT_PAYLOAD" \
+        --data-urlencode "signature=$PORT_SIGNATURE" \
+        "https://$WG_HOSTNAME:19999/bindPort")
+      echo "bindPort response: $bindPortResponse"
+    }
+  '';
+}
--- a/common/network/pia-vpn/service-container.nix
+++ b/common/network/pia-vpn/service-container.nix
@@ -0,0 +1,87 @@
+{ config, lib, allModules, ... }:
+
+# Generates service containers that route all traffic through the VPN container.
+# Each container gets a static IP on the VPN bridge with default route → VPN container.
+#
+# Uses lazy mapAttrs inside fixed config keys to avoid infinite recursion.
+# (mkMerge + mapAttrsToList at the top level forces eager evaluation of cfg.containers
+# during module structure discovery, which creates a cycle with config evaluation.)
+
+with lib;
+
+let
+  cfg = config.pia-vpn;
+
+  mkContainer = name: ctr: {
+    autoStart = true;
+    ephemeral = true;
+    privateNetwork = true;
+    hostBridge = cfg.bridgeName;
+
+    bindMounts = mapAttrs
+      (_: mount: {
+        hostPath = mount.hostPath;
+        isReadOnly = mount.isReadOnly;
+      })
+      ctr.mounts;
+
+    config = { config, pkgs, lib, ... }: {
+      imports = allModules ++ [ ctr.config ];
+
+      # Static IP with gateway pointing to VPN container
+      networking.useNetworkd = true;
+      systemd.network.enable = true;
+      networking.useDHCP = false;
+
+      systemd.network.networks."20-eth0" = {
+        matchConfig.Name = "eth0";
+        networkConfig = {
+          Address = "${ctr.ip}/${cfg.subnetPrefixLen}";
+          Gateway = cfg.vpnAddress;
+          DNS = [ cfg.vpnAddress ];
+        };
+      };
+
+      networking.hosts = cfg.containerHosts;
+
+      # DNS through VPN container (queries go through WG tunnel = no DNS leak)
+      networking.nameservers = [ cfg.vpnAddress ];
+
+      # Wait for actual VPN connectivity before network-online.target.
+      # Without this, services start before the VPN tunnel is ready and failures
+      # can't be reported to ntfy (no outbound connectivity yet).
+      systemd.services.wait-for-vpn = {
+        description = "Wait for VPN connectivity";
+        before = [ "network-online.target" ];
+        wantedBy = [ "network-online.target" ];
+        after = [ "systemd-networkd-wait-online.service" ];
+        serviceConfig.Type = "oneshot";
+        path = [ pkgs.iputils ];
+        script = ''
+          until ping -c1 -W2 1.1.1.1 >/dev/null 2>&1; do
+            echo "Waiting for VPN connectivity..."
+            sleep 1
+          done
+        '';
+      };
+
+      # Trust the bridge interface (host reaches us directly for nginx)
+      networking.firewall.trustedInterfaces = [ "eth0" ];
+
+      # Disable host resolv.conf — we use our own networkd DNS config
+      networking.useHostResolvConf = false;
+    };
+  };
+
+  mkContainerOrdering = name: _ctr: nameValuePair "container@${name}" {
+    after = [ "container@pia-vpn.service" ];
+    requires = [ "container@pia-vpn.service" ];
+    partOf = [ "container@pia-vpn.service" ];
+  };
+in
+{
+  config = mkIf cfg.enable {
+    containers = mapAttrs mkContainer cfg.containers;
+    systemd.services = mapAttrs' mkContainerOrdering cfg.containers;
+  };
+}
--- a/common/network/pia-vpn/vpn-container.nix
+++ b/common/network/pia-vpn/vpn-container.nix
@@ -0,0 +1,232 @@
+{ config, lib, allModules, ... }:
+
+# VPN container: runs all PIA logic, acts as WireGuard gateway + NAT for service containers.
+
+with lib;
+
+let
+  cfg = config.pia-vpn;
+  scripts = import ./scripts.nix;
+
+  # Port forwarding derived state
+  forwardingContainers = filterAttrs (_: c: c.receiveForwardedPort != null) cfg.containers;
+  portForwarding = forwardingContainers != { };
+  forwardingContainerName = if portForwarding then head (attrNames forwardingContainers) else null;
+  forwardingContainer = if portForwarding then forwardingContainers.${forwardingContainerName} else null;
+
+  serverFile = "/var/lib/pia-vpn/server.json";
+  wgFile = "/var/lib/pia-vpn/wg.conf";
+  portRenewalFile = "/var/lib/pia-vpn/port-renewal.json";
+  proxy = "http://${cfg.hostAddress}:${toString cfg.proxyPort}";
+
+  # DNAT/forwarding rules for port forwarding
+  dnatSetupScript = optionalString portForwarding (
+    let
+      fwd = forwardingContainer.receiveForwardedPort;
+      targetIp = forwardingContainer.ip;
+      dnatTarget = if fwd.port != null then "${targetIp}:${toString fwd.port}" else targetIp;
+      targetPort = if fwd.port != null then toString fwd.port else "$PORT";
+      tcpRules = optionalString (fwd.protocol == "tcp" || fwd.protocol == "both") ''
+        echo "Setting up TCP DNAT: port $PORT → ${targetIp}:${targetPort}"
+        iptables -t nat -A PREROUTING -i ${cfg.interfaceName} -p tcp --dport $PORT -j DNAT --to ${dnatTarget}
+        iptables -A FORWARD -i ${cfg.interfaceName} -d ${targetIp} -p tcp --dport ${targetPort} -j ACCEPT
+      '';
+      udpRules = optionalString (fwd.protocol == "udp" || fwd.protocol == "both") ''
+        echo "Setting up UDP DNAT: port $PORT → ${targetIp}:${targetPort}"
+        iptables -t nat -A PREROUTING -i ${cfg.interfaceName} -p udp --dport $PORT -j DNAT --to ${dnatTarget}
+        iptables -A FORWARD -i ${cfg.interfaceName} -d ${targetIp} -p udp --dport ${targetPort} -j ACCEPT
+      '';
+      onPortForwarded = optionalString (forwardingContainer.onPortForwarded != null) ''
+        TARGET_IP="${targetIp}"
+        export PORT TARGET_IP
+        echo "Running onPortForwarded hook for ${forwardingContainerName} (port=$PORT, target=$TARGET_IP)"
+        ${forwardingContainer.onPortForwarded}
+      '';
+    in
+    ''
+      if [ "$PORT" -lt 10000 ]; then
+        echo "ERROR: PIA assigned low port $PORT (< 10000), refusing to set up DNAT" >&2
+      else
+        ${tcpRules}
+        ${udpRules}
+        ${onPortForwarded}
+      fi
+    ''
+  );
+in
+{
+  config = mkIf cfg.enable {
+    # Give the container more time to boot (pia-vpn-setup retries can delay readiness)
+    systemd.services."container@pia-vpn".serviceConfig.TimeoutStartSec = mkForce "180s";
+
+    containers.pia-vpn = {
+      autoStart = true;
+      ephemeral = true;
+      privateNetwork = true;
+      hostBridge = cfg.bridgeName;
+      interfaces = [ cfg.interfaceName ];
+
+      bindMounts."/run/agenix" = {
+        hostPath = "/run/agenix";
+        isReadOnly = true;
+      };
+
+      config = { config, pkgs, lib, ... }:
+        let
+          scriptPkgs = with pkgs; [ wireguard-tools iproute2 curl jq iptables coreutils ];
+        in
+        {
+          imports = allModules;
+
+          networking.hosts = cfg.containerHosts;
+
+          # Static IP on bridge — no gateway (VPN container routes via WG only)
+          networking.useNetworkd = true;
+          systemd.network.enable = true;
+          networking.useDHCP = false;
+
+          systemd.network.networks."20-eth0" = {
+            matchConfig.Name = "eth0";
+            networkConfig = {
+              Address = "${cfg.vpnAddress}/${cfg.subnetPrefixLen}";
+              DHCPServer = false;
+            };
+          };
+
+          # Ignore WG interface for wait-online (it's configured manually, not by networkd)
+          systemd.network.wait-online.ignoredInterfaces = [ cfg.interfaceName ];
+
+          # Route ntfy alerts through the host proxy (VPN container has no gateway on eth0)
+          ntfy-alerts.curlExtraArgs = "--proxy http://${cfg.hostAddress}:${toString cfg.proxyPort}";
+
+          # Enable forwarding so bridge traffic can go through WG
+          boot.kernel.sysctl."net.ipv4.ip_forward" = 1;
+
+          # Trust bridge interface
+          networking.firewall.trustedInterfaces = [ "eth0" ];
+
+          # DNS: use systemd-resolved listening on bridge IP so service containers
+          # can use VPN container as DNS server (queries go through WG tunnel = no DNS leak)
+          services.resolved = {
+            enable = true;
+            settings.Resolve.DNSStubListenerExtra = cfg.vpnAddress;
+          };
+
+          # Don't use host resolv.conf — resolved manages DNS
+          networking.useHostResolvConf = false;
+
+          # State directory for PIA config files
+          systemd.tmpfiles.rules = [
+            "d /var/lib/pia-vpn 0700 root root -"
+          ];
+
+          # PIA VPN setup service — does all the PIA auth, WG config, and NAT setup
+          systemd.services.pia-vpn-setup = {
+            description = "PIA VPN WireGuard Setup";
+
+            wants = [ "network-online.target" ];
+            after = [ "network.target" "network-online.target" "systemd-networkd.service" ];
+            wantedBy = [ "multi-user.target" ];
+
+            path = scriptPkgs;
+
+            serviceConfig = {
+              Type = "simple";
+              Restart = "always";
+              RestartSec = "10s";
+              RuntimeMaxSec = "30d";
+            };
+
+            script = ''
+              set -euo pipefail
+              ${scripts.scriptCommon}
+
+              trap 'cleanupVpn ${cfg.interfaceName}' EXIT
+              cleanupVpn ${cfg.interfaceName}
+
+              proxy="${proxy}"
+
+              # 1. Authenticate with PIA via proxy (VPN container has no internet yet)
+              echo "Choosing PIA server in region '${cfg.serverLocation}'..."
+              choosePIAServer '${cfg.serverLocation}'
+
+              echo "Fetching PIA authentication token..."
+              fetchPIAToken
+
+              # 2. Generate WG keys and authorize with PIA server
+              echo "Generating WireGuard keypair..."
+              generateWireguardKey
+
+              echo "Authorizing key with PIA server $WG_HOSTNAME..."
+              authorizeKeyWithPIAServer
+
+              # 3. Configure WG interface (already created by host and moved into our namespace)
+              echo "Configuring WireGuard interface ${cfg.interfaceName}..."
+              writeWireguardQuickFile '${wgFile}' ${toString cfg.wireguardListenPort}
+              writeChosenServerToFile '${serverFile}'
+              connectToServer '${wgFile}' '${cfg.interfaceName}'
+
+              # 4. Default route through WG
+              ip route replace default dev ${cfg.interfaceName}
+              echo "Default route set through ${cfg.interfaceName}"
+
+              # 5. NAT: masquerade bridge → WG (so service containers' traffic appears to come from VPN IP)
+              echo "Setting up NAT masquerade..."
+              iptables -t nat -A POSTROUTING -o ${cfg.interfaceName} -j MASQUERADE
+              iptables -A FORWARD -i eth0 -o ${cfg.interfaceName} -j ACCEPT
+              iptables -A FORWARD -i ${cfg.interfaceName} -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
+
+              ${optionalString portForwarding ''
+                # 6. Port forwarding setup
+                echo "Reserving port forward..."
+                reservePortForward
+                writePortRenewalFile '${portRenewalFile}'
+
+                # First bindPort triggers actual port allocation
+                echo "Binding port $PORT..."
+                refreshPIAPort
+
+                echo "PIA assigned port: $PORT"
+
+                # DNAT rules to forward PIA port to target container
+                ${dnatSetupScript}
+              ''}
+
+              echo "PIA VPN setup complete"
+              exec sleep infinity
+            '';
+
+          };
+
+          # Port refresh timer (every 10 min) — keeps PIA port forwarding alive
+          systemd.services.pia-vpn-port-refresh = mkIf portForwarding {
+            description = "PIA VPN Port Forward Refresh";
+            after = [ "pia-vpn-setup.service" ];
+            requires = [ "pia-vpn-setup.service" ];
+
+            path = scriptPkgs;
+
+            serviceConfig.Type = "oneshot";
+
+            script = ''
+              set -euo pipefail
+              ${scripts.scriptCommon}
+              loadChosenServerFromFile '${serverFile}'
+              readPortRenewalFile '${portRenewalFile}'
+              echo "Refreshing PIA port forward..."
+              refreshPIAPort
+            '';
+          };
+
+          systemd.timers.pia-vpn-port-refresh = mkIf portForwarding {
+            partOf = [ "pia-vpn-port-refresh.service" ];
+            wantedBy = [ "timers.target" ];
+            timerConfig = {
+              OnCalendar = "*:0/10";
+              RandomizedDelaySec = "1m";
+            };
+          };
+        };
+    };
+  };
+}