Skip to main content Skip to sidebar

Load Balancing with nftables only

•
configuration load-balancing networking nftables

Sometimes a separate load balancer is overkill. If the requirement is only L4 distribution for TCP or UDP, nftables can do it directly in the kernel with DNAT, maps, and the number generator.

The nftables wiki already has a compact Load balancing page. This note expands the same primitives into a complete gateway ruleset and adds the operational pieces that are easy to forget: forwarding, SNAT, backend updates, and health-check limitations.

The important limitation: this is packet filtering and NAT, not HAProxy or Envoy. There are no HTTP checks, no TLS awareness, no retries, no queueing, and no automatic backend health state. If you need these features, use a real load balancer. If you need a small and predictable gateway that distributes connections between a few backend hosts, nftables is enough.

Example Topology

flowchart LR
    C["client"] -->|"203.0.113.10:443"| LB["nftables gateway"]
    LB -->|"DNAT"| A["10.10.20.11:443"]
    LB -->|"DNAT"| B["10.10.20.12:443"]
    A --> LB
    B --> LB
    LB --> C

For this example:

  • public interface: wan0
  • backend interface: lan0
  • service VIP: 203.0.113.10
  • load balancer LAN address: 10.10.20.1
  • backend servers: 10.10.20.11, 10.10.20.12
  • service port: 443/tcp

Enable forwarding:

sysctl -w net.ipv4.ip_forward=1

Persist it:

printf 'net.ipv4.ip_forward=1\n' > /etc/sysctl.d/99-forwarding.conf
sysctl --system

Minimal Stateful NAT Load Balancer

This version does round-robin backend selection. The NAT chain is stateful: only the first packet of a flow selects the backend, then conntrack applies the same NAT binding to the rest of the flow.

#!/usr/sbin/nft -f

flush ruleset

define WAN = "wan0"
define LAN = "lan0"
define VIP = 203.0.113.10
define LB_LAN_IP = 10.10.20.1
define BACKENDS = { 10.10.20.11, 10.10.20.12 }

table ip lb_nat {
    chain prerouting {
        type nat hook prerouting priority dstnat; policy accept;

        iifname $WAN ip daddr $VIP tcp dport 443 counter dnat to numgen inc mod 2 map {
            0 : 10.10.20.11,
            1 : 10.10.20.12,
        }
    }

    chain postrouting {
        type nat hook postrouting priority srcnat; policy accept;

        oifname $LAN ip daddr $BACKENDS tcp dport 443 counter snat to $LB_LAN_IP
    }
}

table inet lb_filter {
    chain forward {
        type filter hook forward priority filter; policy drop;

        ct state established,related counter accept
        iifname $WAN oifname $LAN ip daddr $BACKENDS tcp dport 443 counter accept
    }
}

The snat rule is optional. Use it when backend servers do not route replies back through the nftables gateway. If the backends use the gateway as their default route, remove the postrouting rule and preserve the real client source address.

Random Distribution

Round-robin is deterministic. I usually prefer random distribution for simple public services:

iifname $WAN ip daddr $VIP tcp dport 443 counter dnat to numgen random mod 2 map {
    0 : 10.10.20.11,
    1 : 10.10.20.12,
}

For TCP, this still selects only once per connection because it is in a stateful NAT chain.

Weighted Distribution

Intervals make simple weights possible. This example sends about 70 percent of new flows to 10.10.20.11 and 30 percent to 10.10.20.12:

iifname $WAN ip daddr $VIP tcp dport 443 counter dnat to numgen random mod 10 map {
    0-6 : 10.10.20.11,
    7-9 : 10.10.20.12,
}

The same idea works with numgen inc mod 10 if strict rotation is preferred.

Port Distribution

The backend port can also come from a map. This is useful when several instances run on one backend host:

iifname $WAN ip daddr $VIP tcp dport 443 counter dnat to 10.10.20.11 : numgen inc mod 2 map {
    0 : 8443,
    1 : 9443,
}

The incoming service is still 203.0.113.10:443, but new flows alternate between 10.10.20.11:8443 and 10.10.20.11:9443.

Source Hashing

If the same client IP should usually land on the same backend, use a hash instead of numgen:

iifname $WAN ip daddr $VIP tcp dport 443 counter dnat to jhash ip saddr mod 2 map {
    0 : 10.10.20.11,
    1 : 10.10.20.12,
}

This is useful for simple source affinity. It is stable while the modulo and backend map stay the same. Changing the backend count changes the mapping, so do not treat this as a Maglev-style load balancer.

If too many clients are hidden behind the same source NAT, hash more fields:

iifname $WAN ip daddr $VIP tcp dport 443 counter dnat to jhash ip saddr . tcp sport mod 2 map {
    0 : 10.10.20.11,
    1 : 10.10.20.12,
}

This distributes more evenly, but multiple connections from the same client can go to different backends.

Stateless NAT

The wiki also shows stateless NAT by changing packet headers directly with ip daddr set. This is lighter than stateful NAT because conntrack is not used for the rewritten packets, but it is much easier to break. Use it only when the packet path is fully controlled and reply traffic does not need the stateful NAT engine to undo the translation.

table ip lb_raw {
    chain prerouting {
        type filter hook prerouting priority raw; policy accept;

        iifname $WAN ip daddr $VIP tcp dport 443 notrack
    }
}

table ip lb_mangle {
    chain prerouting {
        type filter hook prerouting priority mangle; policy accept;

        iifname $WAN ip daddr $VIP tcp dport 443 counter ip daddr set numgen inc mod 2 map {
            0 : 10.10.20.11,
            1 : 10.10.20.12,
        }
    }
}

For normal TCP/UDP services, I would start with stateful DNAT and only use stateless NAT for a measured reason.

Direct Server Return

Direct Server Return keeps the VIP as the destination IP and changes only Ethernet headers. The backend receives the packet for the VIP and replies directly to the client, so return traffic bypasses the load balancer.

Backend requirements:

  • the VIP must be configured locally on each backend, usually on lo
  • backends must not answer ARP for the VIP on the LAN
  • the service must bind to the VIP or to all addresses
  • firewalling and observability must account for asymmetric traffic

Example for TCP using a netdev ingress chain:

table netdev lb_dsr {
    chain wan_ingress {
        type filter hook ingress device "wan0" priority 0; policy accept;

        ip daddr 203.0.113.10 tcp dport 443 counter \
            ether saddr set 02:00:00:00:00:01 \
            ether daddr set jhash ip saddr . tcp sport mod 2 map {
                0 : 02:00:00:00:20:11,
                1 : 02:00:00:00:20:12,
            } \
            fwd to "lan0"
    }
}

Replace 02:00:00:00:00:01 with the load balancer MAC on the backend segment and the map values with the real backend MAC addresses.

Using a Named Map

For a short example, anonymous maps are fine. For operations, named maps are easier to inspect and update:

table ip lb_nat {
    map https_backends {
        type integer : ipv4_addr
        elements = {
            0 : 10.10.20.11,
            1 : 10.10.20.12,
        }
    }

    chain prerouting {
        type nat hook prerouting priority dstnat; policy accept;

        iifname $WAN ip daddr $VIP tcp dport 443 counter dnat to numgen random mod 2 map @https_backends
    }
}

Inspect the map:

nft list map ip lb_nat https_backends

Change a backend with one nftables batch:

nft -f - <<'EOF'
delete element ip lb_nat https_backends { 1 }
add element ip lb_nat https_backends { 1 : 10.10.20.13 }
EOF

Existing conntrack entries keep their old NAT binding. New flows use the updated map.

Health Checks

nftables will not check backend health by itself. The usual pattern is to run a tiny external checker that updates the map or reloads a generated ruleset.

For example, if 10.10.20.12 is unhealthy:

nft -f - <<'EOF'
delete element ip lb_nat https_backends { 1 }
add element ip lb_nat https_backends { 1 : 10.10.20.11 }
EOF

This keeps the modulo unchanged and sends both slots to the healthy backend. When the server recovers:

nft -f - <<'EOF'
delete element ip lb_nat https_backends { 1 }
add element ip lb_nat https_backends { 1 : 10.10.20.12 }
EOF

This is not as graceful as draining in an application load balancer. It is simple and works when the service can tolerate connection-level decisions.

Apply And Check

Validate syntax before loading:

nft -c -f /etc/nftables.conf

Load the ruleset:

nft -f /etc/nftables.conf

Check counters:

nft list ruleset

Check conntrack entries if conntrack-tools is installed:

conntrack -L -p tcp --dport 443

Notes

  • NAT rules belong in type nat chains, not filter chains.
  • The forward filter sees the packet after DNAT, so match backend addresses there.
  • If SNAT is used, backends see the load balancer as the client.
  • If SNAT is not used, the backend route back to the client must pass through the load balancer.
  • For UDP, conntrack still creates flow entries, but timeouts matter more than with TCP.
  • DSR avoids reply-path NAT, but it requires backend VIP and ARP tuning.
  • This design is only L4. Anything requiring HTTP headers, TLS SNI, retries, circuit breaking, or health-aware balancing should be handled by a user-space load balancer.

References