In-Pod IPtables Rule Injection in Istio Ambient Mode Explained

A deep dive into how iptables rules in Istio ambient mode enable transparent traffic interception and control within Pods.

  • Dec 20, 2024
  • Istio
  • 4 Minute
  • 986 words
Copyright
This is an original article by Jimmy Song. You may repost it, but please credit this source: https://jimmysong.io/en/blog/istio-ambient-inpod-iptables/
Click to show the outline

In my previous blog post, I provided an overview of the iptables rules injected within pod network namespaces in Istio ambient mode. This article takes a closer look at these rules, explaining how they achieve transparent traffic interception and redirection within pods.

iptables Rules Inside the Pod

In a pod’s network namespace, the Istio CNI Node Agent sets up a series of iptables rules to enable transparent traffic interception and redirection. The following rules, injected into the mangle and nat tables, demonstrate how Istio processes inbound and outbound traffic.

# Generated by iptables-save v1.8.9 (nf_tables) on Thu Nov 14 08:43:17 2024
*mangle
:PREROUTING ACCEPT [99138:22880045]  # Default ACCEPT policy for the PREROUTING chain in the mangle table.
:INPUT ACCEPT [0:0]                  # Default ACCEPT policy for the INPUT chain in the mangle table.
:FORWARD ACCEPT [0:0]                # Default ACCEPT policy for the FORWARD chain in the mangle table.
:OUTPUT ACCEPT [100900:34940164]     # Default ACCEPT policy for the OUTPUT chain in the mangle table.
:POSTROUTING ACCEPT [0:0]            # Default ACCEPT policy for the POSTROUTING chain in the mangle table.
:ISTIO_OUTPUT - [0:0]                # Custom ISTIO_OUTPUT chain for handling outbound traffic.
:ISTIO_PRERT - [0:0]                 # Custom ISTIO_PRERT chain for handling prerouting traffic.
-A PREROUTING -j ISTIO_PRERT         # Direct all PREROUTING traffic to the ISTIO_PRERT chain.
-A OUTPUT -j ISTIO_OUTPUT            # Direct all OUTPUT traffic to the ISTIO_OUTPUT chain.
-A ISTIO_OUTPUT -m connmark --mark 0x111/0xfff -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff
# Restore connection mark 0x111/0xfff for consistent connection tracking.

-A ISTIO_PRERT -m mark --mark 0x539/0xfff -j CONNMARK --set-xmark 0x111/0xfff
# Set connection mark to 0x111/0xfff for packets marked 0x539/0xfff in PREROUTING.

COMMIT  # Apply mangle table rules.
# Completed on Thu Nov 14 08:43:17 2024

# Generated by iptables-save v1.8.9 (nf_tables) on Thu Nov 14 08:43:17 2024
*nat
:PREROUTING ACCEPT [2:120]           # Default ACCEPT policy for the PREROUTING chain in the nat table.
:INPUT ACCEPT [0:0]                  # Default ACCEPT policy for the INPUT chain in the nat table.
:OUTPUT ACCEPT [119:9344]            # Default ACCEPT policy for the OUTPUT chain in the nat table.
:POSTROUTING ACCEPT [0:0]            # Default ACCEPT policy for the POSTROUTING chain in the nat table.
:ISTIO_OUTPUT - [0:0]                # Custom ISTIO_OUTPUT chain for handling outbound NAT traffic.
:ISTIO_PRERT - [0:0]                 # Custom ISTIO_PRERT chain for handling prerouting NAT traffic.
-A PREROUTING -j ISTIO_PRERT         # Direct all PREROUTING traffic to the ISTIO_PRERT chain.
-A OUTPUT -j ISTIO_OUTPUT            # Direct all OUTPUT traffic to the ISTIO_OUTPUT chain.
-A ISTIO_OUTPUT -d 169.254.7.127/32 -p tcp -m tcp -j ACCEPT
# Allow TCP traffic destined for 169.254.7.127 (likely an internal Istio address).

-A ISTIO_OUTPUT -p tcp -m mark --mark 0x111/0xfff -j ACCEPT
# Allow TCP traffic marked as 0x111/0xfff.

-A ISTIO_OUTPUT ! -d 127.0.0.1/32 -o lo -j ACCEPT
# Allow traffic to the loopback interface excluding 127.0.0.1.

-A ISTIO_OUTPUT ! -d 127.0.0.1/32 -p tcp -m mark ! --mark 0x539/0xfff -j REDIRECT --to-ports 15001
# Redirect outbound TCP traffic (not marked 0x539/0xfff) destined outside 127.0.0.1 to port 15001 (outbound socket).

-A ISTIO_PRERT -s 169.254.7.127/32 -p tcp -m tcp -j ACCEPT
# Allow TCP traffic originating from 169.254.7.127 in the PREROUTING chain.

-A ISTIO_PRERT ! -d 127.0.0.1/32 -p tcp -m tcp ! --dport 15008 -m mark ! --mark 0x539/0xfff -j REDIRECT --to-ports 15006
# Redirect inbound TCP traffic (not marked 0x539/0xfff) to port 15006 (inbound socket) if its destination port is not 15008.

COMMIT  # Apply nat table rules.
# Completed on Thu Nov 14 08:43:17 2024

Role of Specific Ports

These iptables rules differentiate and handle various types of traffic using specific ports:

  • 15008 (HBONE socket): Handles HTTP-based traffic transparently using the HBONE protocol.
  • 15006 (plaintext socket): Manages unencrypted traffic within the mesh for inter-pod communication.
  • 15001 (outbound socket): Controls outbound traffic and enforces policies for accessing external services.

By leveraging these ports, Istio enables transparent management and control of inbound, outbound, and internal traffic, enforcing fine-grained security policies and traffic controls. For more information, refer to Istio Application Requirements.

Significance of 0x539 Mark

The 0x539 mark identifies traffic originating from Istio proxies (e.g., ztunnel). This mark is applied to distinguish packets processed by proxies, ensuring they are not reprocessed or misrouted.

Significance of 0x111 Mark

The 0x111 mark is used for connection-level marking within the Istio mesh, indicating that a connection has been processed by a proxy. The CONNMARK module in iptables extends this mark to the entire connection, speeding up subsequent packet matching.

Visualizing iptables Rules

The following diagram illustrates the execution path of traffic through iptables rules, helping to understand how traffic is matched and redirected:

image
iptables Rules Visualization

For further details on how Istio CNI handles iptables, refer to the source code: istio/cni/pkg/iptables/iptables.go at master · istio/istio · GitHub.

Routing Visualization for Different Traffic Types

Here are visualized traffic paths for encrypted and plaintext communication across and within nodes:

Cross-Node Encrypted Traffic

image
Cross-Node Encrypted Traffic Path

Cross-Node Plaintext Traffic

image
Cross-Node Plaintext Traffic Path

Same-Node Encrypted Traffic

image
Same-Node Encrypted Traffic Path

Same-Node Plaintext Traffic

image
Same-Node Plaintext Traffic Path
  1. Application Sends Request: Traffic originates from the application process and enters the pod’s network namespace.
  2. iptables Rule Matching:
    • Outbound Traffic matches OUTPUT chain rules, redirecting eligible traffic to the ISTIO_OUTPUT chain.
    • Matched traffic is marked and accepted.
  3. REDIRECT Handling: Traffic is captured and redirected by iptables to ztunnel (port 15006 for plaintext, 15008 for encrypted).
  4. ztunnel Processing: ztunnel performs policy enforcement and encryption.
  5. Traffic Forwarded to Target Service: Processed traffic is sent to the target service via a built tunnel.
  6. Response Path: Responses flow back to ztunnel for decryption and policy checks before reaching the application.

Conclusion

By analyzing the iptables rules in Istio ambient mode, we see how Istio’s CNI plugin establishes a transparent traffic interception mechanism within pods. These rules ensure that traffic entering and leaving pods is correctly handled by ztunnel, enabling finer-grained traffic management and security policy enforcement. Stay tuned for more deep dives into Istio ambient mode networking!

This blog was initially published at tetrate.io.

Last updated on Dec 20, 2024

Istio Ambient Iptables