Isito iptables & CNI

众所周知 istio 通过 iptables 拦截流量,对于这个 iptables 的规则,我们仔细的分析下。

IPTABLES 规则

我们找到任意的一个节点先看看我的配置项。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
iptables --table nat --list
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
ISTIO_INBOUND tcp -- anywhere anywhere

Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
ISTIO_OUTPUT tcp -- anywhere anywhere
RETURN udp -- anywhere anywhere udp dpt:domain owner UID match 1337
RETURN udp -- anywhere anywhere udp dpt:domain owner GID match 1337
REDIRECT udp -- anywhere kube-dns.kube-system.svc.cluster.local udp dpt:domain redir ports 15053

Chain POSTROUTING (policy ACCEPT)
target prot opt source destination

Chain ISTIO_INBOUND (1 references)
target prot opt source destination
RETURN tcp -- anywhere anywhere tcp dpt:15008
RETURN tcp -- anywhere anywhere tcp dpt:ssh
RETURN tcp -- anywhere anywhere tcp dpt:15090
RETURN tcp -- anywhere anywhere tcp dpt:15021
RETURN tcp -- anywhere anywhere tcp dpt:15020
ISTIO_IN_REDIRECT tcp -- anywhere anywhere

Chain ISTIO_IN_REDIRECT (3 references)
target prot opt source destination
REDIRECT tcp -- anywhere anywhere redir ports 15006

Chain ISTIO_OUTPUT (1 references)
target prot opt source destination
RETURN all -- 127.0.0.6 anywhere
ISTIO_IN_REDIRECT tcp -- anywhere !localhost tcp dpt:!domain owner UID match 1337
RETURN tcp -- anywhere anywhere tcp dpt:!domain ! owner UID match 1337
RETURN all -- anywhere anywhere owner UID match 1337
ISTIO_IN_REDIRECT all -- anywhere !localhost owner GID match 1337
RETURN tcp -- anywhere anywhere tcp dpt:!domain ! owner GID match 1337
RETURN all -- anywhere anywhere owner GID match 1337
REDIRECT tcp -- anywhere kube-dns.kube-system.svc.cluster.local tcp dpt:domain redir ports 15053
RETURN all -- anywhere localhost
ISTIO_REDIRECT all -- anywhere anywhere

Chain ISTIO_REDIRECT (1 references)
target prot opt source destination
REDIRECT tcp -- anywhere anywhere redir ports 15001

入口流量顺序

  1. PREROUTING: 任意的 TCP 协议都会被 转发到 ISTIO_INBOUND,也就是本机器的所有的外部流量都会被拦击
  2. ISTIO_INBOUND: 15008/22/15090/15021/15020 都会直接被放行
    1. 这里可以参考 Ports used by Istio
    2. 其他请求会被 转发 到 ISTIO_IN_REDIRECT
  3. ISTIO_IN_REDIRECT: 也就是 Envoy Inbound 的监听器

出口流量顺序

  1. OUTPUT: 任意 TCP 都转发至 ISTIO_OUTPUT,除了以下几个特殊的Case
    1. UID 或者 GID 是 1337 (uid=1337(istio-proxy) gid=1337(istio-proxy) groups=1337(istio-proxy) 直接放行
    2. KUBE DNS 请求重定向到 15053(DNS Agent)
  2. ISTIO_OUTPUT:

略微复杂点,主要几个特殊的情况

  1. 127.0.0.6 直接放行,这里是一个特殊处理 inbound passthrough cluster 就是这个 IP
  2. 当服务通过 VIP 访问本身,在本机器的 ENVOY 完成所有逻辑,appN => Envoy (client) => Envoy (server) => appN.
  3. 重定向 53 的 DNS 请求
  4. 忽略访问 Localhost 的请求

istio agnet iptables

众所周知,istio agent 来帮我们配置 iptables,代码主要在 root.go

代码主要帮助完成一些自动化的配置,在此不做分析。

CNI

Istio 同样提供的 CNI 插件,但是这里的 CNI 主要的工作就是帮助系统完成 Iptables 的初始化

主要消除了 Sidecar 部署到 Istio 中的 NET_ADMINNET_RAW 权限的需求。

CNI 作为一个标准的 CS 模型,提供了一套标准的代码模板,Istio 基于这个模式,实现的逻辑在 plugin

CmdAdd githublink
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
log.Debugf("Setting up redirect for pod %v/%v", podNamespace, podName)
if redirect, redirErr := NewRedirect(pi); redirErr != nil {
log.Errorf("Pod %s/%s redirect failed due to bad params: %v", podNamespace, podName, redirErr)
} else {
// Get the constructor for the configured type of InterceptRuleMgr
interceptMgrCtor := GetInterceptRuleMgrCtor(interceptRuleMgrType)
if interceptMgrCtor == nil {
log.Errorf("Pod redirect failed due to unavailable InterceptRuleMgr of type %s",
interceptRuleMgrType)
} else {
rulesMgr := interceptMgrCtor()
if err := rulesMgr.Program(podName, args.Netns, redirect); err != nil {
return err
}
}
}

主要逻辑也是就在处理 NetworkNamespace 上,而这里直接复用了 Agent Iptables 的逻辑,所以任何区别

Program githublink
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
// Program defines a method which programs iptables based on the parameters
// provided in Redirect.
func (ipt *iptables) Program(podName, netns string, rdrct *Redirect) error {
viper.Set(constants.CNIMode, true)
viper.Set(constants.NetworkNamespace, netns)
viper.Set(constants.EnvoyPort, rdrct.targetPort)
viper.Set(constants.ProxyUID, rdrct.noRedirectUID)
viper.Set(constants.InboundInterceptionMode, rdrct.redirectMode)
viper.Set(constants.ServiceCidr, rdrct.includeIPCidrs)
viper.Set(constants.InboundPorts, rdrct.includePorts)
viper.Set(constants.LocalExcludePorts, rdrct.excludeInboundPorts)
viper.Set(constants.ExcludeInterfaces, rdrct.excludeInterfaces)
viper.Set(constants.LocalOutboundPortsExclude, rdrct.excludeOutboundPorts)
viper.Set(constants.ServiceExcludeCidr, rdrct.excludeIPCidrs)
viper.Set(constants.KubeVirtInterfaces, rdrct.kubevirtInterfaces)
drf := dryRunFilePath.Get()
viper.Set(constants.DryRun, drf != "")
viper.Set(constants.OutputPath, drf)
viper.Set(constants.RedirectDNS, rdrct.dnsRedirect)
viper.Set(constants.CaptureAllDNS, rdrct.dnsRedirect)
iptablesCmd := cmd.GetCommand()
log.Infof("============= Start iptables configuration for %v =============", podName)
defer log.Infof("============= End iptables configuration for %v =============", podName)
if err := iptablesCmd.Execute(); err != nil {
return err
}
return nil
}

参考