Ambient mesh uses HTTP CONNECT over mTLS to implement its secure tunnels and insert waypoint proxies in the path, a pattern we call HBONE (HTTP-Based Overlay Network Environment). HBONE provides for a cleaner encapsulation of traffic than TLS on its own while enabling interoperability with common load-balancer infrastructure. FIPS builds are used by default to meet compliance needs. More details on HBONE, its standards-based approach, and plans for UDP and other non-TCP protocols will be provided in a future blog.
funchandleConnect(w http.ResponseWriter, r *http.Request)bool { t0 := time.Now() log.WithLabels("host", r.Host, "source", r.RemoteAddr).Info("Received CONNECT") // Send headers back immediately so we can start getting the body w.(http.Flusher).Flush() ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) defer cancel()
dst, err := (&net.Dialer{}).DialContext(ctx, "tcp", r.Host) if err != nil { w.WriteHeader(http.StatusServiceUnavailable) log.Errorf("failed to dial upstream: %v", err) returntrue } log.Infof("Connected to %v", r.Host) w.WriteHeader(http.StatusOK)
$ k get nodes -owide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME ambient-control-plane Ready control-plane 6h37m v1.25.0 172.18.0.4 <none> Ubuntu 22.04.1 LTS 5.4.0-105-generic containerd://1.6.7 ambient-worker Ready <none> 6h37m v1.25.0 172.18.0.2 <none> Ubuntu 22.04.1 LTS 5.4.0-105-generic containerd://1.6.7 ambient-worker2 Ready <none> 6h37m v1.25.0 172.18.0.3 <none> Ubuntu 22.04.1 LTS 5.4.0-105-generic containerd://1.6.7
我们使用一个常见的 HTTP 请求分析链路
1
kubectl exec deploy/sleep -- curl -s http://productpage:9080/ | head -n1
从 sleep(work2) 访问 productpage(work1)
Sleep -> zTunnel
从 istio-cni-node 的日志里我们可以看到修改的参数
1 2
2022-09-16T05:27:57.759272Z info cni Adding pod 'sleep-7b85956664-nvfkm/default' (d7e030b1-3269-4016-8742-e2322bea7fcb) to ipset 2022-09-16T05:27:57.759275Z info cni Adding route for sleep-7b85956664-nvfkm/default: [table 100 10.244.2.7/32 via 192.168.126.2 dev istioin src 10.244.2.1]
$ kubectl exec sleep-7b85956664-nvfkm -- ip route default via 10.244.2.1 dev eth0 10.244.2.0/24 via 10.244.2.1 dev eth0 src 10.244.2.7 10.244.2.1 dev eth0 scope link src 10.244.2.7
root@ambient-worker:/$ ip route show table 101 default via 192.168.127.2 dev istioout 10.244.2.3 dev veth2e666193 scope link
root@ambient-worker:/$ ip rule 0: from all lookup local 100: from all fwmark 0x200/0x200 goto 32766 101: from all fwmark 0x100/0x100 lookup 101 102: from all fwmark 0x40/0x40 lookup 102 103: from all lookup 100 32766: from all lookup main 32767: from all lookup default
root@ambient-worker:/$ iptables-save -A ztunnel-POSTROUTING -m mark --mark 0x100/0x100 -j ACCEPT -A ztunnel-PREROUTING -m mark --mark 0x100/0x100 -j ACCEPT
通过对请求 Mark 的方式,流量走了 Table 101,然后进入了 istioout
1 2 3 4
6: istioout: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default link/ether 96:99:db:e4:ca:ad brd ff:ff:ff:ff:ff:ff inet 192.168.127.1/30 brd 192.168.127.3 scope global istioout valid_lft forever preferred_lft forever
而这个恰好就是 zTunnel 组件。
zTunnel -> Waypoint
到这里就非常的符合我们的经验了,直接看 Envoy 就好了。
1 2 3 4 5 6 7 8
$ k exec -n istio-system ztunnel-4gkmd -- iptables
root@ambient-worker2:/# ip route show table 100 10.244.1.2 dev veth983ae22d scope link 10.244.1.4 via 192.168.126.2 dev istioin src 10.244.1.1 10.244.1.5 via 192.168.126.2 dev istioin src 10.244.1.1 10.244.1.6 via 192.168.126.2 dev istioin src 10.244.1.1 10.244.1.7 via 192.168.126.2 dev istioin src 10.244.1.1 10.244.1.8 via 192.168.126.2 dev istioin src 10.244.1.1
$ ./istioctl pc listener -n istio-system ztunnel-q55t5 ADDRESS PORT MATCH DESTINATION 0 ALL Cluster: outbound_tunnel_clus_spiffe://cluster.local/ns/default/sa/bookinfo-reviews 0 ALL Cluster: outbound_tunnel_clus_spiffe://cluster.local/ns/default/sa/bookinfo-details 0 ALL Cluster: outbound_tunnel_clus_spiffe://cluster.local/ns/default/sa/notsleep 0 ALL Cluster: outbound_tunnel_clus_spiffe://cluster.local/ns/default/sa/bookinfo-ratings 0 ALL Cluster: outbound_tunnel_clus_spiffe://cluster.local/ns/default/sa/bookinfo-productpage 0 ALL Cluster: outbound_tunnel_clus_spiffe://cluster.local/ns/default/sa/sleep 0.0.0.0 15001 ALL PassthroughCluster 0.0.0.0 15001 ALL PassthroughCluster 0.0.0.0 15001 ALL Non-HTTP/Non-TCP 0.0.0.0 15001 ALL Cluster: spiffe://cluster.local/ns/default/sa/bookinfo-reviews_to_status-port_istio-ingressgateway.istio-system.svc.cluster.local_outbound_internal 0.0.0.0 15001 ALL Cluster: spiffe://cluster.local/ns/default/sa/bookinfo-reviews_to_server_waypoint_proxy_spiffe://cluster.local/ns/default/sa/bookinfo-productpage 0.0.0.0 15001 ALL Cluster: spiffe://cluster.local/ns/default/sa/bookinfo-reviews_to_metrics_kube-dns.kube-system.svc.cluster.local_outbound_internal 0.0.0.0 15001 ALL Cluster: spiffe://cluster.local/ns/default/sa/bookinfo-reviews_to_https_istio-ingressgateway.istio-system.svc.cluster.local_outbound_internal 0.0.0.0 15001 ALL Cluster: spiffe://cluster.local/ns/default/sa/bookinfo-reviews_to_https-webhook_istiod.istio-system.svc.cluster.local_outbound_internal 0.0.0.0 15001 ALL Cluster: spiffe://cluster.local/ns/default/sa/bookinfo-reviews_to_https-dns_istiod.istio-system.svc.cluster.local_outbound_internal 0.0.0.0 15001 ALL Cluster: spiffe://cluster.local/ns/default/sa/bookinfo-reviews_to_http_sleep.default.svc.cluster.local_outbound_internal 0.0.0.0 15001 ALL Cluster: spiffe://cluster.local/ns/default/sa/bookinfo-reviews_to_http_reviews.default.svc.cluster.local_outbound_internal 0.0.0.0 15001 ALL Cluster: spiffe://cluster.local/ns/default/sa/bookinfo-reviews_to_http_ratings.default.svc.cluster.local_outbound_internal 0.0.0.0 15001 ALL Cluster: spiffe://cluster.local/ns/default/sa/bookinfo-reviews_to_http_notsleep.default.svc.cluster.local_outbound_internal 0.0.0.0 15001 ALL Cluster: spiffe://cluster.local/ns/default/sa/bookinfo-reviews_to_http_details.default.svc.cluster.local_outbound_internal 0.0.0.0 15001 ALL Cluster: spiffe://cluster.local/ns/default/sa/bookinfo-reviews_to_http2_istio-ingressgateway.istio-system.svc.cluster.local_outbound_internal