Guest post originally published on Istio’s blog by Lin Sun, Solo.io, and John Howard, Google
Introducing the new destination oriented waypoint proxy for simplicity and scalability.
Ambient splits Istio’s functionality into two distinct layers, a secure overlay layer and a Layer 7 processing layer. The waypoint proxy is an optional component that is Envoy-based and handles L7 processing for workloads it manages. Since the initial ambient launch in 2022, we have made significant changes to simplify waypoint configuration, debuggability and scalability.
Architecture of waypoint proxies
Similar to sidecar, the waypoint proxy is also Envoy-based and is dynamically configured by Istio to serve your applications configuration. What is unique about the waypoint proxy is that it runs either per-namespace (default) or per-service account. By running outside of the application pod, a waypoint proxy can install, upgrade, and scale independently from the application, as well as reduce operational costs.
Waypoint proxies are deployed declaratively using Kubernetes Gateway resources or the helpful
$ istioctl experimental waypoint generate apiVersion: gateway.networking.k8s.io/v1beta1 kind: Gateway metadata: name: namespace spec: gatewayClassName: istio-waypoint listeners: - name: mesh port: 15008 protocol: HBONE
Istiod will monitor these resources and deploy and manage the corresponding waypoint deployment for users automatically.
Shift source proxy configuration to destination proxy
In the existing sidecar architecture, most traffic-shaping (for example request routing or traffic shifting or fault injection) policies are implemented by the source (client) proxy while most security policies are implemented by the destination (server) proxy. This leads to a number of concerns:
- Scaling – each source sidecar needs to know information about every other destination in the mesh. This is a polynomial scaling problem. Worse, if any destination configuration changes, we need to notify all sidecars at once.
- Debugging – because policy enforcement is split between the client and server sidecars, it can be hard to understand the behavior of the system when troubleshooting.
- Mixed environments – if we have systems where not all clients are part of the mesh, we get inconsistent behavior. For example, a non-mesh client wouldn’t respect a canary rollout policy, leading to unexpected traffic distribution.
- Ownership and attribution – ideally a policy written in one namespace should only affect work done by proxies running in the same namespace. However, in this model, it is distributed and enforced by each sidecar. While Istio has designed around this constraint to make this secure, it is still not optimal.
In ambient, all policies are enforced by the destination waypoint. In many ways, the waypoint acts as a gateway into the namespace (default scope) or service account. Istio enforces that all traffic coming into the namespace goes through the waypoint, which then enforces all policies for that namespace. Because of this, each waypoint only needs to know about configuration for its own namespace.
The scalability problem, in particular, is a nuisance for users running in large clusters. If we visualize it, we can see just how big an improvement the new architecture is.
Consider a simple deployment, where we have 2 namespaces, each with 2 (color coded) deployments. The Envoy (XDS) configuration required to program the sidecars is shown as circles:
In the sidecar model, we have 4 workloads, each with 4 sets of configuration. If any of those configurations changed, all of them would need to be updated. In total there are 16 configurations distributed.
In the waypoint architecture, however, the configuration is dramatically simplified:
Here, we see a very different story. We have only 2 waypoint proxies, as each one is able to serve the entire namespace, and each one only needs configuration for its own namespace. In total we have 25% of the amount of configuration sent, even for a simple example.
If we scale each namespace up to 25 deployments with 10 pods each and each waypoint deployment with 2 pods for high availability, the numbers are even more impressive – the waypoint config distribution requires just 0.8% of the configuration distribution of the sidecar, as the table below illustrates!
|Config Distribution||Namespace 1||Namespace 2||Total|
|Sidecars||25 configurations * 250 sidecars||25 configurations * 250 sidecars||12500|
|Waypoints||25 configurations * 2 waypoints||25 configurations * 2 waypoints||100|
|Waypoints / Sidecars||0.8%||0.8%||0.8%|
While we use namespace scoped waypoint proxies to illustrate the simplification above, the simplification is similar when you apply it to service account waypoint proxies.
This reduced configuration means lower resource usage (CPU, RAM, and network bandwidth) for both the control plane and data plane. While users today can see similar improvements with careful usage of
exportTo in their Istio networking resources or of the Sidecar API, in ambient mode this is no longer required, making scaling a breeze.
What if my destination doesn’t have a waypoint proxy?
The design of ambient mode centers around the assumption that most configuration is best implemented by the service producer, rather than the service consumer. However, this isn’t always the case – sometimes we need to configure traffic management for destinations we don’t control. A common example of this would be connecting to an external service with improved resilience to handle occasional connection issues (e.g., to add a timeout for calls to
This is an area under active development in the community, where we design how traffic can be routed to your egress gateway and how you can configure the egress gateway with your desired policies. Look out for future blog posts in this area!
A deep-dive of waypoint configuration
Assuming you have followed the ambient get started guide up to and including the control traffic section, you have deployed a waypoint proxy for the bookinfo-reviews service account to direct 90% traffic to reviews v1 and 10% traffic to reviews v2.
istioctl to retrieve the listeners for the
reviews waypoint proxy:
$ istioctl proxy-config listener deploy/bookinfo-reviews-istio-waypoint --waypoint LISTENER CHAIN MATCH DESTINATION envoy://connect_originate ALL Cluster: connect_originate envoy://main_internal inbound-vip|9080||reviews.default.svc.cluster.local-http ip=10.96.104.108 -> port=9080 Inline Route: /* envoy://main_internal direct-tcp ip=10.244.2.14 -> ANY Cluster: encap envoy://main_internal direct-tcp ip=10.244.1.6 -> ANY Cluster: encap envoy://main_internal direct-tcp ip=10.244.2.11 -> ANY Cluster: encap envoy://main_internal direct-http ip=10.244.2.11 -> application-protocol='h2c' Cluster: encap envoy://main_internal direct-http ip=10.244.2.11 -> application-protocol='http/1.1' Cluster: encap envoy://main_internal direct-http ip=10.244.2.14 -> application-protocol='http/1.1' Cluster: encap envoy://main_internal direct-http ip=10.244.2.14 -> application-protocol='h2c' Cluster: encap envoy://main_internal direct-http ip=10.244.1.6 -> application-protocol='h2c' Cluster: encap envoy://main_internal direct-http ip=10.244.1.6 -> application-protocol='http/1.1' Cluster: encap envoy://connect_terminate default ALL
For requests arriving on port
15008, which by default is Istio’s inbound HBONE port, the waypoint proxy terminates the HBONE connection and forwards the request to the
main_internal listener to enforce any workload policies such as AuthorizationPolicy. If you are not familiar with internal listeners, they are Envoy listeners that accepts user space connections without using the system network API. The
--waypoint flag added to the
istioctl proxy-config command, above, instructs it to show the details of the
main_internal listener, its filter chains, chain matches, and destinations.
10.96.104.108 is the reviews’ service VIP and
10.244.x.x are the reviews’ v1/v2/v3 pod IPs, which you can view for your cluster using the
kubectl get svc,pod -o wide command. For plain text or HBONE terminated inbound traffic, it will be matched on the service VIP and port 9080 for reviews or by pod IP address and application protocol (either
Checking out the clusters for the
reviews waypoint proxy, you get the
main_internal cluster along with a few inbound clusters. Other than the clusters for infrastructure, the only Envoy clusters created are for services and pods running in the same service account. No clusters are created for services or pods running elsewhere.
$ istioctl proxy-config clusters deploy/bookinfo-reviews-istio-waypoint SERVICE FQDN PORT SUBSET DIRECTION TYPE DESTINATION RULE agent - - - STATIC connect_originate - - - ORIGINAL_DST encap - - - STATIC kubernetes.default.svc.cluster.local 443 tcp inbound-vip EDS main_internal - - - STATIC prometheus_stats - - - STATIC reviews.default.svc.cluster.local 9080 http inbound-vip EDS reviews.default.svc.cluster.local 9080 http/v1 inbound-vip EDS reviews.default.svc.cluster.local 9080 http/v2 inbound-vip EDS reviews.default.svc.cluster.local 9080 http/v3 inbound-vip EDS sds-grpc - - - STATIC xds-grpc - - - STATIC zipkin - - - STRICT_DNS
Note that there are no
outbound clusters in the list, which you can confirm using
istioctl proxy-config cluster deploy/bookinfo-reviews-istio-waypoint --direction outbound! What’s nice is that you didn’t need to configure
exportTo on any other bookinfo services (for example, the
ratings services). In other words, the
reviews waypoint is not made aware of any unnecessary clusters, without any extra manual configuration from you.
Display the list of routes for the
reviews waypoint proxy:
$ istioctl proxy-config routes deploy/bookinfo-reviews-istio-waypoint NAME DOMAINS MATCH VIRTUAL SERVICE encap * /* inbound-vip|9080|http|reviews.default.svc.cluster.local * /* reviews.default default
Recall that you didn’t configure any Sidecar resources or
exportTo configuration on your Istio networking resources. You did, however, deploy the
bookinfo-productpage route to configure an ingress gateway to route to
productpage but the
reviews waypoint has not been made aware of any such irrelevant routes.
Displaying the detailed information for the
inbound-vip|9080|http|reviews.default.svc.cluster.local route, you’ll see the weight-based routing configuration directing 90% of the traffic to
reviews v1 and 10% of the traffic to
reviews v2, along with some of Istio’s default retry and timeout configurations. This confirms the traffic and resiliency policies are shifted from the source to destination oriented waypoint as discussed earlier.
$ istioctl proxy-config routes deploy/bookinfo-reviews-istio-waypoint --name "inbound-vip|9080|http|reviews.default.svc.cluster.local" -o yaml - name: inbound-vip|9080|http|reviews.default.svc.cluster.local validateClusters: false virtualHosts: - domains: - '*' name: inbound|http|9080 routes: - decorator: operation: reviews:9080/* match: prefix: / metadata: filterMetadata: istio: config: /apis/networking.istio.io/v1alpha3/namespaces/default/virtual-service/reviews route: maxGrpcTimeout: 0s retryPolicy: hostSelectionRetryMaxAttempts: "5" numRetries: 2 retriableStatusCodes: - 503 retryHostPredicate: - name: envoy.retry_host_predicates.previous_hosts typedConfig: '@type': type.googleapis.com/envoy.extensions.retry.host.previous_hosts.v3.PreviousHostsPredicate retryOn: connect-failure,refused-stream,unavailable,cancelled,retriable-status-codes timeout: 0s weightedClusters: clusters: - name: inbound-vip|9080|http/v1|reviews.default.svc.cluster.local weight: 90 - name: inbound-vip|9080|http/v2|reviews.default.svc.cluster.local weight: 10
Check out the endpoints for
reviews waypoint proxy:
$ istioctl proxy-config endpoints deploy/bookinfo-reviews-istio-waypoint ENDPOINT STATUS OUTLIER CHECK CLUSTER 127.0.0.1:15000 HEALTHY OK prometheus_stats 127.0.0.1:15020 HEALTHY OK agent envoy://connect_originate/ HEALTHY OK encap envoy://connect_originate/10.244.1.6:9080 HEALTHY OK inbound-vip|9080|http/v2|reviews.default.svc.cluster.local envoy://connect_originate/10.244.1.6:9080 HEALTHY OK inbound-vip|9080|http|reviews.default.svc.cluster.local envoy://connect_originate/10.244.2.11:9080 HEALTHY OK inbound-vip|9080|http/v1|reviews.default.svc.cluster.local envoy://connect_originate/10.244.2.11:9080 HEALTHY OK inbound-vip|9080|http|reviews.default.svc.cluster.local envoy://connect_originate/10.244.2.14:9080 HEALTHY OK inbound-vip|9080|http/v3|reviews.default.svc.cluster.local envoy://connect_originate/10.244.2.14:9080 HEALTHY OK inbound-vip|9080|http|reviews.default.svc.cluster.local envoy://main_internal/ HEALTHY OK main_internal unix://./etc/istio/proxy/XDS HEALTHY OK xds-grpc unix://./var/run/secrets/workload-spiffe-uds/socket HEALTHY OK sds-grpc
Note that you don’t get any endpoints related to any services other than reviews, even though you have a few other services in the
We are very excited about the waypoint simplification focusing on destination oriented waypoint proxies. This is another significant step towards simplifying Istio’s usability, scalability and debuggability which are top priorities on Istio’s roadmap. Follow our getting started guide to try the ambient alpha build today and experience the simplified waypoint proxy!