Istio - multiple subsets for traffic policy overrides on match

[clarkjohnd]

👋 Hey folks, I'm investigating a new use case where we use Istio subsets and virtual services to category client-based traffic and apply different policies (connection pools, retries, so on) based on the source client name, while also using Istio-based Argo Rollouts.

Right now, our virtual services expect to route using destination rule subsets - canary and stable, with dynamic routing from the Rollouts controller. Destination rule is pretty simple, same subsets, controller-managed switching of targets.

apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: my-service
  namespace: ns
spec:
  gateways:
  - gateway.istio-system.svc.cluster.local
  hosts:
  - my-service.domain.com
  http:
  - match:
    - name: http
      port: 443
    name: http
    route:
    - destination:
        host: my-service.ns.svc.cluster.local
        port:
          number: 80
        subset: stable
      weight: 100
    - destination:
        host: my-service.ns.svc.cluster.local
        port:
          number: 80
        subset: canary
      weight: 0
---
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: my-service
  namespace: ns
spec:
  host: my-service.ns.svc.cluster.local
  subsets:
  - labels:
      app.kubernetes.io/name: my-service
      deployment_mechanism: gitops
      rollouts-pod-template-hash: 54884c6654
    name: canary
  - labels:
      app.kubernetes.io/name: my-service
      deployment_mechanism: gitops
      rollouts-pod-template-hash: 54884c6654
    name: stable
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST
    tls:
      mode: ISTIO_MUTUAL

I suppose my question here is, is there (and if not would the community be open to) any support for using multiple matches and subsets, but the subsets instead are suffixed (or something along these lines) with stable/canary, and the canary/gold-canary/whatever-canary weights update together? For example, changes to the above config with header-based matching for different Istio rules:

apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: my-service
  namespace: ns
spec:
  gateways:
  - gateway.istio-system.svc.cluster.local
  hosts:
  - my-service.domain.com
  http:
  # header-based matching
  - name: gold
    match:
    - headers:
        # some header-based match to use this route instead
      name: http
      port: 443
    retries:
      # override "GOLD" retry policy for example
    route:
    # same destinations, different DR subsets and overrides
    - destination:
        host: my-service.ns.svc.cluster.local
        port:
          number: 80
        subset: gold-canary
      weight: 100 # weights get updated in-line with standard routing
    - destination:
        host: my-service.ns.svc.cluster.local
        port:
          number: 80
        subset: gold-stable
      weight: 0
  # standard match
  - match:
    - name: http
      port: 443
    name: http
    route:
    - destination:
        host: my-service.ns.svc.cluster.local
        port:
          number: 80
        subset: stable
      weight: 100
    - destination:
        host: my-service.ns.svc.cluster.local
        port:
          number: 80
        subset: canary
      weight: 0
---
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: my-service
  namespace: ns
spec:
  host: my-service.ns.svc.cluster.local
  subsets:
  # standard subsets
  - labels:
      app.kubernetes.io/name: my-service
      deployment_mechanism: gitops
      rollouts-pod-template-hash: 54884c6654
    name: canary
  - labels:
      app.kubernetes.io/name: my-service
      deployment_mechanism: gitops
      rollouts-pod-template-hash: 54884c6654
    name: stable
  # gold subsets
  - labels:
      app.kubernetes.io/name: my-service
      deployment_mechanism: gitops
      rollouts-pod-template-hash: 54884c6654
    name: gold-canary
    trafficPolicy:
      # some gold-specific overrides
  - labels:
      app.kubernetes.io/name: my-service
      deployment_mechanism: gitops
      rollouts-pod-template-hash: 54884c6654
    name: gold-stable
    trafficPolicy:
      # some gold-specific overrides
  trafficPolicy:
    loadBalancer:
      simple: LEAST_REQUEST
    tls:
      mode: ISTIO_MUTUAL

So ultimately the requests all land in the same place (canary or stable pods of a rollout), but it would allow us to support different traffic policies etc. for prioritised clients/paths whatever, while still supporting Argo Rollouts.

Maybe this exists (but I haven't been able to find it), and if it doesn't I'd happily take a stab at it and can create an issue.