Traffic shifting in Istio

Want to know more? Get in touch with us, or delve into the details of the latest release. Or just take a look at some of the Istio features that Backyards automates and simplifies for you, and which we've already blogged about.

At Banzai Cloud we work with Istio quite a bit and run a lot of Istio-based service meshes for our customers. Earlier this year we opensourced the Banzai Cloud Istio operator in order to simplify provisioning, management, upgrades, and multi-cluster scenarios. However, we still felt that this was not simple enough. A few months ago we announced Backyards, a method of simplifying, creating and managing Istio, with planned support for all Istio features - the more complicated, the better! As Backyards will be GA in September, we'd like to start a series of posts that take a deep dive into some intertesting Istio features, exploring their benefits, configuring/using them on Istio, and also showcasing how extremely simple it is to do that with Backyards (now Cisco Service Mesh Manager), even in multi- and hybridcloud environments. In this particular post, we'll be focusing on Istio's traffic shifting feature.

Backyards is a core component of the Pipeline platform as well.

Traffic shifting introduction

Traffic shifting makes it possible to gradually migrate traffic from one version of a microservice to another. This usually happens when migrating from an older version of an app to a newer one.

When the migration process itself is automated and rolled back automatically on failure, it's called a canary release. To learn more about how to execute canary releases with Backyards, check out our introductory blog post on automatic canary deployments.

Traffic shifting in Istio

Istio's traffic shifting can be configured by two Istio Custom Resources, namely Destination Rule and Virtual Service. In short, in a Destination Rule so-called subsets need to be defined to identify specific versions of a service, and in a Virtual Service different weights can be assigned to these subsets in order to control how much traffic should be directed to each service version. For the subsets, you'll need to define a Destination Rule like this:

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: movies
spec:
  host: movies
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
  - name: v3
    labels:
      version: v3

This defines the following subset names: v1, v2 and v3. These subset names can be used later in Virtual Services to route traffic to the appropriate version of our movies service, based on predefined version labels.

There's one thing which you need to pay special attention to when you manually create the Destination Rule resource, which is whether or not you have mutual TLS enabled for this service. If you do, you'll also need to set the field below within your Destination Rule, otherwise your caller services will probably receive 503 responses when calling the movies service:

trafficPolicy:
  tls:
    mode: ISTIO_MUTUAL

Mutual TLS can be enabled globally for a specific namespace or for a specific service, as well. You should be aware of these settings in order to determine whether you should set trafficPolicy.tls.mode to ISTIO_MUTUAL or not. More importantly, it is very easy to forget to set this field when you are trying to configure a completely different feature (e.g. traffic shifting).

Tip: Always think about mutual TLS before creating a Destination Rule!

For the weights, you'll need to define a Virtual Service like this:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: movies
spec:
  hosts:
  - movies
  http:
  - route:
    - destination:
        host: movies
        subset: v1
      weight: 33
    - destination:
        host: movies
        subset: v2
      weight: 33
    - destination:
        host: movies
        subset: v3
      weight: 34

Now the Virtual Service is set to route 33% of all traffic to subset v1, 33% to v2 and 34% to v3. To make sure that these settings are actually functional, you can check the application logs or use a monitoring tool to examine the flow of traffic.

Traffic shifting with Backyards (now Cisco Service Mesh Manager)

When using Backyards, you don't need to manually edit Destination Rules and Virtual Services to set traffic shifting rules. Instead, you can achieve the same result via a convenient UI, or, if you prefer, through the Backyards CLI command line tool.

We'll introduce the Backyards CLI tool in detail in an upcoming blog post!

You don't need to worry about misconfiguring your Destination Rule by forgetting to set trafficPolicy.tls.mode to ISTIO_MUTUAL. Backyards takes care of this for you; it finds out whether your service has mutual TLS enabled or not and sets the aforementioned field accordingly.

This is just one example of the validation features of Backyards which can protect you from a potential misconfiguration. There are many more to come which will be introduced soon!

On top of this, you can see visualizations of your services and requests, so you can easily determine how much traffic is being directed to each of your workloads.

Try it out!

Create a cluster

First of all, we'll need a Kubernetes cluster.

I created a Kubernetes cluster on GKE via the Pipeline platform. If you'd like to do likewise, go ahead and create your cluster on any of the several cloud providers we support or on-premise using Pipeline for free.

Install Backyards

The easiest way by far of installing Istio, Backyards, and a demo application on a brand new cluster is to use the Backyards CLI. You just need to issue one command (KUBECONFIG must be set for your cluster):

$ backyards install -a --run-demo

This command first installs Istio with our open-source Istio operator, then installs Backyards itself as well as a demo application for demonstration purposes. After the installation of each component has finished, the Backyards UI will automatically open and send some traffic to the demo application. By issuing this one simple command you can watch as Backyards starts a brand new Istio cluster in just a few minutes! Give it a try!

You can do all these steps in a sequential order as well. Backyards requires an Istio cluster - if you don't have one, you can install Istio with $ backyards istio install. Once you have Istio installed, you can install Backyards with $ backyards install. Finally, you can deploy the demo application with backyards demoapp install.

Pro tip: Backyards is a core component of the Pipeline platform as well - you can try the hosted developer version here: https://try.pipeline.banzai.cloud/ (Service Mesh tab).

Traffic shifting using Backyards UI

View traffic shifting rules

If you've installed Backyards with the Backyards CLI, by default, traffic shifting will be configured for the movies service so that it routes 33% of all traffic to subset v1, 33% to v2 and 34% to v3. You don't have to fetch the Virtual Service (e.g. with kubectl) to get this information. It is immediately available on the right side of the Backyards UI when you click on the movies service icon. When traffic begins flowing, the request rates from the movies service to its workloads should be very close to each other. You don't need to check application logs to find out how much traffic is flowing to each subset, since you can see that immediately on the UI. See it in action: https://youtu.be/n1jEjuSAqLA

Set traffic shifting rules

You don't need to edit the corresponding Virtual Service resource manually, you can easily change the weights for each subset from the UI. Let's set it so that all traffic goes to v2.

As you will see, Backyards (compared e.g. with Kiali) is not just a web based UI mainly for observability, but also a featureful management tool for your service mesh, which is able to work with single- and multi-cluster service meshes, have a powerful CLI and GraphQL API as well.

After setting this rule and sending a workload to the service, we can make sure that all traffic is actually being routed to v2. See it in action: https://youtu.be/fIb_1oyLoZw 

Remove traffic shifting rules

You can easily remove traffic shifting rules with the Remove rules button. If you do so, traffic will return to being distributed equally across the workloads, so request rates should also return to about equal: See it in action: https://youtu.be/CWMgvb8W_l8

Traffic shifting using backyards-cli

View traffic shifting rules

You can list the traffic shifting rules of a service in a given namespace with the following command:

$ backyards routing ts get backyards-demo/movies
INFO[0000] traffic shifting for backyards-demo/movies is currently set to v1=33, v2=33, v3=34

Set traffic shifting rules

Now let's make the same changes we did earlier, but this time through the CLI.

$ backyards routing ts set backyards-demo/movies v2=100
INFO[0001] traffic shifting for backyards-demo/movies set to v2=100 successfully

To verify that the command was successful:

$ backyards routing ts get backyards-demo/movies
INFO[0000] traffic shifting for backyards-demo/movies is currently set to v2=100

Remove traffic shifting rules

To remove the traffic shifting rules:

$ backyards routing ts delete backyards-demo/movies
INFO[0001] traffic shifting rules set to backyards-demo/movies successfully deleted

To verify that the command was successful:

$ backyards routing ts get backyards-demo/movies
INFO[0000] no traffic shifting rules set for backyards-demo/movies

Traffic shifting using the Backyards GraphQL API

Backyards is composed of several components, like Istio, Banzai Cloud's Istio operator, our multi-cluster Canary release operator, as well as several backends. However, all of these are behind Backyards' GraphQL API. The Backyards UI and CLI both use Backyards' GraphQL API, which will be released with the GA version (we are in the middle of a refactor). Thus, users will soon be able to either use our tools to manage Istio or build their own clients!

Cleanup

To remove the demo application, Backyards, and Istio from your cluster, you just need to apply one command, which takes care of removing these components in the correct order:

$ backyards uninstall -a

Takeaway

With Backyards, you don't necessarily need to be familiar with Istio's Custom Resources, and you don't have to edit them manually to set traffic shifting rules. Instead, you can easily configure these rules from a convenient UI or with the Backyards CLI command line tool. You can then check the visualized traffic flow to make sure that the rules and your services are working correctly.

As you see, even a relatively simple feature like traffic shifting can lead to potential misconfigurations (e.g. omitting to set ISTIO_MUTUAL in a Destination Rule). Backyards automatically protects you from such > mistakes.

Next up: Circuit Breaker.

About Backyards

Banzai Cloud’s Backyards (now Cisco Service Mesh Manager) is a multi and hybrid-cloud enabled service mesh platform for constructing modern applications. Built on Kubernetes and our Istio operator, it gives you flexibility, portability, and consistency across on-premise datacenters and cloud environments. Use our simple, yet extremely powerful UI and CLI, and experience automated canary releases, traffic shifting, routing, secure service communication, in-depth observability and more, for yourself.

About Banzai Cloud

Banzai Cloud is changing how private clouds are built: simplifying the development, deployment, and scaling of complex applications, and putting the power of Kubernetes and Cloud Native technologies in the hands of developers and enterprises, everywhere. #multicloud #hybridcloud #BanzaiCloud