Published on 00/00/0000
Last updated on 00/00/0000
Published on 00/00/0000
Last updated on 00/00/0000
Share
INSIGHTS
15 min read
Published on 02/10/2020
Last updated on 08/19/2024
Istio ingress and egress gateways
Share
In today's blogpost we're going to be discussing ingress and egress gateways. First, we'll cover the basics, then we'll go into detail and explore how they work through a series of practical examples. Ingress and egress gateways are load balancers that operate at the edges of any network receiving incoming or outgoing HTTP/TCP connections. Ingress gateways make it possible to define an entry points into an Istio mesh for all incoming traffic to flow through. Egress gateways are similar: they define exit points from the mesh, but also allow for the application of Istio features to the traffic exiting the mesh. Some examples of these features are monitoring, routing rules and retries. In Istio, both gateways are based on Envoy. Envoy handles reverse proxying and load balancing for services running inside a service mesh's network, and also for external services outside the mesh. An Istio gateway in a Kubernetes cluster consists of, at minimum, a Deployment and a Service. For an ingress gateway the latter is typically a LoadBalancer-type service, or, when an ingress gateway is used solely within a cluster, a ClusterIP-type service. For an egress gateway the service type is almost always ClusterIP. Although Istio can be configured to support Kubernetes Ingress Resources, a better approach would be to use Istio's custom resources (Gateway, VirtualService). That way you can use Istio features for more than internal services, including ingresses, giving you access to way more features than you'd have with just Kubernetes' Ingress Resources.
The
Gatewayresource describes the port configuration of the gateway deployment that operates at the edge of the mesh and receives incoming or outgoing HTTP/TCP connections. The specification describes a set of ports that should be exposed, the type of protocol to use, TLS configuration – if any – of the exposed ports, and so on. For more information about Gateways, see the Istio documentation.VirtualServicedefines a set of traffic routing rules to apply when a host is addressed. Each routing rule defines matching criteria for the traffic of a specific protocol. If the traffic matches a routing rule, then it is sent to a named destination service defined in the registry. For example, it can route requests to different versions of a service or to a completely different service than was requested. Requests can be routed based on the request source and destination, HTTP paths and header fields, and weights associated with individual service versions. For more information about VirtualServices, see the Istio documentation.
Istio Ingress and Egress Gateways, showtime
Our only prerequisite before exploring these concepts through examples is the creation of a Kubernetes cluster.
You can create a Kubernetes cluster on five different cloud providers, or on-premise via the free developer version of the Pipeline platform. But you can also bring your own cluster.
Install Istio using Backyards
The easiest way to install a production ready Istio and a demo application on a brand new cluster is to use the Backyards CLI. Register for an evaluation version and run the following command to install the CLI tool (KUBECONFIG must be set for your cluster): Register for the free tier version of Cisco Service Mesh Manager (formerly called Banzai Cloud Backyards) and follow the Getting Started Guide for up-to-date instructions on the installation. This command installs Istio with the Banzai Cloud open-source Istio operator, then installs Backyards (now Cisco Service Mesh Manager) itself, as well as an application for demonstration purposes. After the installation has finished, the Backyards UI will automatically open and send some traffic to the demo application. Issuing this one simple command causes Backyards to start a new Istio mesh in just a few minutes!
You can read more about the latest Backyards release > here
Using the main ingress gateway
Remember, as we talked about earlier in this post, ingress gateways enable us to expose services to the external world. Accordingly, an ingress gateway serves as the entry point for all services running within the mesh. 
Determining the ingress IP
In order to expose a service, you must first know the external IP of the ingress gateway. Fortunately, the Banzai Cloud Istio operator helps us with this.
❯ kubectl -n istio-system get istio mesh
NAME STATUS ERROR GATEWAYS AGE
mesh Available [18.184.240.108 18.196.72.62] 15mIn general, you should manually set an external hostname that points to these addresses, but for demo purposes you can use xip.io, which is a domain name that provides wildcard DNS for any IP address.
Expose a service through the ingress gateway
The demo application that comes with Backyards (now Cisco Service Mesh Manager) contains several microservices. The frontpage service serves as the entry point of that application. Now, let's create a Gateway and a VirtualService resource to expose the frontpage service. If you're using xip.io, the external hostname for the service is going to be either frontpage.18.184.240.108.xip.io or frontpage.18.196.72.62.xip.io.
Create Gateway resource
The following Gateway resource configures listening ports on the matching gateway deployment.
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: frontpage
namespace: backyards-demo
spec:
selector:
istio: ingressgateway # use Istio default gateway implementation
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "frontpage.18.184.240.108.xip.io"
- "frontpage.18.196.72.62.xip.io"Create VirtualService resource
The following VirtualService resource configures routing for the external hosts within the mesh.
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: frontpage
namespace: backyards-demo
spec:
hosts:
- "frontpage.18.184.240.108.xip.io"
- "frontpage.18.196.72.62.xip.io"
gateways:
- frontpage
http:
- route:
- destination:
port:
number: 8080
host: frontpage.backyards-demo.svc.cluster.local
Access the service on the external address
Try to access the service on the external address you just configured, on host frontpage.18.184.240.108.xip.io. If everything is set correctly, the following command will return an HTTP 200 status code.
❯ curl -i frontpage.18.184.240.108.xip.io
HTTP/1.1 200 OK
content-type: text/plain
date: Sun, 02 Feb 2020 19:15:14 GMT
content-length: 9
x-envoy-upstream-service-time: 16
server: istio-envoy
frontpage
Create and use multiple ingress gateways
Having one ingress and egress gateway to handle incoming and outgoing traffic from the mesh is part of a basic Istio installation and has been supported by the Banzai Cloud Istio operator from day one, but in large enterprise deployments our customers typically use Backyards (now Cisco Service Mesh Manager) with multiple ingress or egress gateways. 
The Banzai Cloud Istio operator and multiple gateways
The Banzai Cloud Istio operator has an Istio custom resource that defines mesh configurations. The main ingress/egress gateways are part of the specifications of that resource.
spec:
...
gateways:
egress:
enabled: true
maxReplicas: 1
minReplicas: 1
ports:
- name: http2
port: 80
protocol: TCP
targetPort: 80
- name: https
port: 443
protocol: TCP
targetPort: 443
replicaCount: 1
sds:
enabled: false
image: docker.io/istio/node-agent-k8s:1.4.3
serviceType: ClusterIP
enabled: true
ingress:
enabled: true
maxReplicas: 1
minReplicas: 1
ports:
- name: status-port
port: 15020
protocol: TCP
targetPort: 15020
- name: http2
port: 80
protocol: TCP
targetPort: 80
- name: https
port: 443
protocol: TCP
targetPort: 443
- name: tls
port: 15443
protocol: TCP
targetPort: 15443
replicaCount: 1
sds:
enabled: false
image: docker.io/istio/node-agent-k8s:1.4.3
serviceType: LoadBalancer
...
One way to support multiple gateways would have been to add support for specifying them in the existing custom resource. But we chose a radically different approach for the following reasons:
- a separate controller should reconcile gateways, as there could be multiple gateways in multiple namespaces
- RBAC: having a separate CR allows us to properly control who can manage gateways, without having permissions to modify other parts of the Istio mesh configuration
Thus, we have added a new CRD to the Banzai Cloud Istio operator, called the MeshGateway, that can be used to add and configure a new Istio ingress or egress gateway into the mesh. When you create a new MeshGateway CR, the Banzai Cloud Istio operator will take care of configuring and reconciling the necessary resources, including the Envoy deployment and its related Kubernetes service.
Create a new service in the default namespace and expose it
To demonstrate how to create and use multiple ingress gateways, let's add a simple service to the default namespace.
❯ kubectl apply -f https://raw.githubusercontent.com/banzaicloud/istio-operator/release-1.4/docs/federation/multimesh/echo-service.yaml
deployment.extensions/echo created
service/echo createdIt would be possible to expose this echo service through the existing ingress gateway, similar to the way we would for the frontpage service, but let's assume we need to expose this service on port 8000, without modifying the existing ingress gateway.
Create a new ingress gateway using the MeshGateway resource
Apply the following resource and the operator will create a new ingress gateway deployment, and a corresponding service.
apiVersion: istio.banzaicloud.io/v1beta1
kind: MeshGateway
metadata:
name: echo-ingress
namespace: default
spec:
maxReplicas: 1
minReplicas: 1
ports:
- name: http
port: 8000
protocol: TCP
targetPort: 8000
serviceType: LoadBalancer
type: ingressThe
MeshGatewayresource automatically labels the createdServiceandDeploymentresources with thegateway-nameandgateway-typelabels and their corresponding values.
Get the echo ingress gateway IP
❯ kubectl -n default get meshgateways echo-ingress
NAME TYPE SERVICE TYPE STATUS INGRESS IPS ERROR AGE
echo-ingress ingress LoadBalancer Available [18.185.173.38 18.197.110.20] 29mCreate Gateway and VirtualService resources
Just like in the first example, the following Gateway and VirtualService resources are necessary to configure listening ports on the matching gateway deployment.
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: echo
namespace: default
spec:
selector:
gateway-name: echo-ingress
gateway-type: ingress
servers:
- port:
number: 8000
name: http
protocol: HTTP
hosts:
- "echo.18.197.110.20.xip.io"
- "echo.18.185.173.38.xip.io"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: echo
namespace: default
spec:
hosts:
- "echo.18.197.110.20.xip.io"
- "echo.18.185.173.38.xip.io"
gateways:
- echo
http:
- route:
- destination:
port:
number: 80
host: echo.default.svc.cluster.local
Access the echo service on the external address
The service should be accessible on host echo.18.197.110.20.xip.io and port 8000.
❯ curl -i echo.18.197.110.20.xip.io:8000
HTTP/1.1 200 OK
date: Sun, 02 Feb 2020 19:22:15 GMT
content-type: text/plain
server: istio-envoy
x-envoy-upstream-service-time: 1
Hostname: echo-68578cf9d9-874rz
...Our ability to easily create ingress gateways gives you fine-grained control over how services are exposed to the outside world. That way, teams can manage the exposure of their own services without running the risk of misconfiguring the services of other teams. Another way of tackling this potential issue is to have separate load balancer configurations with, for example, different port level settings.
Accessing External Services
Direct outbound traffic
Any traffic that's outbound from a pod with an Istio sidecar will also pass through that sidecar's container, or, more precisely, through Envoy. Therefore, the accessibility of external services depends on the configuration of that Envoy proxy. By default, Istio configures the Envoy proxy to passthrough requests for unknown services. Although this provides a convenient way of getting started with Istio, its generally a good idea to put stricter controls in place.
To read more about the Sidecar object configuration, check out this informative blog post:.
Check the default outbound traffic policy
❯ kubectl -n istio-system get istio mesh -o jsonpath='{@.spec.outboundTrafficPolicy.mode}{"\n"}'
ALLOW_ANYThis traffic policy should be set to ALLOW_ANY by default.
Make an HTTP request to httpbin.org from the echo service container in the default namespace
❯ kubectl -n default exec -ti deploy/echo -- curl -sD /dev/stderr http://httpbin.org -o/dev/null |head -5
Defaulting container name to echo-service.
Use 'kubectl describe pod/echo-68578cf9d9-874rz -n default' to see all of the containers in this pod.
HTTP/1.1 200 OK
date: Sun, 02 Feb 2020 20:43:40 GMT
content-type: text/html; charset=utf-8
content-length: 9593
server: envoyThis should work fine, since, by default, every sidecar sends traffic towards unknown services through its passhtrough proxy.
Change the default outbound traffic policy to block unknown services
Now we're going to demonstrate a more controlled way of enabling access to external services. Change the spec.outboundTrafficPolicy.mode option from the ALLOW_ANY mode to the REGISTRY_ONLY mode in the mesh Istio resource in the istio-system namespace.
❯ kubectl -n istio-system patch istio mesh -p '{"spec":{"outboundTrafficPolicy":{"mode":"REGISTRY_ONLY"}}}' --type='merge'
istio.istio.banzaicloud.io/mesh patchedMake another HTTP request to httpbin.org from the echo service container in the default namespace
❯ kubectl -n default exec -ti deploy/echo -- curl -sD /dev/stderr http://httpbin.org -o/dev/null |head -5
HTTP/1.1 502 Bad Gateway
location: http://httpbin.org/
date: Sun, 02 Feb 2020 20:53:44 GMT
server: envoy
content-length: 0Now we're getting a 502 response code, since now the traffic towards external services is blocked and it is going through Envoy's blackhole cluster.
Create a ServiceEntry to allow HTTP access to httpbin.org
ServiceEntry resources enable adding additional entries into Istio’s internal service registry, so that auto-discovered services in the mesh can access/route to these manually specified services. A service entry describes the properties of a service (DNS name, VIPs, ports, protocols, endpoints). These services could be external to the mesh (for example, web APIs) or mesh-internal services that are not part of the platform’s service registry. For more information about the ServiceEntry resource, see the Istio documentation.
Apply the following ServiceEntry to allow for HTTP access to httpbin.org
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: httpbin-ext
namespace: default
spec:
hosts:
- httpbin.org
ports:
- number: 80
name: http
protocol: HTTP
resolution: DNS
location: MESH_EXTERNALMake another HTTP request to httpbin.org from the echo service container in the default namespace
❯ kubectl -n default exec -ti deploy/echo -- curl -sD /dev/stderr http://httpbin.org -o/dev/null |head -5
Defaulting container name to echo-service.
Use 'kubectl describe pod/echo-68578cf9d9-874rz -n default' to see all of the containers in this pod.
HTTP/1.1 200 OK
date: Sun, 02 Feb 2020 21:13:40 GMT
content-type: text/html; charset=utf-8
content-length: 9593
server: envoyOutbound traffic through an egress gateway
As you probably recall from earlier in this blogpost, egress gateways are exit points from the mesh that allow us to apply Istio features. This includes applying features like monitoring and route rules to traffic that's exiting the mesh.
Use cases
Let's take a quick look at some use cases. Consider an organization which requires some, or all, outbound traffic to go through dedicated nodes. These nodes could be separated from the rest of the nodes for the purposes of monitoring and policy enforcement. Now imagine a cluster where the application nodes don’t have public IPs, so the in-mesh services that run on them cannot access the internet directly. Defining an egress gateway and routing egress traffic through it, then allocating public IPs to the gateway nodes would allow for controlled access to external services.
Create an egress gateway with the MeshGateway resource
Apply the following resource and the Istio operator will create a new egress gateway deployment and a corresponding service.
apiVersion: istio.banzaicloud.io/v1beta1
kind: MeshGateway
metadata:
name: egress
namespace: default
spec:
maxReplicas: 1
minReplicas: 1
ports:
- name: http
port: 80
protocol: TCP
targetPort: 80
serviceType: ClusterIP
type: egressCreate a Gateway resource for the egress gateway
Similar to the ingress gateway configuration, a Gateway resource must be created that will be a bridge between Istio configuration resources and the deployment of a matching gateway. Apply the following Gateway resource to configure the outbound port, 80, on the egress gateway that was just defined in the previous step.
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: egress
namespace: default
spec:
selector:
gateway-name: egress
gateway-type: egress
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"Define a VirtualService resource to direct traffic from the sidecars to the egress gateway
Apply the following VirtualService to direct traffic from the sidecars to the egress gateway and also from the egress gateway to the external service.
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin-egress
spec:
hosts:
- httpbin.org
gateways:
- egress
- mesh
http:
- match:
- gateways:
- mesh
port: 80
route:
- destination:
host: egress.default.svc.cluster.local
port:
number: 80
- match:
- gateways:
- egress
port: 80
route:
- destination:
host: httpbin.org
port:
number: 80Resend an HTTP request to httpbin.org from the echo service container in the default namespace
❯ kubectl -n default exec -ti deploy/echo -- curl -sD /dev/stderr http://httpbin.org/headers -o/dev/null |head -5
Defaulting container name to echo-service.
Use 'kubectl describe pod/echo-68578cf9d9-874rz -n default' to see all of the containers in this pod.
HTTP/1.1 200 OK
date: Sun, 02 Feb 2020 22:20:28 GMT
content-type: application/json
content-length: 1850
server: envoyOutput should be the same as earlier, but if we check the logs of the egress gateway, it shows that the request actually went through the egress gateway.
❯ kubectl -n default logs -l gateway-name=egress,gateway-type=egress -c istio-proxy | tail -1
[2020-02-02T22:20:29.191Z] "GET /headers HTTP/1.1" 200 - "-" "-" 0 1850 173 173 "10.20.5.1" "curl/7.47.0" "bf26bf45-4a0b-4d78-a968-98515a890a91" "httpbin.org" "52.202.2.199:80" outbound|80||httpbin.org - 10.20.5.16:80 10.20.5.1:58584 - -Takeaway
Ingress and egress gateways are core concepts of a service mesh. Although Istio itself provides the basic building blocks, having an easy and simple way to create and manage multiple mesh gateways is a must. The Banzai Cloud Istio operator provides support with a new CRD called MeshGateway. Give it a try, and quickstart your Istio experience with Backyards (now Cisco Service Mesh Manager)!
About Banzai Cloud Istio operator
Banzai Cloud Istio operator is a simple way to deploy, manage and maintain Istio service meshes, even in multi-cluster topologies.
About Backyards
Banzai Cloud’s Backyards (now Cisco Service Mesh Manager) is a multi and hybrid-cloud enabled service mesh platform for constructing modern applications. Built on Kubernetes and our Istio operator, it gives you flexibility, portability, and consistency across on-premise datacenters and cloud environments. Use our simple, yet extremely powerful UI and CLI, and experience automated canary releases, traffic shifting, routing, secure service communication, in-depth observability and more, for yourself.
About Banzai Cloud
Banzai Cloud is changing how private clouds are built: simplifying the development, deployment, and scaling of complex applications, and putting the power of Kubernetes and Cloud Native technologies in the hands of developers and enterprises, everywhere. #multicloud #hybridcloud #BanzaiCloud
Subscribe to
the Shift!
Get emerging insights on innovative technology straight to your inbox.
