Outshift Logo


7 min read

Blog thumbnail
Published on 06/30/2020
Last updated on 05/03/2024

Pod level external HTTP(S) proxy configuration with Istio


Companies frequently use proxies to act as a link between an internal network and the Internet. This is often frustrating for employees, even non-IT ones, when they can't access a specific site from the company network. For engineers it's even more obnoxious, since they have to configure all kinds of compute infrastructure to connect to external networks via these proxies. It's debatable if this is the best way to harden corporate network security, but it's still the most widely spread method to restrict outgoing traffic. The intentions behind introducing a proxy are usually good, as the main reason is almost always to prevent data breaches by adding an additional layer of security. Proxies come with other useful features, however. They can audit network traffic, filter undesired content or even act as a cache. This post will talk about corporate proxies from an infrastructure engineer's standpoint, when they have to deal with the configuration in an environment including Kubernetes and Istio.


When utilizing dynamic services that run on Kubernetes and Istio, a proxy requires an increased level of configuration automation for all the components running behind it. We're going to give a brief overview of how Backyards (now Cisco Service Mesh Manager) helps with the pod-level configuration of these settings in the Kubernetes cluster.
Backyards (now Cisco Service Mesh Manager) is the Banzai Cloud Istio distribution that just works. Backyards (now Cisco Service Mesh Manager) operationalizes the service mesh to bring deep observability, convenient management, and policy-based security to modern container-based applications.
istio proxy

Proxying on Kubernetes

The proxy configuration for client applications usually uses environment variables, such as HTTP_PROXY , HTTPS_PROXY, and NO_PROXY. Let's assume we have the following proxy addresses: If that were the case, the following environment variables would need to be set in the YAML spec of a pod running on the Kubernetes cluster:
  - env:
      - name: "HTTP_PROXY"
        value: "http://proxy.example.com:3128"
      - name: "HTTPS_PROXY"
        value: "http://proxy.example.com:3129"
      - name: "NO_PROXY"
        value: "svc,local"
Manually inserting those variables into each and every pod would be far too tedious and error-prone a process, and would only work in a very small cluster anyway.

Admission webhooks to the rescue

The better option to inject these environment variables to a pod's containers is to use an admission webhook. Since Backyards (now Cisco Service Mesh Manager) already has a webhook to inject Istio sidecar proxies into pods, we can leverage and extend the existing webhook functionality to support proxy environment variable injection as well. Backyards (now Cisco Service Mesh Manager) uses the open source Banzai Cloud Istio operator under the hood, which provides support for exactly the kind of feature we were just talking about; we can specify the settings of the external proxy in the Istio custom resource and Backyards (now Cisco Service Mesh Manager) will take care of the rest. The corresponding Istio custom resource configuration is as follows:
    httpProxy: "http://proxy.example.com:3128"
    httpsProxy: "http://proxy.example.com:3129"
    noProxy: "svc,local"



  1. Create a Kubernetes cluster.
    You can create a Kubernetes cluster with the Banzai Cloud Pipeline platform on-prem, or over any one of five different cloud providers.
  2. Point KUBECONFIG to your cluster.
  3. Install Istio with Backyards (now Cisco Service Mesh Manager):This step usually takes a few minutes, and installs Istio and Backyards on your cluster.

Deploy an HTTPS proxy

Let's deploy a simple proxy inside the cluster to showcase external proxy usage for this example. Also, we're going to use the pod IP address to make the scenario more realistic. This example uses Tinyproxy, but any proxy that supports  HTTP CONNECT will do.
  1. <create an="" external="" namespace
Create a new namespace without the Istio sidecar injection enabled to simulate the proxy being outside of the cluster.
$ kubectl create namespace external namespace/external
  1. Deploy Tinyproxy
    $ kubectl apply -f - <<EOF
    apiVersion: apps/v1
    kind: Deployment
      name: tinyproxy
      namespace: external
      replicas: 1
          app: tinyproxy
            app: tinyproxy
          - image: vimagick/tinyproxy
            imagePullPolicy: IfNotPresent
            name: tinyproxy
  1. Obtain the IP address of the proxy pod
Store the proxy pod IP in the PROXY_IP environment variable for later use.
$ export PROXY_IP=$(kubectl get pod -n external -l
app=tinyproxy -o jsonpath='{@.items..podIP}')

Configure the proxy settings

  1. Set the proxy settings in the Istio custom resource
The default Tinyproxy container will listen on port 8888 , so that's the port that we'll use.
$ kubectl -n istio-system patch istio mesh --type=merge --patch "
    httpProxy: "http://$PROXY_IP:8888"
    httpsProxy: "http://$PROXY_IP:8888"
    noProxy: "svc,local"

Deploy a simple pod to check outgoing connections

  1. Label the default namespace for automatic sidecar injection
$ backyards sp ai on default INFO[0002] auto sidecar
injection successfully set to namespace default
  1. Use the following snippet to create a test pod.
    $ kubectl create -f - <<EOF
    apiVersion: v1
    kind: Pod
      name: curl-test
      - name: curl
        image: curlimages/curl:7.70.0
        command: [ "/bin/sh", "-c", "--" ]
        args: [ "while true; do sleep 3000; done;" ]
  2. Check the environment variables of the created pod to see if the automatic injection worked.
    $ kubectl -n default get pods curl-test -o yaml |grep "env:" -A 12|head -13
        - name: http_proxy
        - name: HTTP_PROXY
        - name: https_proxy
        - name: HTTPS_PROXY
        - name: no_proxy
          value: svc,local
        - name: NO_PROXY
          value: svc,local
    The HTTP(S) proxy environment variables were automatically injected to the pod, and the variables are set in both uppercase and lowercase.
  1. Send a request from the curl-test pod in the default namespace to an external service.
    $ kubectl exec -it curl-test -c curl -- sh -c "curl http://httpbin.org/headers"
      "headers": {
        "Accept": "*/*",
        "Host": "httpbin.org",
        "User-Agent": "curl/7.70.0-DEV",
        "X-Amzn-Trace-Id": "Root=1-5ef23da7-4c7406e8ef392f32b61d7ddb"
  1. Check the access logs of the proxy for your request
    $ kubectl -n external logs -l app=tinyproxy |grep CONNECT
    CONNECT   Jun 23 17:36:27 [10]: Established connection to host "httpbin.org" using file descriptor 10.
    CONNECT   Jun 23 17:36:38 [6]: Connect (file descriptor 9): ip-10-20-1-173.eu-west-1.compute.internal []
    CONNECT   Jun 23 17:36:38 [6]: Request (file descriptor 9): GET http://httpbin.org/headers HTTP/1.1
    CONNECT   Jun 23 17:36:39 [6]: Established connection to host "httpbin.org" using file descriptor 10.
    Kubernates Cluster

The future: transparent external proxy with Envoy

An even better solution for the use case we've discussed would be to make it so the injected Envoy sidecar transparently supported routing through an external proxy. That way it would be totally unnecessary to configure via environment variables or provide support in the client application. There is ongoing work to support such a feature in Envoy, and hopefully it will land in the near future. The CONNECT support is already merged , and there is an open issue about CONNECT support for HTTP/1.1 upstream as well. Once these features are merged and released upstream, Backyards (now Cisco Service Mesh Manager) will switch to its Envoy-based method of configuring external proxies. 'Til then, happy proxying.
Subscribe card background
Subscribe to
the Shift!

Get emerging insights on emerging technology straight to your inbox.

Unlocking Multi-Cloud Security: Panoptica's Graph-Based Approach

Discover why security teams rely on Panoptica's graph-based technology to navigate and prioritize risks across multi-cloud landscapes, enhancing accuracy and resilience in safeguarding diverse ecosystems.

the Shift
emerging insights
on emerging technology straight to your inbox.

The Shift keeps you at the forefront of cloud native modern applications, application security, generative AI, quantum computing, and other groundbreaking innovations that are shaping the future of technology.

Outshift Background