Implementing a real time vulnerability management system for containers

Securing Kubernetes environments remains a major challenge, which means you need a vulnerability management system that can provide you code to cloud coverage. In recent years we’ve seen many container vulnerabilities, data breaches, and plain misconfiguration that had severe business implications. It’s time to step things up! 

A real-time vulnerability management system to reduce container vulnerabilities

Many development and DevOps teams create their container-based infrastructure locally before pushing into a repository. It’s incredibly common to set up Docker on a local workstation, yet if an attacker can take over that workstation, it becomes an immediate avenue for exploitation. 

There’s been staggering growth in use of containers and container technology recently, which means there’s also been a growth in container vulnerabilities. "Every time there’s a widespread adoption in technologies, there are bugs. Container technology is no different." 

These are just some examples of container vulnerabilities container runtime engines and cloud service platforms have experienced:

• Docker Race Condition: CVE-2018-15664 gave root file system access
• Docker Container Escape: CVE-2019-5736 for runc runtime is a full escape
• Docker Hub database breach affected over 190,000 accounts in April 2019

There has been a recent escalation in attacks on container technology as is seen here: It will come as no surprise then that we need better visibility in container management, auditing needs to be turned on and we need a better model on detection and response capabilities. Many containers only run on a short timeframe, frequently less than an hour. The highly ephemeral nature of the way that container-based infrastructure is built, deployed, and operated doesn’t work with traditional security systems we’ve had in the past. As organizations come to rely more and more on containers and Kubernetes, they’ll need to put processes in place to vulnerability scan container images, secure orchestration tools, and leverage secure code repositories "all in real-time."

The problems with the ‘traditional’ and ‘current’ approach to container security

There are three different approaches to container security – the traditional approach, the one currently used by the masses, and Panoptica's, Cisco Cloud Native Application Security Solution, approach. The traditional approach relies on tightening the container’s code to address vulnerabilities and sandboxing images and packages. During deployment and runtime, the configuration includes defined and controlled access privileges, including on volume and OS/Host. This then needs to be incorporated into the orchestration side by applying RBAC/API authorization and Secrets. There are significant issues with the traditional approach such as not being able to fine-tune roles and not maintaining networking security sufficiently. The current approach involves hooking the vulnerability scanning into the CI/CD. It is a good solution because it enables the developers to get feedback but it is not enough. 

Flaws in the approach include:

• It requires preliminary integrations, which someone needs to take responsibility for
• It relies on external repositories & Sidecars

• New Vulnerabilities can be found after deployment, which leads to a continuous struggle to keep the system secure

• The Kubernetes Infrastructure itself may be left vulnerable e.g. – For example there were 2 new Kubernetes vulnerabilities discovered in June, CVE-2020-10749 (container networking plugins), and CVE-2020-8555 (kube-controller-manager), which were only fixed in versions 115/116 or above.

• Constant monitoring of Kubernetes clusters is critical

Pods deployment

When containers are under attack, the attacker often moves from the breached container to the host to gain exposure to other containers that share the same host. To stay protected, it’s vital to have a comprehensive pod security context in place. It enables users to set up the needed configurations that minimize the blast radius of an attack. The Kubernetes Pod Security Policy is a cluster-level resource that controls security-sensitive aspects of the pod specification. The PodSecurityPolicy objects define a set of conditions that pods must run with to be accepted, as well as defaults for the related field.

Network policies

Network policies reduce the impact of potential breaches by limiting exposure. They can be used to:

• Create networking segments in the cluster by restricting communication between applications

• Control the incoming and outgoing traffic – eliminating malicious activities (e.g. crypto-mining) A great way to help network security is to use a service mesh, such as Istio. Istio allows users to create policies based on Kubernetes Services (not pods) and offers integrated options to solve all exceptions. Ingress and Egress policies are simpler with Istio. Istio allows the Kubernetes network policies to be expanded to include non-containerized elements. A newer approach is called for, one that is able to secure containers '''in runtime''' without leaving Kubernetes vulnerable to attack, without relying on external repositories, and without relying on pre-deployment measures.

Introducing Panoptica

To tackle this challenge, we created Panoptica, Cisco Cloud Native Application Security Solution, an open source runtime vulnerability scanner tool for Kubernetes. 

The new approach, based on open-source Kubei, tackles security "from a runtime perspective." The big change to the traditional container security approach is that instead of relying on CI/CD security, it scans "all runtime images," regardless of their origin, and without requiring integration.

• Detects malicious pods and provide an extra layer of security to your cluster
• Scans accessed public images not hosted in your registry
• Gains visibility into Kubernetes system elements

• Runs inside the cluster and provides real-time assessment of deployed containers’ vulnerabilities and cluster’s health

Advanced risk protection

To help you defend your containers against vulnerabilities and build a real-time vulnerability management system, Panoptica’s approach adds a new security check step that’s baked into the CI/CD pipeline and runtime. It assesses the risk potential of a container prior to and during deployment. 

By reviewing the Role-Based Action Control (RBAC) permissions and examining the roles specified in the deployments/statefulsets. It eliminates risks before they occur. Panoptica carries out a runtime audit on all APIs invoked towards the API server that have an associated risk. It blocks commands that create risks or violate security best practices and it detects abnormal scenarios. By doing this it adds an extra level of security, keeping your Kubernetes safe and secure.

Try Panoptica today

Navigate complex cloud risks and vulnerabilities, identify and prioritize critical attack paths, and quickly remediate them with Panoptica’s cloud native application security platform (CNAPP). Request a demo.