Kubernetes (K8s) is the universal solution for container orchestration nowadays. This open-source tool allows a cluster to automatically scale, distribute, and handle containers for several different workloads. K8s can be considered a cloud operating system; it thus prevents vendor lock-in and is not dependent on how each cloud provider implements its offering.
As popular as Kubernetes has become, the platform introduces new risks that users should be aware of. First, because it is so widely used, any vulnerabilities will have higher visibility, and so K8s is more targeted than other services. Another aspect that should be considered is that Kubernetes manages all of the clusters and thus a great deal of sensitive information.
As a set of machines, a cluster can host multiple services from multiple teams, which can attract attackers and increase the impact of a compromised cluster. In order to prevent attackers from compromising the cluster, security teams must collaborate with developers and DevOps to guarantee every deliverable is properly secured and, on the other hand, implement the right security controls and tools to secure the cluster as a whole.
For any production system, security must be a top priority, especially if it is exposed to the internet or to many users. Some attack vectors include compromised valid credentials, misconfiguration exploitation, and software vulnerability exploitation. Kubernetes is a complex system made up of several subsystems (some of which are much older than Kubernetes itself) that must be correctly configured and patched.
Measures for Securing Your Cluster
Keeping Containers Safer
Kubernetes is a container manager. It is therefore crucial that you first and foremost ensure your containers are secure. Absent of this, it will be impossible to keep all other components secure. Containers are a mature technology, and there are a number of established best practices for ensuring their security:
- Single application per container: Docker—and many container tools—are designed to keep just one application per container. While there are indeed a number of ways to bypass this, running multiple services in a container will increase the container’s complexity and surface of attack.
- Opt for Alpine or distroless base images: These images don’t come with additional tools, which can pose security risks.
- Opt for official public images: If using public images, it is best to use official ones when possible. If this is not a possibility, avoid using labels and use the SHA identifier instead. Labels may be used for a new image version, which can introduce bugs or security issues that can go undetected.
Keep Your Kubernetes Instance Updated
When a new software version is released—especially open source—this places those organizations still using the older version at risk, as attackers scramble to discover what vulnerabilities were solved with the new release and thus which still exist in older versions. The security personnel of these vulnerable organizations must therefore address these issues before attackers take advantage.
For this reason, it is crucial that you update your K8s instance as soon as a new patch is released. The cluster, however, does not need to be updated to the latest major version, since older major versions get security patches too.
According to Kubernetes, fixes—security fixes included—may be backported to the three most recent branches depending on the severity of the vulnerability. If the K8s version used is no longer supported, updating to the major version as soon as possible is recommended.
Not only must the Kubernetes instance always be up to date; all of the services running within the platform must too. All container layers should be updated (e.g., Java and application server versions) as well as software dependencies to avoid security breaches.
Secure the Kubernetes API
Kubernetes API should be private. Do not expose it to the public internet and set up authentication.
RBAC to Limit Access
RBAC, or role-based access control, is a set of permissions that grants users access to restricted resources. The permissions for each and every Kubernetes resource should be carefully selected so that only those who need it have access (write or read) to the resource.
The majority of cloud providers allow cloud user permissions to be shared with K8s, which simplifies user access management since they can all be kept in one place only.
Limit Outages with Resource Quotas
Even the most diligent operator may experience outages due to bugs or security issues. Creating resource quotas can limit the scale of an outage by restricting the buggy application. Without these restrictions in place, a service might use all of the cluster resources, making it difficult to pinpoint which service is preventing the cluster from working properly.
Kubernetes resource quotas can limit CPU, memory, and storage resources by many types of resources, such as pods, services, namespaces, and more. Namespace resource quotas allow you to create constraints that limit resources of a set of services and applications (like a specific project, for example) and can be useful for a multi-project cluster with shared resources.
Not only do quotas ensure cluster health, they also allow for a more predictable monthly bill.
PodSecurityPolicy is a feature every pod has that allows fine-grained permissions for all pods deployed in the cluster. It can be integrated with RBAC so that only designated users can set these policies, for example. While the policies are vast, they can allow a pod to work in privileged mode (where the container has access to the host resources), host policies, file system policies, and more.
Only applications that need to communicate should be allowed to send network packages to each other. Kubernetes enables the creation of network policies to quickly allow only particular services to communicate with others.
A new type of service known as service meshes is now being offered to address several network use cases. These work by creating an abstraction layer between the network and the software running. A service mesh will therefore be responsible for handling network security such as data encryption and access policies as well as service discovery and more advanced techniques such as circuit breakers.
Service meshes create a new network layer that uses end-to-end encryption between users, which prevents anyone on the network from eavesdropping.
Secrets are any sensitive information that must be properly secured and known by the least number of people possible. Secrets should not be hard coded or appear in the configuration files. If a tool requires secret information (e.g., a database password or SSL private key), this data must be transmitted through secure means, such as through environment variables, encrypted files, or configuration services.
Kubernetes provides an abstraction layer that provides access to Secrets so that operators can securely store sensitive information. Though Kubernetes Secrets encrypts the data, this may not be sufficient for meeting enterprise compliance requirements. In such cases, a more advanced tool (e.g., AWS KMS, Azure Key Vault, or HashiCorp Vault) can be plugged in for added security.
Each node machine running inside the cluster must be properly secured. Basic configurations should be applied, including:
- Limiting network access to the host by using firewalls.
- Disabling interactive users (The machines should be created and destroyed by Kubernetes and no human should have access to it.). If accessing a node is required, the user should only be granted the necessary permissions and only for the amount of time required.
- Keeping OS images updated and applying all security patches ASAP.
Cloud Workload Protection Platform (CWPP)
CWPPs, like Intezer Protect, are tools designed to proactively protect cloud workloads at runtime. By monitoring all running code and activity, CWPP helps reduce the attack surface, detect breaches when they occur, and respond to incidents. These tools provide visibility and control to the entire cloud stack, including container environments and traditional workloads.
Monitoring and Logging
Monitoring and logging can provide insights about an outage or even discover a breach early on. Logs are also difficult to configure, as they can be divided into three types:
- Cluster logs: These are the logs written by Kubernetes itself. They inform about cluster health and cluster resource usage trends.
- Container logs: Kubernetes and Docker provide information about running containers, such as CPU, network, and memory usage. This history of usage makes it possible to detect anomalies that could potentially be security issues.
- Application logs: Application logs consist of customized data intended to provide insights about the application and its rules.
Fast Response to Attacks
In the case of a security outage, should all other security measures fail to prevent this, the responsible security team needs to have the right tools and controls to investigate and mitigate the attack.
Developing a new product with the help of a leading technology such as Kubernetes is a major investment. But without proper security measures in place, all that hard work can go to waste.
Securing a cluster with so many moving parts can be a serious challenge, and can take time away from your core business. Security is an ongoing job that is constantly becoming more complex and nuanced. Implementing the security measures outlined in this article combined with constant monitoring, keeping up to date with best practices (such as official documentation), and patching will ensure your Kubernetes instances remain as secure as possible.