Project post by KubeEdge maintainers

The security of cloud native edge computing has been of concern to many users. It was difficult for users to perform effective security hardening on their edge systems due to no security threat models. Today, we are thrilled to announce that we have completed the security threat model analysis for KubeEdge with the aim to provide guidance for users and vendors, the industry’s first cloud native edge computing project. Detailed analysis on cloud native edge computing business process based on KubeEdge project and security hardening suggestions are available here.

Background

As the first cloud-native edge computing community, KubeEdge provides solutions for Edge-Cloud synergy and has been widely adopted in industries such as transportation, energy, Internet, CDN, manufacturing, and smart campus. The expanding deployment of KubeEdge poses higher requirement on the security of cloud-native edge computing.

As more and more users apply KubeEdge project in production environments, the KubeEdge community is making security a priority. The KubeEdge community attaches great importance to security and has set up Sig Security and Security Team to design KubeEdge system security and quickly respond to security vulnerabilities. To conduct a more comprehensive security assessment, the KubeEdge community cooperates with Ada Logics Ltd. and The Open Source Technology Improvement Fund to perform a holistic security audit of KubeEdge, completed with a report, including the security threat model and related security issues. Special thanks to experts Adam Korczynski and David Korczynski of Ada Logics for their professional and comprehensive evaluation on KubeEdge, which provides important insights for the KubeEdge security design. Thank you Amir Montazery and Derek Zimmer of OSTIF and Cloud Native Computing Foundation (CNCF) who helped with this engagement.

The discovered security issues have been fixed and patched to the latest three minor release versions (v1.11.1, v1.10.2, and v1.9.4) by KubeEdge maintainers according to the kubeedge security policy. Security advisories have been published here.

This article will analyze the threat model of KubeEdge, and carries out the mitigations.

Target audience

Audience who may find it useful to understand some of the risks and possible mitigations inherent in the use of KubeEdge:

KubeEdge threat model and mitigations

According to the KubeEdge security auditing report, there are the threat actors:

In the following sections, an overall system security analysis will be performed on KubeEdge based on the ASTRIDE Low Level threat modeling method and the KubeEdge security audit report, with the aim to help developers or users understand the potential security threats in the system. The content includes a list of corresponding mitigations currently available in the KubeEdge community and security reinforce recommendations for users and developers.

Attack surface of external malicious attackers

We construct the attack surface of external malicious attackers by way of the KubeEdge architectural diagram, which is shown in the following figure:

Diagram showing trust relationships in KubeEdge system architecture

As shown in the diagram above, when the trust relationship flows from low to high, there is a trust boundary, which is marked with sequence numbers in the diagram. The following will analyze in detail the trust boundary in the KubeEdge system architecture and give a list of corresponding mitigations currently available in the KubeEdge community and security hardening recommendations.

Threat ID 1: CloudCore connection to EdgeCore

Description

EdgeCore has a separate level of authority in the overall system, in that an attacker in control of EdgeCore should not be able to negatively affect other edge nodes, e.g. by way of manipulation of CloudCore. In this sense the trust relationship flows from low to high in that EdgeCore has lower authority over KubeEdge than CloudCore, and CloudCore should be considered the highest level of trust in the KubeEdge ecosystem (Reference from the security audit report under section `KubeEdge attack surface enumeration`).

Impact

The attacker maliciously modifies the traffic packets between the CloudCore and EdgeCore, resulting in the threat of spoofing and tampering.

Mitigation

Recommendations

Threat ID 2: HTTP server connection to EdgeCore

Description

The HTTP server in EdgeCore exposes EdgeCore locally on the system and the applications on the local system do not have a high level of authority over the entire KubeEdge cluster. In this context, EdgeCore should be protected against attempts from the local system to thwart KubeEdge. In this sense, the trust relationship flows from low to high in that the local applications have lower authority over the KubeEdge ecosystem than EdgeCore (Reference from the security audit report under section `KubeEdge attack surface enumeration`).

Impact

The data received by the ServiceBus component is often not controlled by the control plane, and attackers can launch malicious packet attacks on the ServiceBus component, resulting in the risk of spoofing and tampering in the communication process. ServiceBus exceptions cause service failure only on a single edge node and will not crash the entire KubeEdge system.

Mitigation

Recommendation

Threat ID 3: MQTT broker connection to EdgeCore

Description

The Devices can reach EdgeCore by way of the MQTT broker. There is a trust boundary here, with which the Devices themselves should not be able to negatively affect the overall KubeEdge system. For example, if an attacker controls a device, the overall system should be protected against possible attacks from this device. In this sense, the trust relationship flows from low to high in that the devices have lower authority over the KubeEdge system than EdgeCore (Reference from the security audit report under section `KubeEdge attack surface enumeration`).

Impact

The data received by the EdgeCore component is often not controlled by the control plane, and attackers can launch malicious packet attacks on the EdgeCore component, resulting in the risk of spoofing and tampering in the communication process. EdgeCore exceptions cause service failure only on a single edge node and will not crash the entire KubeEdge system.

Mitigation

Recommendation

Threat ID 4: Edged manages and monitors K8s resources

Description

Edged handles the running of Pods. The containers in these pods have low authority over the KubeEdge system and privileges thus flow from low to high. An attacker in control of some components in the containers should not be able to grow a foothold of more of the KubeEdge system (Reference from the security audit report under section `KubeEdge attack surface enumeration`).

Impact

If the CRI plugin accessed by Edged is maliciously forged by an attacker, there is a threat of spoofing and tampering of the CRI plugin, resulting in abnormal local services.

Mitigation

Recommendation

Threat ID 5:MetaServer provides HTTP services at edge nodes

Description

The MetaServer starts an HTTP server and acts as an edge api-server for Kubernetes operators. It is a highly trusted piece of the KubeEdge architecture. This makes it susceptible to attention from attackers that would seek to control the operators communicating with the MetaServer (Reference from the security audit report under section `KubeEdge attack surface enumeration`).

Impact

The data received by the MetaServer component is often not controlled by the control plane, and attackers can launch malicious packet attacks on the MetaServer component, resulting in the risk of spoofing and tampering in the communication process. MetaServer exceptions cause service failure only on a single edge node and will not crash the entire KubeEdge system.

Mitigation

Recommendation

Attack surface of internal malicious attackers

Possible risks from internal malicious attackers include issues accidentally caused by administrators or operators when deploying malicious deployments into the cluster, performing high-risk configurations, and so on.

Mitigation

Recommendation

Attack surface of supply chain attackers

Supply chain attacks are an ever-present threat, exploiting weak points to interfere with software. Attacks can occur at every link in a typical software supply chain, which puts an extra burden on anyone involved in complex critical systems.

Mitigation

Recommendation

Recommendations

Recommendation ID 1 – Encrypt with a private key of a safe length

Description

It is recommended that users generate a private key of at least 2048 bits, encrypt and store the private key locally, and set the private key file with minimal access permissions as read-only to the owner.

Recommendation ID 2 – Use a CA certificate from a trusted source

Description

It is recommended that the certificates be issued from a mutually trusted Certificate Authority (CA).

Recommendation ID 3 – Strictly restrict user login permissions of edge nodes

Strictly limit the local permissions of edge nodes, limit the user login permissions of external users, and reduce unnecessary authorization.

Recommendation ID 4 – Strictly restrict permissions for app deployment

Description

Strictly limit the authority of edge node application deployment. Only system services and administrators are authorized to deploy privileged containers.

Recommendation ID 5 – Strictly verify the validity of the application image

Description

Strictly verify the validity of application images when deploying applications on edge nodes to prevent malicious applications from being deployed.

Recommendation ID 6 – Strictly restrict the access of edge devices

Description

Strictly review the devices connected to edge nodes to avoid unnecessary device access.

Recommendation ID 7 – Strictly review the configuration of CRI plugins on edge nodes

Description

Strictly review the configuration of the CRI plug-in, and use the official recommended plug-in configuration.

Recommendation ID 8 – Strictly divide the permissions of each role in the system

Description

Strictly divide the permissions of each role in the system, and implement fine-grained control of system role permissions through RBAC, OPA, etc.

Recommendation ID 9 – Follow up and update the KubeEdge version in time

Description

Vulnerability fixes will be merged in the latest 3 minor release versions, and it is recommended that users follow the community security advisory to obtain vulnerability disclosure information, and follow up and update the KubeEdge version in time.

Recommendation ID 10 – Strictly verify the binary files according to the verification files released by the community

Description

Before taking the binary files released by the community, users should strictly verify the binary files with the verification files (file format: checksum_{binary file name}.txt). The release package address is here.

Recommendation ID 11 – Achieve SLSA compliance in your projects

Description

In the process of building from source code, users or vendors should refer to the SLSA to improve integrity, and secure packages and infrastructure in your projects, businesses or enterprises.

Appendix

Important Links