Cloud computing and confidential computing solve the surge in demand for security and high availability in today’s data driven world. Various applications deal with sensitive data and need trusted execution environments (TEE) that protect specific parts of user data and code from malicious attacks and any unauthorized access. One such hardware-based memory encryption technology is Intel® Software Guard Extensions (Intel® SGX). It provides isolated and encrypted memory sections called enclaves for sensitive portions of your application. Even with root access to the system, the enclave’s memory dump would only return encrypted gibberish. For example, private encryption keys for transport layer security (TLS) communication could be created and stored within an enclave, and certificate signing could occur within that same enclave.
Most of the cloud architectures today include support for security as an integral part. When you have full control over your local system, it is straightforward to deploy Intel SGX applications. However, it is more complex to manage in large cloud-native and container-based environments. Intel developed a plugin to make this process easier. With the Intel SGX plugin, workloads can easily request and specify an encrypted chunk of memory called an enclave page cache (EPC). To manage the lifecycle of the plugin, Intel provides a certified Intel Device Plugin Operator for OpenShift Container Platform (OCP) making the task of securing your cloud-native environments much easier.
Benefits of Intel SGX
Before the advent of Intel SGX, application-specific or generalized encryption algorithms were widely used. However, most of them could not secure the application from the most privileged root or administrator access in the system. With Intel SGX, the data being actively used is secured from such privileged access. Introduced in 2015 in the Intel® Core™ processor, Intel SGX is a set of specialized CPU instructions that allocate a designated part of memory as enclaves using CPU encryption keys while maintaining integrity and restricting access. The application logic can be divided into enclave and non-enclave parts, with the former running in encrypted memory. Intel SGX is currently supported in Intel® Xeon® Scalable processors. Different amounts of EPC memory can be allocated to Intel SGX by configuring the systems BIOS. A list of CPU models that support Intel SGX is available on Intel’s Product Specifications website.
A big advantage to using Intel SGX is the ability to interact between enclaves on the same or different platforms. Attestation is used to maintain the trustworthiness between the enclaves. There are two types of attestation: local attestation and remote attestation. Local attestation verifies that enclaves on the same platform are utilizing the SGX hardware, and remote attestation is used to verify that ability on a different platform.
Intel SGX on Cloud environments
While it is simpler to manage Intel SGX applications on a system you control, it’s more difficult to control on cloud native environments. Cloud environments handle high scalability and accessibility, which are two of the most in-demand requirements for application deployments today.
Kubernetes is an open-source container platform that is one of the most popular ways to orchestrate workloads. Red Hat OpenShift Container Platform is a certified Kubernetes platform that is widely used for production workloads and has additional management capabilities. Cluster tasks such as deployment, networking, monitoring, and security are automated in OCP.
One of the ways to automate the creation, configuration, and management of Kubernetes-native applications is through OCP operators. All the application-specific objects are packaged in an operator with the help of a custom resource extending the Kubernetes API server. Without the operator, management of all the application-specific resources would require manual intervention. OperatorHub.io is a community catalog offering a collection of operators to be installed on vanilla Kubernetes.
When logged in as the administrator for the OCP, the OperatorHub can be viewed from the web console, as shown in Figure 1 below:
A big benefit of OCP is that operators are certified. Intel provides a device plugins operator that enables various Intel hardware features including the Intel SGX plugin.
Intel Device Plugins Operator
This Red Hat certified operator can be easily installed on OCP. It could be installed directly from the OCP console as shown below or through the CLI using the “oc” command according to instructions. When new versions are released, it is easy to upgrade. That is one of the advantages of the operator model. Before installing the Intel Device Plugins Operator, the node feature discovery (NFD) operator from Red Hat should be installed. The NFD operator detects detailed hardware and software features of each node in a cluster and labels those nodes accordingly. This is a necessary pre-requisite to identify which nodes have the Intel SGX capability for the Intel SGX Device Plugin to be deployed on. Once NFD is installed, its NodeFeatureRule and NodeFeatureDiscovery can be created. The custom label created for Intel SGX-supported systems follows the format intel.feature.node.kubernetes.io/sgx: ‘true’.
Now the Intel Device Plugins Operator can be installed.
After it is installed, create a “Intel Software Guard Extensions Device Plugin” instance. This runs the plugin on each node that has the Intel SGX label enabled.
Below is a screenshot of the YAML view. The default enclaveLimit and provisionLimit are each set to 110, which equates to the total number of enclaves allowed per node and the total number of enclaves a single workload can create.
Within a pod spec, sgx.intel.com/enclave is the number of enclaves the plugin will assign to a pod, and sgx.intel.com/epc is the size of those enclaves. Most workloads only need one enclave assigned, but there could be situations where multiple need to be specified.
Installing the Intel Device Plugins operator is easy on OpenShift. However, in a regular Kubernetes cluster it could be installed from OperatorHub.io, or it could be installed directly from the Intel Device Plugins for Kubernetes GitHub using yaml files. Below is a high-level flow of how things are installed and how to start a pod that can create and access Intel SGX enclaves.
This covers the basics of how to use the Intel SGX plugin to run a workload with access to the hosts /dev/sgx_enclave and /dev/sgx_provision. However, if a workload uses the Architectural Enclave Service Manager Daemon (AESMD) to provide launch support, key provisioning, and remote attestation services then a AESMD pod needs to be run on the host as well. This will create the /var/run/aesmd/aesm.socket on every Intel SGX supported node and the SGX pods with attestation needs can access it. The sample instructions on the upstream Intel Device Plugins for Kubernetes GitHub can be adapted to run within OpenShift.
Finally, if remote attestation is needed then somewhere in the cluster a provisioning certificate cache service (PCCS) needs to be running and correctly registered with a unique API key. This could be run as a pod on any node in the cluster or as a service on a dedicated node. It just needs to be accessible to all the Intel SGX capable nodes.
SELinux
In the case of Intel SGX applications, the container needs to access /dev/sgx_enclave and /dev/sgx_provision on the host. To avoid running the pod as privileged, an SELinux option “container_device_t” can be added to each Intel SGX pod spec.
Below is an example of the pod spec.
With OpenShift v4.10, these SELinux policies need to be enabled on each node. This can easily be done by adding the user-container-selinux policies to each node in the cluster. This provides fine granular access control over the host file system. This policy is integrated as a part of OpenShift v4.11 and newer. In that case, only the pod spec needs to be updated to add the SELinux “container_device_t” policy.
Interesting Use Cases
Below are some examples of how Intel SGX can be used to provide extra security where it matters.
- Trusted certificate service: This is a Kubernetes certificate signing service that supports cert-manager, Kubernetes certificate signing requests (CSR), and Istio. It uses Intel SGX enclaves to create and store private keys. In addition, it plugs into the native Kubernetes workflow for certificate signing and can sign certificates within Intel SGX enclaves. There is a trusted certificate issuer that can create multiple certificate authorities that are cluster wide or ones that reside within a namespace. It is functionally a hardware security module (HSM) for your cluster.
- Securing the 5G control plane: ACL Digital helps their clients to design innovative cloud-based products. At Mobile World Congress 2022 in Las Vegas and India Mobile Congress 2022 in New Delhi, India, they demonstrated a commercial 5G control plane on OpenShift that uses Intel SGX enclaves to store the private key used for their service mesh communication. The Intel SGX operator was used to manage the lifecycle of the Intel SGX plugin, which facilitates the assignment and creation of encrypted memory enclaves within OpenShift. This plugin, along with the Trusted Certificate Service for Kubernetes, created a more secure communication path between services. All certificate signing for this secure communication path was done within the Intel SGX enclave where even a privileged user on the node does not have access.
- SGX with 5G networking: Fortanix Confidential Computing Manager Node Agent is a Red Hat OpenShift Certified Operator. The operator offers a workflow to implement 5G applications using Intel SGX, for attestation of scaled 5G virtualized network functions (VNFs). It also supports existing applications to run as enclaves without any adjustments to the application code. The Fortanix Data Security Manager also uses Intel SGX enclaves for its Key Management Service (KMS) for security of any cryptographic material. Fortanix Data Security Manager is available as FIPS 140-2 level-3 SOC-2 certified SaaS and can help all business data encrypted and tokenized, at-rest, in-motion, and in-use. All the functionality is implemented inside Intel SGX enclaves giving a higher level of security. With these cloud technologies, 5G and networking application’s data confidentiality is ensured. To learn more about it, please read the whitepaper.
Anjuna Confidential Computing Software with HashiCorp Vault: Anjuna's software allows it to easily create multi-cloud high-trust environments where data is always encrypted, and code is verified for authenticity. With native support for Intel SGX, Anjuna allows users to build, deploy, run, and validate Intel SGX workloads without code changes or refactoring. The workflow includes secret encryption management based on HashiCorp Vault using Intel SGX supported systems. The secrets can be accessed with Intel SGX attestation services. This solution contributes towards security and access control for any application related secrets. To learn more about Anjuna confidential computing and how it uses Intel SGX with HashiCorp Vault on Red Hat Openshift, listen to this podcast or read this whitepaper.
