Intel Trust Domain Extension Research and Assurance

ID 777394
Updated 4/24/2023
Version 1.0
Public

Intel TDX Security Assurance

  • Intel and industry partners collaborated on advanced security assurance actions to ensure Intel TDX provides robust security.

  • Security assurance actions included hackathons, offensive security research, enhanced SDL, and more.

author-image

By

Executive Summary

Intel® Trust Domain Extensions (Intel® TDX) introduces new architectural elements to help deploy hardware-isolated Virtual Machines (VMs) called Trust Domains (TDs). Intel TDX is designed to isolate TDs from the virtual-machine manager (VMM)/hypervisor and any other non-TD software on the platform to protect TDs from a broad range of software attacks. Intel TDX is one of Intel’s key Confidential Computing technologies.  

This article describes the advanced security assurance actions and the extra focus Intel has dedicated to assuring Intel TDX provides robust security. We describe how, with Intel TDX, we went  above and beyond the standard Security Development Lifecycle (SDL) requirements that apply to all Intel product development. SDL is a set of tasks and activities required at each stage of our product definition, architecture, design, development, and security validation to ensure a comprehensive security assurance approach to product development at Intel.    

In addition to the standard SDL requirements, the advanced security actions we took for Intel TDX include:

  1. Opened the source code of the Intel TDX Module and Secure Arbitration Mode (SEAM) Loader for external audit to enable public reviews and research. We continuously receive and address inputs from the external community as part of development of these software modules.  External developers can find the steps to download the code and instructions for reproducing the builds below:
  2. Offensive Security Research. We performed security analysis of Intel TDX products by applying an “outside in” approach; we approached the code with an attacker’s mindset to complement the conventional SDL practices. The security researchers conducted architectural reviews, emerging threat analysis, design reviews, and RTL reviews of key hardware IPs in the Intel TDX flows. Additionally, researchers participated in five hackathons against MCHECK, the Intel TDX Module, Intel TDX SEAM Loader, Linux* software stack, and end-to-end Intel TDX flows at the platform level. These efforts represent four person-years of security research and yielded dozens of vulnerabilities and more than a dozen defense-in-depth and architecture/design improvements, which were all implemented in 4th Generation Intel® Xeon® Scalable Processors code named Sapphire Rapids (SPR). As part of this proactive research, Intel is continuously researching side channel and transient execution attack scenarios and proactively providing controls and options for the latest technologies, including Intel TDX. Guidance for TD developers is documented and periodically updated at the Intel TDX Specifications home.
  3. Collaboration with customers and the external security community. Getting customers’ and third-party security researchers’ perspectives were another critical piece of our advanced security assurance for Intel TDX. In this area we collaborated with the following external teams:
    • Security review with Google*. The primary goal of Intel’s engagement with Google was to provide assurances that cloud customers and providers can confidently use Intel TDX. A secondary goal was to provide a better understanding of the expected threat model for Intel TDX and to identify limitations in its design and implementation that would better inform Google's deployment of the technology. The security reviews involved five security researchers from Google Cloud Engineering and Google Project Zero over a period of six months in 2022. Together, we evaluated and verified 81 attack scenarios, and identified nine confirmed security vulnerabilities, one BIOS documentation issue, and five defense-in-depth, all of which were mitigated in the 4th Generation of Xeon Scalable Processors. The security engagement met both Intel’s and Google’s goals, and together both companies released a joint public report on Intel TDX to describe the security collaboration in detail.  
    • Security review with NCC Group*. Intel obtained NCC’s security review service for the Intel TDX Module, which consists of approximately 20,000 lines of code. The reviews focused on the areas below and resulted in no new vulnerabilities being found in the reviews:
      • Assess the Intel TDX Module implementation, ensuring all security objectives are met.
      • Review mitigations for known side channel attacks, as well as for other known vulnerabilities.
      • Perform a general secure coding assessment to ensure security posture and hygiene.
      • Review cryptographic constructions for correct algorithm usage and secret handling.
    • Invited security reviews through Intel Project Circuit Breaker under the codename Trusted Crossings. This project, conducted between July and December of 2022, invited 12 external researchers to receive exclusive Intel TDX training and access to early Intel TDX-enabled platforms to “hack” the product and earn bounties for any issues they found. Every week, the researchers submitted various attack scenarios they attempted to Intel, and Intel engineers responded. No significant security concerns were discovered during this project; only two functional issues were found, both of which were addressed prior to the release of the 4th Generation Intel® Xeon® Scalable Processors.  

Intel TDX Architecture

Intel TDX refers to an Intel technology that extends Virtual Machines Extensions (VMX) and Multi-Key Total Memory Encryption (MKTME) with a new kind of VM guest called a Trust Domain (TD).  

Intel TDX 1.0 Architectural Overview

A TD runs in a CPU mode that is designed to protect the confidentiality of its memory contents and its CPU state from any other software, including the host VMM or hypervisor, unless the memory contents are explicitly shared by the TD itself.

The Intel TDX solution is built using a combination of Intel’s VMX and TME-MK, as extended by the Intel® Trust Domain Extensions Instruction Set Architecture (Intel® TDX ISA). To enforce the security policies for the TDs, a new CPU mode called Secure Arbitration Mode (SEAM) is introduced as an extension to the VMX ISA to host an Intel-signed security services module called the Intel TDX Module. 

The platform is managed by an Intel TDX-aware host VMM. As shown in Figure 1 below, a host VMM can launch and manage both guest TDs and legacy guest VMs. The host VMM maintains all legacy functionality from the legacy VMs’ perspective; it is restricted only with regards to the TDs it manages.

Figure 1: Intel Trust Domain Extension Architectural Overview

 

For more architectural details, refer to “Section 2: Overview of Intel® Trust Domain Extensions” of the Intel® TDX Module 1.0 Specification.

Intel TDX Module and Intel TDX Transitions

The Intel TDX Module is the Intel-signed security services module responsible for enforcing the security policies for TDs. The Intel TDX Module is designed to provide two main new logical modes of operation built upon the new SEAM root and SEAM non-root CPU modes that have been added to the Intel VMX architecture: Intel TDX Root Mode, and Intel TDX Non-Root Mode. Figure 2 below shows the Intel TDX logical modes and transitions (in red) on top of the CPU architectural modes.

Figure 2: Overview of Intel TDX Modes and Transitions based on VMX and SEAM Modes and Transitions

 

Intel TDX transitions between TDX root operation and TDX non-root operation include TD Entries, from Intel TDX root to Intel TDX non-root mode, and TD Exits from Intel TDX non-root to Intel TDX root mode.  A TD Exit might be asynchronous, triggered by some external event (for example, an external interrupt or system management interrupt (SMI)) or an exception, or it might be synchronous, triggered by a TDCALL (TDG.VP.VMCALL) function.

For more details, refer to “Section 2.1 Intel TDX Module Lifecycle” of the Intel® TDX Module 1.0 Specification.

Intel TDX Security Objectives and Threat Model

This section provides a very high-level overview of Security Objectives (SO), Threat Model (TM) and TCB (Trusted Computing Base) definition of Intel TDX Module 1.0. Please note that this is not a comprehensive list of SO and TM, but rather is intended to give a glimpse of our security research approach to identify the high-risk areas, assess the threat model around them, and to conduct security research activities.

Primary Security Objectives for Intel TDX

  • Offer Confidentiality and Integrity protections to Trust Domains.
  • SEAM (SEcure Arbitration Mode): A new Intel x86 CPU Mode to support Intel TDX.
  • Per-domain memory integrity:
    • Mitigate software replay attacks on TD.
    • Protect against modification, relocation and cross-domain corruption by software.
    • Simple hardware attack protection.
  • The Intel TDX Module sets the ephemeral key for TD Key IDs
    • Keys for TD KeyIDs can be set only in SEAM.

The Primary Threat Model includes the attacker’s key motivations:

  • Primary motivation:
    • Defeat the main security objective of Intel TDX, which is to protect the guest TD’s confidentiality and integrity.
  • Other Motivations:
    • Privilege escalation: run arbitrary code in SEAM root mode.
  • Attackers may be:
    • Malicious host VMMs.
    • Malicious guest TD, probably colluding with a malicious host VMM.
    • Other software such as legacy VMs, or firmware, mostly for software side channels.

Example Threat Model and Security Objectives

Table 1: Threat Model for Guest TD Confidentiality and Integrity
SO Category Asset Protection Required Actor Attack Point Technique

Protect TD’s confidentiality and integrity.

Secure EPT Integrity, Confidentiality VMM Intel TDX Module as confused deputy. Use SEAMCALL API to modify state in an architecturally incorrect way.
Use the CPU state (for example, MTRR) to impact Intel TDX Module operation.
Mitigation:
Mitigations of attacks using the TDX Module APIs    
Protect TD’s confidentiality and integrity. Secure EPT Confidentiality, Integrity TD Intel TDX Module as confused deputy Use TDCALL API to leak state or to modify state in an architecturally incorrect way.
Use the CPU state (for example, XCR0) to impact Intel TDX Module operation.
 
Mitigation:
Mitigations of attacks using the Intel TDX Module APIs                      
Protect TD’s confidentiality and integrity. 1: TD saved VCPU state
2: TD meta-state (per-TD and per-VCPU)
3: TD memory
 
Confidentiality, Integrity Any software Intel TDX Module as confused deputy Use SEAMCALL or TDCALL API to leak or modify state.
Use the CPU state (for example, MTRR) to impact Intel TDX Module memory access
 
Mitigation:
Mitigations of attacks using the Intel TDX Module APIs                  

 

Table 2: Threat Model: Intel TDX Module Confidentiality and Integrity
SO Category Asset Protection Required Actor  Attack Point Technique
Protect TDX Module code and state. Code and data in SEAMRR-protected memory Confidentiality (TD-related data only), 
Integrity
1: Any software
2: DMA
 
Direct access to memory Attempt to directly access memory
Mitigation: Provided by SEAM arch using Direct Access to Memory
Protect TDX Module code and state. Code (in SEAMRR-protected memory) and data Confidentiality (TD-related data only), Integrity Any software Intel TDX Module as confused deputy Use the CPU state (for example, XCR0) to impact Intel TDX Module operation
Mitigations:
  • Mitigations of attacks using Out-of-SEAM CPU state.
  • Applicable to MSRs and control registers that can be locked.
  • Checks and locks done before TDX Module initialization.
  • Checks done at TDX Module initialization time.
  • Intel TDX Module initialization APIs verify some CPU state that has not been verified before.
  • Check that SMRR/SMRR2 which configured and locked don’t overlap with any Convertible Memory Range (CMR).

 

Intel Security Development Lifecycle Overview

The objective of the security development lifecycle (SDL) is to ensure a trustworthy product is delivered and supported throughout its lifecycle. The SDL is a set of tasks and activities that can drive high-quality security outcomes in product and services development at Intel. SDL is Intel's approach to make security and privacy an integral part of our product definition, design, development, validation and post-release support. SDL integrates with the Intel corporate product life cycle (PLC) process to ensure that Intel products meet Intel security and privacy requirements.

Intel SDL Activities Summary

The SDL ensures that security becomes an integral part of our product design, which consists of the following phases:

Figure 3: Intel SDL Activities

Intel TDX 1.0 SDL Activities Summary

Key Tasks

  • Security Architecture and Threat Model Review.
  • Design review of Intel TDX ingredients at hardware IP and CPU Core.
  • Security code and architecture review of firmware and software ingredients.
  • Penetration and targeted testing. 
  • Provide ongoing security consulting to internal business partners.

Methodology

The Intel TDX firmware and software components were developed based on security-focused coding guidelines and design rules. A large part of the functional validation focused on the negative space with multiple coverage metrics being used: such as code, input space, and multi-threading space. On top of that, a strict SDL activity was applied along the multi-year effort to define, develop and validate Intel TDX.

  • Reviewing the specs and highlighting potential ways in which an attacker can violate a security objective.
  • Reviewing recent CVEs in related products\technologies and their relevance to Intel TDX.
  • Conducting brainstorming sessions with functional validation teams and other stakeholders regarding potential attack vectors.
  • Performing a deeper analysis of code that was identified as potential risk during the security code review phase.
Figure 4: Intel TDX SDL Methodology


 

Intel Offensive Security Research Overview

The primary objective of the Offensive Security Research (OSR) team is to evaluate Intel’s high risk security technologies and products by applying an “outside in” approach with an attacker mindset and thereby complement the conventional security assurance and validation practices at Intel. The same strategy has been applied to Intel TDX security research activities.

Intel TDX Offensive Security Research Overview
Architecture Hardware Component Firmware Component Software Component Platform
  • Pre-SAFE and SAFE Reviews
  • Review and define threat modeling
  • Emerging Threat Analysis and Mitigation Research
  • Component-level architecture reviews and threat modeling
  • CPU, hardware IP blocks, and System-on-Chip
  • RTL design reviews
  • Pre-silicon and post-silicon security testing
  • Component-level architecture reviews and threat modeling
  • Code review and fuzzing
  • *code (ucode, XuCode, oCode, pCode) 
  • MCHECK, SEAMLDR, ACTM
  • Component-level architecture reviews and threat modeling
  • Code review and fuzzing
  • Intel TDX Module
  • Linux Software Stack (FMM enhancements and TD OS)
  • TDX Attestation
  • End-to-End Intel TDX feature flows and attacks
  • Platform-level penetration testing using production configuration

Offensive Security Research Topics

The following sections describe the key research topics we covered, along with a few interesting examples.

Emerging Threat Analysis and Driving Mitigations

The objective of this research area is to understand emerging threats by referring to both internal and external research collateral and evaluate the applicability of those findings to Intel products and technologies. When the OSR team finds appropriate material, the OSR team develops Proof of Concept to demonstrate the risk those exploits might pose to the corresponding projects. The activity includes not only demonstrating such attacks but also working with product teams to mitigate such issues. Intel TDX security research resulted in a few interesting, as well as some business-critical, potential security vulnerabilities in the following areas. These have all been mitigated in Intel TDX 1.0.

  • TDx-Step attack and mitigation: Leveraging Intel® Software Guard Extensions (Intel® SGX)-Step type attack to compromise Intel TDX, which is coined as a TDx-Step attack.
  • A class of RowHammer attacks and mitigations: ECC disable and DDR refresh control-based attacks.
  • A class of Reliability, Availability and Serviceability (RAS) features for confidential computing: In this category, attacks were demonstrated in the context of memory disturbance errors and their impact on confidential computing. These attacks are mitigated as well.

Security Architecture Review and Threat Modeling

The objective of this research area is to engage early with product architecture teams, ranging from hardware architecture to server platform architecture teams and evaluate the security architecture definition by reviewing security architecture claims, identify gaps, if any, and drive them to closure way before the design phase. This approach has helped to address many architectural gaps in Intel TDX 1.0 and to resolve multiple issues that were categorized as Gating Product Release (GPR). The team also plays a key role in reviewing and enhancing the threat model definitions so that larger security validation teams could refer to them. In addition, the OSR team played a key role in defining and delivering the Platform Security Threat Model (PSTM), which consists of a set of platform level threats targeting Intel TDX, in collaboration with the platform architecture team.

Vulnerability Research

Intel TDX as a technology depends on multiple Intel hardware, firmware, software and platform extensions. The key focus of this research area is to evaluate each of these components for potential security vulnerabilities through design/code review and penetration testing activities. These research activities helped to uncover and mitigate more than 75 security vulnerabilities.

  • Hardware vulnerability research: This category includes the issues related to hardware components, which include CPU/SoC RTL, microcode (uCode), and related low-level firmware issues.
  • Firmware vulnerability research: The MCHECK module, which gets loaded as part of uCode patches, acts as the firmware Root of Trust for Intel TDX. Firmware security analysis mainly focused on MCHECK-centric issues.
  • Software vulnerability research: Intel TDX technology mainly relies upon two Intel-signed software components: SEAMLDR and Intel TDX Module. The software vulnerability research activity mainly involved these two components.
  • Platform-level vulnerability research: In this category the focus was mainly on Intel-supplied software components for Intel TDX, such as the Linux software stack, TD VF and TD Guest OSes, and also other Intel TDX Trusted Computing Base (TCB) platform ingredients like Authenticated Code Modules (ACMs), Intel SGX components (enclaves for remote attestation) etc.

Platform-level Penetration Testing

Intel TDX is a data-centric platform security technology that co-exists with other security technologies such as MKTME, SGX-TEM, etc. The objective of this research is to identify high-risk areas of focus at the platform level and asses the behavior of an Intel TDX TD in presence of hostile platform components from a security standpoint. The platform-level penetration testing effort involved reproducing the cloud setup with Production/Locked Server CPUs and used Production Keys (TD keys) for testing. This activity helped the teams uncover and mitigate issues at the platform level and confirm the security aspects of Intel TDX in a production environment.

Hackathons

Hackathon (HaT) is another assurance methodology at Intel where we bring product and security experts together with the sole purpose of finding security vulnerabilities within the product through all means available to complement the structured security evaluation process implemented in the SDL. Intel TDX is a technology comprised of hardware, firmware, software, and platform ingredients. Intel conducted five hackathons, focusing in the areas of MCHECK, Intel TDX Module, SEAM Loader, the Linux software stack, and Intel TDX end-to-end flows at the platform level. The effort resulted in uncovering 76 vulnerabilities and 12 architectural recommendations, which have been mitigated in 4th Generation Intel Xeon processors code named Sapphire Rapids. The following subsections provide brief summaries of each hackathon:

Intel TDX Module Hackathon Key Objectives
  • To conduct deep dive security architecture, design, and code review to identify high-risk areas of Intel TDX Module and drive mitigations.
  • To establish advanced security research focus on the following topics:
    • Applying formal verification techniques to strengthen Intel TDX security.
    • Sanitizers for better security.
    • Fuzzing techniques to strengthen Intel TDX Module’s security.
Intel TDX Module Hackathon Areas of Focus
  • Intel TDX Module Lifecycle - Intel TDX Operation Modes and Transition
  • Intel TDX Interface Functions – Security review and analysis
  • Measurement and Attestation
  • Key Management
  • TD Non-Memory State (Metadata) and Control Structures
  • Physical Memory Management
  • TD Private Memory Management
  • TD VCPU and CPU Virtualization (Non-Root Mode Operation) 
  • Concurrency issues, Secret Clearing, TLB Flushing, Context store/restore 
  • Error handling (Machine Check Error etc.) and debug/profiling
  • Security analysis by using address and memory sanitizing tools
  • Debug flows and performance monitoring 
  • Intel TDX and Intel® Control Flow Enforcement Technology (Intel® CET) interaction
Intel TDX Module Hackathon Key Recommendations
  • System level security review and penetration testing of Intel TDX Module: Intel TDX Module exposes interface functions to both Host (Hypervisor) and Guest (TD). From a threat modelling standpoint, both the TD and Hypervisor are not in the TCB of the Intel TDX Module, and hence could potentially turn malicious. The recommendation is to evaluate the Intel TDX Module’s interfaces and interaction with untrusted agents at the platform level to strengthen the overall Intel TDX security.

  • Imposing access control restrictions on ACMs: From the Intel TDX standpoint, ACMs are in the TCB for Intel TDX in Sapphire Rapids. Hence the recommendation is to restrict the ACM privilege to access only specific/allowed Intel TDX assets.

Intel TDX SEAMLDR Hackathon Key Objectives

The primary objective of this Hackathon was to conduct a comprehensive security review of NP-SEAMLDR and P-SEAMLDR architecture and source code.

Intel TDX SEAMLDR Hackathon Areas of Focus
  • NP-SEAMLDR pre-load configuration and status  
  • NP-SEAMLDR loading process  
  • P-SEAMLDR pre-load status and configuration  
  • P-SEAMLDR load and installation  
  • P-SEMALDR APIs that are used for TDX Module installation and update
  • P-SEAMLDR Shutdown API and status
  • NP-SEMALDR and P-SEMALDR error handling and access control  
  • Memory management and access control procedure 
  • Security Version Number (SVN) and anti-rollback mechanisms  
Intel TDX Linux Software Stack Hackathon Key Objectives
  • To conduct deep dive security architecture, design, and code reviews of the identified high-risk areas and drive timely mitigations.
  • To identify potential areas for penetration testing features during the Intel TDX Platform-level Hackathon.
Intel TDX Linux Software Stack Hackathon Areas of Focus
  • TDVF Intel TDX changes and Libraries
  • TD Guest Kernel - Drivers (virtIO ring and drivers)
  • TD Guest Kernel - PCI Config and resource allocation, ACPI tables, Lazy memory acceptance logic
  • Intel TDX Guest Kernel - Quote driver, TDX Attestation
  • KVM and QEMU Intel TDX changes
  • Interrupts/exceptions
  • Fuzzing: Intel TDX guest kernel and guest virtIO DMA shared pages
  • Error handling flows
Sapphire Rapids MCHECK Hackathon Key Objectives
  • To conduct system-level security review of high-risk areas that include:
    • Interaction with ACTM
    • Intel TDX support (for example, CMRs checks)
    • New BIOS/MCHECK parameters (such as TOPOLOGY_INFO)
    • Sapphire Rapids hardware changes in the context of Intel TDX/MCHECK flow
  • To identify key areas in MCHECK to target platform-level penetration -testing.
Sapphire Rapids MCHECK Hackathon Areas of Focus
  • Changes in MCHECK from 3rd Generation Intel Xeon Processors code name Ice Lake to 4th Generation Intel Xeon Processors code name Sapphire Rapids
  • Interaction with new ACTM
  • TME-MK integrity 
  • MCHECK Sapphire Rapids new BIOS parameters like TOPOLOGY_INFO
  • CMR Checks
  • Scrubbing of locals in crypto functions
  • Access Control Checks
Intel TDX End-to-End Platform-level Hackathon Key Objectives
  • Focus on platform level security reviews that were not covered as part of prior Intel TDX Hackathons or security review efforts.
  • Identify scenarios that can be exercised only on fully integrated Intel TDX setup as close to production as possible.
  • Exercise end-to-end flows and conduct platform-level penetration testing.
  • Enable Intel TDX attestation flows to perform end-to-end security validation.
Intel TDX End-to-End Platform-level Hackathon Areas of Focus
  • DMA Protection and hardware access control
    • Focus on access control checks with Intel TDX-enabled platform
    • DMA Access to MCheck Communication Mailbox
    • Hardware Range Registers
    • RowHammer-centric Memory Controller Registers
  • Fundamental attacks from VMM and BIOS
  • Debug/Production features disabled or reported
  • Attestation (Changes in attestation enclave)
    • Quoting Generation
    • Quoting data provided
  • Malicious TD vs VM and VMM (Platform denial of service is in scope)
  • Reset Flow Impact (Warm Vs Cold)
  • Machine Check scenarios (Ways to crash and response evaluation) 
  • TD vs TD/VMM malicious communication
Key Learnings and Recommendations from Intel TDX End-to-End Platform-level Hackathon 
  • Strong security assurance practices by Intel TDX teams: The results from the hackathon demonstrated the quality of the adoption of strong SDL and offensive security research practices by Intel teams and their collaboration with external researchers to uncover and mitigate security vulnerabilities well within the product development lifecycle.
  • Opportunities to scale and automate the security validation: In general, the hackathon events involve manual review and penetration testing of high-risk areas. However, to make the security validation scalable and automated, the recommendation for Intel TDX teams to develop techniques, tools and methodologies that could help to automate and reuse the security validation process for upcoming CPU generations supporting Intel TDX.
Overall Intel TDX Hackathon Summary of Findings
Intel TDX Hackathon Summary of Findings
Hackathon Name Vulnerability Details
  Critical High Medium Low  Recommendations
Intel TDX Module   6 4 11 3
Intel TDX SEAMLDR   1 6 4  
Intel TDX Linux Software Stack 2 5 11 3 4
Sapphire Rapids MCHECK   1 4 15 4
Intel TDX End-to-End Platform   3 0 0 2
Total Vulnerabilities 2 16 25 33 13

 

Figure 5: Summary of Hackathon Findings by Topic

Example Vulnerabilities Discovered from Research

The following sections describe three classes of interesting vulnerabilities uncovered as part of the Intel TDX OSR effort. The three categories include architectural, design and implementation issues. All these issues have been mitigated in Intel TDX Module 1.0 on processors code named Sapphire Rapids.

Architectural Vulnerability

TDX-Step Attack and mitigation: The SGX-Step attack was demonstrated on Intel SGX wherein an attacker with ring-0 software privilege shall be able to cause AEX (Asynchronous Exit) from an Intel SGX secure application to the untrusted world on every instruction boundary by abusing the underlying hardware APIC timer. The security research team worked with the Intel TDX architecture team to do feasibility analysis of SGX-Step attacks in the context of Intel TDX. The security research team was given a stringent timeline of two months to develop a proof of concept exploit to demonstrate this attack in the context of Intel TDX. Though Intel TDX technology was still under the definition and development stage, the security research team not only demonstrated a working exploit well within the timeline but also collaborated closely with the Intel TDX architecture team to review and refine the mitigation for the vulnerability. Intel TDX 1.0 contains a mitigation for the demonstrated TDX-Step attacks in the Intel TDX Module.

Design Vulnerability

Tracking of secure version number of SEAMLDR modules: The SEAMLDR module. which is used to load the Intel TDX Module, consists of two images: P-SEAMLDR and NP-SEAMLDR ACM. There is a single SVN associated with the full image. Since the NP-SEAMLDR ACM and the P-SEAMLDR images are built separately, there is a possibility of losing track of the correct version of the images. As part of the design review of the P-SEAMLDR image, we observed that the version number of P-SEAMLDR module was not mapped to the SVN of NP-SEAMLDR. Based on the finding and discussion with the Intel TDX teams, the design was enhanced to keep the version number of P-SEAMLDR mapped to the SVN of NP-SEAMLDR as part of the build process. That way the SEAMLDR build framework aligns the tracking of version numbers of both the images.

Implementation Vulnerability

Memory range checks for Intel TDX memory regions: TD private memory regions in Intel TDX are designated as Trust Domain Memory Regions (TDMRs) in DRAM. The host VMM allocates Convertible Memory Regions (CMRs) in DRAM. From a security standpoint, the TDMR should be contained within a valid CMR. Otherwise, the TDMR could potentially be exposed to untrusted agents (including the VMM). Hence, it’s the responsibility of the Intel TDX Module to help ensure that TDMRs are secure. From the security review of the Intel TDX Module code, we uncovered that the memory range checks are vulnerable to integer overflow attacks by malicious VMMs, and hence could eventually get access to the TDMR ranges. The security code review effort caught these types of issues and mitigated them in Intel TDX 1.0.

Defense-in-depth

Because of the continuous research effort, Intel TDX has adopted the latest hardening and defense-in-depth measures for side channel attacks. For example, to help mitigate Branch History Injection and Intra-mode Branch Target Injection, the Intel TDX module currently clears branch history after TD exits and after the hypervisor uses SEAMCALL, mitigating potential Branch History Injection attacks against TDs, hypervisors, and the Intel TDX module itself.

External Collaboration

Intel TDX Security Research Engagement with Google*

This is the Intel’s first-of-its kind security research collaboration with a CSP customer. The security research engagement involved multiple organizations and engineering teams from Intel. On the Google side, the key teams involved were the Google Cloud Security and Google Project Zero teams. The primary objective for the Google team in this security research collaboration was to provide assurances that the Intel TDX feature is secure, has no obvious defects, and works as expected so that it can be confidently used by both cloud customers and providers. Any defects or weaknesses discovered during the review were fed back to Intel for remediation.  A secondary objective was to have a better understanding of the expected threat model for Intel TDX and to identify limitations in the design and implementation that would better inform Google's deployment.  The published paper on this research collaboration can be found in Google's full report on Intel TDX.

Figure 6: Intel TDX 1.0 Security Review by Google: Areas of Focus

 

Figure 7: Intel TDX 1.0 Security Review by Google: Findings by Topic

Intel Security Research Engagement Through Trusted Crossings

As part of continuing to shift left and expand security coverage, the Data Platforms Engineering and Architecture (DPEA) team, Security Assurance and Research (DSAR), partnered with the Intel TDX team and Intel Product Assurance and Security (IPAS) Circuit Breakers program to conduct security research on Intel TDX with external security researchers called Trusted Crossings. Intigriti* was a key partner in enabling Intel to connect with external researchers in the security community and provided logistical assistance for the event. A total of 43 external researchers applied for the Trusted Crossings event, and 12 were selected based on their qualifications, skills, and availability of Eagle Stream platforms. The Trusted Crossings event was kicked off in July 2022 and concluded in December 2022.

There were three key objectives for Trusted Crossings:

  1. Proactively identify and fix vulnerabilities in Intel TDX 1.0 on Sapphire Rapids and Eagle Stream platforms.
  2. Learn attack techniques used by external researchers to enhance internal validation/research and identify the scenarios covered by researchers to increase confidence in Intel TDX technology.
  3. Continue to establish the Project Circuit Breakers program in the researcher community to increase engagement on Intel platforms and technologies.

Key Learnings and Results

Although no significant security vulnerabilities were discovered, two functional issues were found and shared with the Intel TDX team. All issues reported by the researchers are captured in an engineering defect tracking database. In addition to security assessment of Intel TDX pre-launch, the Intel team learned techniques used by the external researchers. Attack scenarios attempted by the researchers were submitted in the form of weekly reports. In some cases, functional proof of concept code was submitted with the weekly report. A total of 15 weekly reports were submitted during the event.

There were no major issues associated with DevCloud infrastructure and support. Some minor improvements have been captured as part of the learnings.

Intel TDX Security Research Engagement with NCC Group*

During November and December of 2021, Intel engaged NCC Group to conduct a security assessment of the Intel Trust Domain Extensions v1.0 solution, specifically the Intel TDX Module software.  The Intel TDX Module consists of approximately 20,000 lines of C code. As per the request for proposals (RFP), The following focus areas for the evaluation includes: 

  • Assess the Intel TDX Module implementation, ensuring all security objectives are met.
  • Review mitigations for known side channel attacks, as well as for other known vulnerabilities.
  • Perform a general secure coding assessment to ensure security posture and hygiene.
  • Review cryptographic constructions for correct algorithm usage and secret handling.

Testing was performed via manual code review and analysis of the public Intel TDX documentation. The summary of the review provided by NCC Group is available for download.

Glossary

  • ACM: Authenticated Code Module 
  • CMR: Convertible Memory Ranges
  • CPU: Central Processing Unit
  • DPEA: Datacenter Platform Engineering and Architecture
  • DDR: Double Data Rate
  • ECC: Error Correction Code
  • EGS: Eagle Stream (Server Platform) based on Sapphire Rapids
  • FAS: Firmware Architecture Specification
  • FW: Firmware
  • HaT: Hackathon
  • HSD: High Speed Database
  • HW: Hardware
  • IPAS: Intel Product Assurance and Security
  • ISA: Instruction Set Architecture
  • MCHECK: Non-Persistent PPPE Module; Intel-signed FW
  • Intel® TME-MK: Intel® Total Memory Encryption – Multi-Key; this engine is used to provide confidentiality of data for both VM-Isolation and SGX-TEM.
  • ocode: Out-of-band unit code
  • OSR: Offensive Security Research
  • pcode: Power management unit code
  • RFP: Request For Proposal
  • RTL: Register Transfer Level
  • SAFE: Security Architecture Forum
  • SDL: Security Development Lifecycle
  • SEAM: Secure Arbitration Mode
  • SEAMRR: Secure Arbitration Mode Register Range
  • Intel SGX: Intel® Software Guard Extensions
  • Intel SGX-TEM: Intel® Software Guard Extension – Trusted Environment Mode
  • SPR: Sapphire Rapids (10 nm server CPU)
  • SW: Software
  • TCB: Trusted Computing Base
  • TD: Trust Domain
  • Intel TDX: Intel Trust Domain Extensions
  • TOCTOU: Time Of Check Time Of Use
  • Intel TXT: Intel® Trusted Execution Technology 
  • uCode: Microcode
  • VM: Virtual Machine
  • VMM: Virtual Machine Monitor
  • VMX: Virtual Machine Extensions
  • XuCode: Extended Microcode