Data Operand Independent Timing Instruction Set Architecture (ISA) Guidance

Published: 05/10/2022  

Last Updated: 12/09/2022

This document describes a new feature supported by some recent Intel processors, adding a data operand independent timing mode which can be used to ensure “constant time” execution for a specific subset of instructions. Although not relevant for the majority of code, data operand independent timing (DOIT) is a useful property for code which has specifically been written to execute in constant time such as cryptographic algorithms. This mode allows constant time code to inform the processor that data operand independent timing is needed.

DOIT requires disabling hardware optimizations and/or performance features on some processors; for example, enabling data operand independent timing might disable data-dependent prefetching. This means that the DOIT mode may have a performance impact, and Intel expects the performance impact of this mode may be significantly higher on future processors. 

This functionality is intended for use by software which has already applied other techniques to mitigate software timing side channels, such as those documented in Intel's Guidelines for Mitigating Timing Side Channels Against Cryptographic Implementations. Such software is typically limited to cryptographic implementations. This mode is not intended to provide any additional security benefit for software that is not already applying these software techniques. Due to its potential performance impact, Intel does not recommend enabling this mode globally. Instead, the mode should be enabled only for software that is specifically designed to benefit from the additional properties provided by the DOIT mode. In order to allow this mode to be enabled for the subset of an application that is constant time, Intel plans to add new capabilities in future processors to allow applications to dynamically enabled/disabled when system software permits.

The data operand independent timing instructions section provides a list of the instructions that have data-independent timing that can be used in conjunction with the previous guidelines. Software using these instructions for “constant time” code can enable the DOIT mode, which ensures data operand independent timing, on a per-thread or per-process basis. Future enhancements may allow more fine-grain user mode control inside an application or library. Lastly, on certain processors, MXCSR may also need to be configured to avoid data-dependent behavior for the instructions listed in the Instructions that May Exhibit MCDT Behavior section. 

Instructions with Data Operand Independent Timing (DOIT)

Refer to the list of documented data operand independent timing instructions for a list of instructions with data operand independent timing. Note that these data operand independent timing instructions may have variable latency for reasons unrelated to the data values (for example, loading data from memory or the encoding of the instruction). Furthermore, an instruction being included on this list does not mean that its usage is resistant to power, thermal, or frequency-based side channels. Refer to the frequency-throttling side channel guidance for more information. 

Data Values vs. Address Values

The latency of a data operand independent timing instruction does not depend on the data values on which it operates. However, the latency of these instructions that fetch data from memory may vary based on the memory address that the load accesses to get that data, as discussed in Intel's Guidelines for Mitigating Timing Side Channels Against Cryptographic Implementations

Instruction Encodings and Immediate Values

Anything that changes the code bytes of an instruction may impact the latency of that instruction. For example, an instruction may have different latency if: 

  • It has a different immediate value.
  • It uses a different memory addressing mode (for example, just a base register instead of having both a base and index register) even when the address being accessed is the same. 
  • It uses different registers (for example, RAX vs. AX vs. RBX) or different condition code (such as CF vs. ZF). 

Masked Operations

Masked operations have mask registers as inputs to the instruction. For example, certain Intel® Advanced Vector Extensions 512 (Intel® AVX-512) operations take a K mask register input. Some non-Intel AVX-512 operations (such as VMASKMOV, MASKMOVDQU, Gather, among others) have a mask register that is not a K mask register. For masked operations that do not access memory, the instruction latency will be invariant with respect to the mask register value.

For masked operations that access memory, the mask register (whether a K mask register or a separate register) may affect which memory addresses are accessed. Therefore, data operand independent timing instructions that read or write to memory may have different timing depending on mask register values.  

Data Operand Independent Timing Mode (DOITM)

To support use cases such as constant time code, a new model specific register (MSR) control enables data operand independent timing for the listed data operand independent timing instructions. 

Specifically, when data operand independent timing mode is enabled, instructions within the data operand independent timing subset execute with timing (in terms of processor cycles) that is independent of the data values in the sources of the instruction, except for potential timing differences due to power or thermal variations. Moreover, while in data operand independent timing mode, the data values operated on by these data independent instructions will not impact the timing of other instructions. This property is true regardless of the status of data operand independent timing mode when the other instructions are executed. 

When an instruction outside of the data operand independent timing subset or outside of the data operand independent timing mode is executed (either speculatively or non-speculatively), the data values operated on could potentially affect the timing both of the instruction operating on the data and of other instructions. Additionally, power consumed, CPU frequency, and telemetry recorded (such as performance monitoring events or running average power limit (RAPL) power measurements1) may differ based on the data values operated on within data operand independent timing mode. It is also possible that subsystems outside the core may implement data operand dependent features that may impact the timing in a data-dependent manner for data independent instructions. 

Developers handling secret data that should only ever be processed in a data operand independent timing manner may need to consider speculative execution vulnerabilities. These vulnerabilities may cause the secret data to be handled in a data operand dependent manner and developers may need to apply additional mitigations. Refer to Mitigating Timing Side Channels Against Cryptographic Implementations

Data Operand Independent Timing Mode Controls

IA32_UARCH_MISC_CTL[DOITM] is a data operand independent timing mode control mechanism that restricts non-data operand independent timing behavior for the listed instructions. A processor supports IA32_UARCH_MISC_CTL[DOITM] if IA32_ARCH_CAPABILITIES[12] is 1. WRMSR to IA32_UARCH_MISC_CTL (MSR index 1B01H) is not defined as a serializing instruction.  

Table 1: Enumeration of Data Operand Independent Timing Mode on IA32_ARCH_CAPABILITIES
Register Address Hex Register Address Dec Register Name /
Bit Fields
Bit Description Comment
10AH 266 IA32_ARCH_CAPABILITIES Enumeration of Architectural Features (RO) If CPUID.(EAX-07H, ECX=0):EDX[29]=1
10AH 266 12 DOITM: The processor supports data operand independent timing mode.  

 

For Intel® Core™ family processors based on microarchitectures before Ice Lake and Intel Atom® family processors based on microarchitectures before Gracemont that do not enumerate IA32_UARCH_MISC_CTL, developers may assume that the instructions listed here operate as if DOITM is enabled.

Intel Core family processors based on Ice Lake and later, such as Tiger Lake, Lakefield, and Rocket Lake will enumerate DOITM. Intel Atom family processors based on Gracemont and later will also enumerate DOITM. Refer to the Enumeration and Architectural MSRs section for more information.

Software Enabling

On certain processors, microcode updates may need to be loaded for IA32_UARCH_MISC_CTL to be enumerated.

Software can enable data operand independent timing operation on a logical processor by setting IA32_UARCH_MISC_CTL[DOITM] to 1. Setting DOITM to 1 may impact performance, and that impact may increase in future processor generations.

Users should evaluate their threat model to decide whether this is a significant threat to their applications and then ask the operating system to only deploy DOIT mode to applications that they deem necessary. Note that DOIT mode is not expected to significantly improve resistance to side channel attacks unless the software was carefully written to avoid such attacks (specifically, following the guidance in Intel’s Guidelines for Mitigating Timing Side Channels Against Cryptographic Implementations). 

Operating systems may wish to support IA32_UARCH_MISC_CTL in a manner similar to speculative store bypass disable.

Interactions with Other Modes of Operation

Interactions with Intel® Software Guard Extensions (Intel® SGX)

While executing inside enclave mode, processors that enumerate support for DOITM will enable data operand independent timing mode irrespective of the settings that control the mode.

The existing Intel SGX architecture cannot manipulate DOITM and cannot trust the OS to manipulate DOITM.

To mitigate attacks on constant time code running in enclave mode, processors that enumerate support for DOITM will always enable data operand independent timing mode. This behavior is independent from the setting of DOITM itself.

This design choice considers both the complexity of extending the Intel SGX architecture to allow more dynamic control of mitigations and the existing reduced performance within enclave mode.

Note: Software with similar complexity concerns and which can tolerate reduced performance may also converge on a design that opts to leave DOITM always enabled. 

Interactions with Intel® Trust Domain Extension (Intel® TDX) 

There are no special interactions or interactions between data operand independent timing mode and Secure Arbitration Mode-Virtual Machines Extensions (SEAM VMX) operation. In particular, the enabling of data operand independent timing mode is not impacted while operating in SEAM VMX mode.

The DOIT mode control will be made visible to trust domains (TDs) and the Intel TDX module will context switch the control between TDs. Unlike Intel SGX enclaves, TDs can decide to set this based on their threat model. 

MXCSR Configuration Dependent Timing (MCDT)

On processors listed in the MCDT Enumeration section, even when DOIT mode is enumerated and enabled, some data-independent timing vector instructions may have subtle data-dependent timing due to MXCSR configuration. Specifically, specific data values may delay instruction retirement by, at most, one cycle. This is a small enough delay that it may not be observable in common practice, but this small delay is still data-dependent timing. This data operand-dependent timing may impact software following Intel’s Guidelines for Mitigating Timing Side Channels Against Cryptographic Implementations.

Hardware

Future processors are expected to not exhibit data operand dependent timing due to MXCSR configuration. This will be enumerated by CPUID.(EAX=7H,ECX=2):EDX[5]=1. Many current processors which are also not affected are listed in the Processors That Do Not Exhibit MCDT Behavior section.

Software Mitigations

Some environments may conclude that this side channel is minor and does not need mitigation. 

For systems that do not enumerate CPUID.(EAX=7H,ECX=2):EDX[5]=1, software that does need mitigation can mitigate the data-dependent timing by loading MXCSR with the value of 0x1fbf before using data operand independent timing instructions impacted by the MXCSR configuration. Software must use pre- and post-serialized LDMXCSR instructions before using impacted data operand independent timing instructions.

Prolog:
STMXCSR save_val
LDMXCSR value_0x1fbf
LFENCE
// Constant-time code using affected instructions
Epilog:
LFENCE
LDMXCSR save_val

For applications that do not otherwise depend on a specific value of MXCSR, the value of 0x1fbf can be established during the initialization of the application and mitigate the data dependent timing with minimal overhead.

Instructions That May Exhibit MCDT Behavior

This list is based on Intel's reasonable investigation and is current as of the date of publication. Intel will update this list if additional instructions with these characteristics are discovered.

  • PMADDUBSW
  • PMULUDQ
  • VPMULHRSW
  • PMADDWD
  • VPLZCNTD    
  • VPMULHUW
  • PMULDQ
  • VPLZCNTQ
  • VPMULHW
  • PMULHRSW
  • VPMADD52HUQ
  • VPMULLD
  • PMULHUW
  • VPMADD52LUQ
  • VPMULLQ
  • PMULHW
  • VPMADDUBSW
  • VPMULLW
  • PMULLD
  • VPMADDWD
  • VPMULUDQ
  • PMULLW
  • VPMULDQ

Intel SGX

Although instructions inside enclave mode will act as if data operand independent timing mode is set, they may still exhibit MCDT behavior.

If the intended operation of an Intel SGX enclave, for the lifetime of the enclave, is achievable with MXCSR=0x1FBF, then any loads of MXCSR (via LDMXCSR, XRSTOR, etc.) should be of 0x1FBF. There should also be an LFENCE between any load of MXCSR (even of 0x1FBF) and subsequent use of any affected instruction. 

If the enclave is compatible with MXCSR=0x1FBF and uses any of the affected instructions, then the beginning of each enclave ECALL should change MXCSR to 0x1FBF and then execute an LFENCE instruction. The Intel SGX SDK will be changed to do this.

The Intel-provided SGX architectural enclaves (AEs) fall into this “compatible with MXCSR = 0x1FBF” category.

If the intended operation of an Intel SGX enclave is not achievable with MXCSR=0x1FBF, then the general sequence in the Software Mitigations section should be used.

Enumeration and Architectural MSRs

DOIT Enumeration

IA32_ARCH_CAPABILITIES[12] enumerates support for the IA32_UARCH_MISC_CTL MSR (addr 0x1B01) and bit 0 of that MSR (DOITM). This bit implements a data operand independent timing mode. This enumeration bit is called Data Operand Independent Timing Mode (DOITM). 

Processors that do not enumerate IA32_ARCH_CAPABILITIES[DOITM] when the latest microcode is applied do not need to set IA32_UARCH_MISC_CTL [DOITM] in order to have the behavior described in this document, although they may be affected by MXCSR Configuration Dependent Timing (MCDT) as described in the MXCSR Configuration Dependent Timing (MCDT) section. 

MCDT Enumeration

CPUID.(EAX=7H,ECX=2):EDX[5] enumerates MCDT_NO. Processors that enumerate this bit as 1 do not exhibit MCDT behavior and do not need to be mitigated. Note that Intel Atom and pre-Skylake Intel Core processors may not enumerate MCDT_NO but nevertheless do not exhibit MCDT behavior. Refer to the later sections, Processors That May Exhibit MCDT Behavior and Processors That Do Not Exhibit MCDT Behavior.

MCDT Enumeration Guidance for Cryptographic Software

CPUID leaf 7 subleaf 2 EDX bit 5 enumerates MCDT_NO. Processors that enumerate this bit as 1 do not exhibit MXCSR Configuration Dependent Timing behavior and do not need to mitigate it. Intel Atom and Intel Core processors based on microarchitectures prior to Skylake may not exhibit MCDT behavior despite not enumerating MCDT_NO.  See the Processors That Do Not Exhibit MCDT Behavior section for more information.

Cryptography developers who wish to apply MCDT mitigations should use the following steps to determine if MCDT mitigation should be applied:

  1. The cryptography software should determine if MCDT_NO is enumerated as 1. If it is, then no mitigation needs to be applied.
  2. If MCDT_NO is enumerated as 0, the cryptography software should determine if it is running in a virtualized environment by checking the value of CPUID leaf 1 ECX bit 31.

If CPUID leaf 1 ECX bit 31 is 0, then a hypervisor is not present, and the cryptography software should use CPUID family/model/stepping to determine if it is running on a processor documented as not exhibiting MCDT behavior. 

If CPUID leaf 1 ECX bit 31 is 1, then a hypervisor is present and the cryptography software may be running on a processor that does not exhibit MCDT behavior, regardless of the family/model/stepping that is enumerated due to virtualization of CPUID.

VMM Guidance for Non-Migratable and Migratable VMs

For non-migratable VMs, VMM software should enumerate MCDT_NO2  as 1 to a guest only if the host physical machine either enumerates MCDT_NO as 1 or a CPUID family/model/stepping that matches a processor documented to not exhibit MCDT behavior in the Processors That Do Not Exhibit MCDT Behavior section.

For migratable VMs, VMM software should enumerate MCDT_NO as 1 only if every host physical machine that the VM may potentially migrate to either enumerates MCDT_NO as 1 or a CPUID family/model/stepping that matches a processor documented to not exhibit MCDT behavior in the Processors That Do Not Exhibit MCDT Behavior section.

IA32_UARCH_MISC_CTL MSR Definition 

Table 2: IA32_UARCH_MISC_CTL
Register Address Hex Register Name / Bit Fields Permission Bit Description Comment
1B01H IA32_UARCH_MISC_CTL     If IA32_ARCH_CAPABILITIES[DOITM]=1
1B01H 0 R/W Data Operand Independent Timing Mode (DOITM)  If IA32_ARCH_CAPABILITIES[DOITM]=1
1B01H 63:1 RO Reserved  

This MSR is logical processor scoped and has a reset value of 0.

Processors That May Exhibit MCDT Behavior

Intel Core processors based on microarchitecture code named Skylake and later exhibit this behavior for at least one instruction from the list in the Instructions That May Exhibit MCDT Behavior section.

Table 3: Processors That May Exhibit MCDT Behavior
Processor Stepping (All unless otherwise noted) Code Name (s) / Microarchitecture (s) Product Family
06_4EH 3 1. Skylake Y
2. Skylake U
3. Skylake U23e
6th Generation Intel® Core™ Processor Family
06_55H 3, 4 1. Skylake Server
2. Skylake D, Bakerville
3. Skylake W
4. Skylake X
 
1. Intel® Xeon® Scalable processor family
2. Intel® Xeon® D processor family
3. Intel® Xeon® W processor family
4. Intel® Core™ X-series Processors
06_55H 6.7 1. Cascade Lake Server
2. Cascade Lake W
3. Cascade Lake X
1. 2nd Generation Intel® Xeon® Scalable processor family
2. Intel® Xeon® W processor family
3. Intel® Core™ X-series Processor
06_55H <=B Cooper Lake 3rd Generation Intel® Xeon® Scalable processor family
06_5EH 3 1. Skylake Xeon E3
2. Skylake H
3. Skylake S 

1. Intel® Xeon® E processor family
2, 3. 6th Generation Intel® Core™ Processor Family

06_6AH 4,5,6 Ice Lake Xeon-SP 3rd Gen Intel® Xeon® Scalable processor family
06_6CH 1 Ice Lake D Intel® Xeon® D Processor
06_7EH 5 Ice Lake U,Y 10th Generation Intel® Core™ Processor Family
06_8AH 1 Lakefield B-step  Intel® Core™ Processors with Intel® Hybrid Technology
06_8CH 1,2 Tiger Lake U
Tiger Lake U Refresh  
Tiger Lake H35
11th Generation Intel® Core™ Processor Family
06_8DH 1 Tiger Lake H   11th Generation Intel® Core™ Processor Family
Intel® Xeon® Processor Family
06_8EH 9 1. Amber Lake-Y 
2. Kaby Lake U
3. Kaby Lake U23e
4. Kaby Lake Y
1. 8th Generation Intel® Core™ Processor Family
2,3,4. 7th Generation Intel® Core™ Processor Family
06_8EH A Coffee Lake U43e
Kaby Lake Refresh U
8th Generation Intel® Core™ Processor Family
06_8EH B,C 1. Whiskey Lake U
2,3,4. Comet Lake U42
5. Amber Lake Y
1. 8th Generation Intel® Core™ Processors
2. 10th Generation Intel® Core™ Processor Family
3. Intel® Pentium® Gold Processor Series
4. Intel® Celeron® Processor 5000 Series
5. 10th Generation Intel® Core™ Processor Family
06_97H 2, 5 Alder Lake S 12th Generation Intel® Core™ Processor Family
06_9AH 3 1. Alder Lake H
2. Alder Lake P
1. 12th Generation Intel® Core™ Processor Family
2. 12th Generation Intel® Core™ Processor Family
06_9AH 4 Alder Lake U 12th Generation Intel® Core™ Processor Family
Intel® Pentium® Gold Processor Family
Intel® Celeron® Processor Family
06_9EH 9 1. Kaby Lake S
2. Kaby Lake H
3. Kaby Lake G
4. Kaby Lake X
5. Kaby Lake Xeon E3
1. 7th Generation Intel® Core™ Processor Family
2. 7th Generation Intel® Core™ Processor Family
3. 8th Generation Intel® Core™ Processor Family
Intel® Pentium® Processor Family
4. Intel® Core™ X-series Processors
5. Intel® Xeon® E processor family
06_9EH A, B, C, D     1.    Coffee Lake H
2.    Coffee Lake Xeon E
3. Coffee Lake S Xeon E
4. Coffee Lake S x/KBP
5. Coffee Lake S
1. 8th Generation Intel® Core™ Processor Family
2. Intel® Xeon® E processor family
3. Intel® Xeon® E processor family
4. 8th Generation Intel® Core™ Processor Family
5. 8th Generation Intel® Core™ Processor Family
06_A5H 2, 3, 5 1. Comet Lake H
2. Comet Lake-S
1, 2. 10th Generation Intel® Core™ Processor Family
1, 2. Intel® Xeon® W processor family
06_A6H <=1 Comet Lake U62 10th Generation Intel® Core™ Processor Family
Intel® Xeon® W processor family
06_A7H 1 Rocket Lake 11th Generation Intel® Core™ Processor Family
Intel® Xeon® E-2300 processor family
06_A8H 1 Rocket Lake Intel® Xeon® W-1300 Processor Family

 

  • Note that a limited number of future processors which are not on the above list of affected processors, may also exhibit MCDT behavior. They will not enumerate MCDT_NO
  • Processors that may exhibit MCDT behavior include those based on microarchitectures code named Skylake server, Cascade Lake, Cooper Lake, Ice Lake server, Skylake, Kaby Lake, Coffee Lake, Whiskey Lake, Comet Lake, Ice Lake client, Lakefield, Tiger Lake, Rocket Lake, and Alder Lake.
  • Processors that do not exhibit MCDT behavior include Intel Atom processors and Intel Core processors based on microarchitectures before Skylake.

Processors That Do Not Exhibit MCDT Behavior

Intel Core processors based on microarchitectures before Ice Lake and Intel Atom family processors based on microarchitectures before Gracemont do not exhibit MXCSR Dependent Timing behavior.

Table 4: Processors That Do Not Exhibit MCDT Behavior
Processor Stepping (All unless otherwise noted) Code Name (s) / Microarchitecture(s) Product Family
06_3FH 2 Haswell Server EP, EP4S Intel® Xeon® E processor family
06_3FH 4 Elkhart Lake (Tremont) Intel® Xeon® E processor family
06_4CH   Cherryview (Airmont) Intel® Atom® Processor X Series
06_4FH   Broadwell Server E, EP, EP4S, EX Intel® Xeon® E processor family
06_56H 3 Broadwell DE V2,V3 Intel® Xeon®  D processor family
06_56H 4 Broadwell DE Y0 Intel® Xeon®  D processor family
06_56H 5 Broadwell DE A1, Hewitt Lake (Broadwell DE) Intel® Xeon®  D processor family
06_5AH   Anniedale (Airmont) Intel® Atom® Processors
06_5CH 9 1. Apollo Lake (Goldmont - no SGX)
2. Apollo Lake
3. Apollo Lake
1. Intel® Pentium® Processor J Series
Intel® Pentium® Processor N Series
2. Intel® Celeron® Processor J Series,
Intel® Celeron® Processor N Series
3. Intel® Atom® Processor A Series
06_5CH A Apollo Lake Intel® Atom® Processor E3900 Series
06_5FH 1 Denverton (Goldmont) Intel® Atom® C processor family
06_65H   XMM7272  (Airmont) Intel® Atom® Processors
06_6EH   Cougar Mountain (Airmont)

Intel® Puma™ 7 Family 
Intel® Atom® Processors

06_75H   Butter (Airmont)  Intel® Atom® Processors
06_7AH 1 Gemini Lake

Intel® Pentium® Processor Silver Series
Intel® Celeron® Processor J Series
Intel® Celeron® Processor N Series

06_7AH 8 Gemini Lake

Intel® Celeron® Processor J Series
Intel® Celeron® Processor N Series

06_86H 4 Snowridge (Tremont) Intel® Atom® Processors
06_86H 5 (B step) Snowridge (Tremont) Intel® Xeon® D processor family
06_86H 7 (C step) Snowridge (Tremont) Intel® Xeon® D processor family
06_8AH 1 Lakefield B-step (Tremont) Intel® Core™ Processors with Intel® Hybrid Technology
06_8AH 1 Lakefield B-step (Sunnycove) Intel® Core™ Processors with Intel® Hybrid Technology
06_96H 1 Elkhart Lake (Tremont) Intel® Atom® Processors
06_9CH 0 Jasper Lake (Tremont) Intel® Atom® Processors
 

 

Footnotes

  1. Refer to Running Average Power Limit Energy Reporting.
  2. MCDT_NO is CPUID.(EAX=7H,ECX=2):EDX[5].

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.