One of the strengths of Intel® Software Guard Extensions Data Center Attestation Primitives (Intel® SGX DCAP) is that it allows data centers to own their own attestation infrastructure. The attestation collateral needed to both generate and verify quotes is contained completely on premise, which eliminates runtime dependancies on external services and enables trust decisions to be made in-house. Attestations are signed and verified using ECDSA signed collateral that is obtained from Intel when Intel SGX capable systems are provisioned, and then saved in the data center’s caching service.
In this article, we’ll demonstrate a remote attestation using the sample code from the Intel SGX DCAP source code repository. While this demo will be a very simplified version of an attestation—we’ll use standalone programs to generate and verify the quote, and we’ll manually copy the quote from the runtime system to the relying party—it is nonetheless a complete procedure. We’ll also demonstrate that the relying party can perform quote verification both in an Intel SGX enclave and without Intel SGX support at all. This procedure can also serve a known-good test for new environments to help ensure that all the Intel SGX DCAP components are installed and properly configured.
Before You Begin
This demo requires a functioning Intel SGX DCAP environment, and specicially builds off the one created in the article Intel® Software Guard Extensions Data Center Attestation Primitives (Intel® SGX DCAP): A Quick Install Guide. If you have not run through this procedure, you should do so now.
If you need an introduction to Intel SGX DCAP, see the following white papers:
- Attestation for Data Center Orientation Guide
- Supporting Third Party Attestation for Intel® SGX with Intel® Data Center Attestation Primitives
And the following videos from Intel® Network Builders University (free registration required):
- Introduction to Intel® SGX Data Center Attestation Primitives
- Introduction to Intel® SGX Data Center Attestation Primitives: Components and Flows
- Intel® Software Guard Extensions Data Center Attestation Primitives: The Data Center Caching Service
ECDSA Attestation Overview
Attestation is the process of proving your identity to a relying party. In the context of Intel SGX, this means providing evidence that you are running on an Intel SGX enabled platform, inside of a properly instantiated enclave, on a system with a known security configuration. In a remote attestation, the Intel SGX workload attests to a remote entity over a secure communication channel.
The high-level flow of an Intel SGX ECDSA attestation is shown in Figure 1.
- The Intel SGX workload contacts the relying party and requests access to a service or resource.
- The relying party responds by issuing a challenge: it asks the Intel SGX workload to identify itself and provide proof that its credentials are valid.
- To satisfy the challenge, the Intel SGX workload generates a quote, which is a cryptographic measurement of the instantiated enclave. The quote is signed using the attesttion collateral that’s stored in the data center caching service.
- The quote is sent it to the relying party over the secure communications channel.
- The relying party verifies the quote. It fetches the attestation collateral associated with the quote from the data center caching service and uses it to verify the signature.
- Assuming the quote is valid, it examines the quote metadata and the trusted compute base (TCB) level that is associated with the signing key. The service then applies its security policy and decides whether it should trust the enclave.
This demo will focus on steps 3-6.
Intel does not define a specific protocol for communications between the Intel SGX workload and the relying party. How the two systems establish their secure communications channel and exchange information is up to the solution provider.
Attestation Demo
To perform an attestation, we first need to set up an Intel SGX DCAP environment with three systems:
- A system to host the data center caching service.
- A system with Intel SGX support that has been provisioned for running runtime workloads.
- A system with Intel SGX support to function as the relying party.
The Intel SGX DCAP Quick Install Guide steps through the process of setting up a caching service, provisioning Intel SGX enabled systems, and configuring them for runtime workloads. You should follow that guide and set up your caching service along with two Intel SGX enabled systems. Choose one of these systems to represent the Intel SGX workload, and the other to represent the relying party.
We’ll perform our attestation using sample applications from the Intel SGX DCAP source code repository. They are:
- QuoteGenerationSample
- QuoteVerificationSample
The QuoteGenerationSample app will need to be built on the system that will function as our Intel SGX workload, and the QuoteVerificationSample app will be built on the system serving as our relying party.
System Requirements
As with the Quick Install Guide, this demo will use Ubuntu* Linux* 20.04 LTS as the OS.
Configure the Intel SGX workload system
To build the sample code we’ll need the Intel® Software Guard Extensions SDK for Linux* OS (Intel® SGX SDK for Linux* OS) which we can download as a binary installation package using wget. Intel SGX SDK releases can be found on 01.org, and at the time of this writing the most recent version was 2.12.
Download the installer and then make it executable.
wget https://download.01.org/intel-sgx/latest/dcap-latest/linux/distro/ubuntu20.04-server/sgx_linux_x64_sdk_2.12.100.3.bin chmod 755 sgx_linux_x64_sdk_2.12.100.3.bin
Now run the installer and give it the prefix /opt/intel. This will install the SDK into /opt/intel/sgxsdk.
sudo ./sgx_linux_x64_sdk_2.12.100.3.bin --prefix=/opt/intel
Next, source the SDK’s environment setup file so that we can build the code samples.
source /opt/intel/sgxsdk/environment
Fetch the Intel SGX DCAP source code repository, which contains our sample code.
git clone https://github.com/intel/SGXDataCenterAttestationPrimitives
To build the sample code, we’ll need to install the header files for two components: the Intel SGX DCAP quoting library and the unified quoting service.
sudo apt -y install libsgx-dcap-ql-dev libsgx-quote-ex-dev
Now we’re ready to build. Change to the QuoteGenerationSample directory and run make. We want to build the sample in debug mode so that we don’t have to set up a launch policy or use a two-step enclave signing procedure, so we add the SGX_DEBUG=1 definition to the command line.
cd SGXDataCenterAttestationPrimitives/SampleCode/QuoteGenerationSample make SGX_DEBUG=1
Debug mode is sufficient for our demonstration purposes, but production enclaves should always be built in release mode.
Generate the Quote
Now we can run the sample. It will generate a quote and write it to a file named “quote.dat” in the current directory.
QuoteGenerationSample$ ./app
sgx_qe_set_enclave_load_policy is valid in in-proc mode only and it is optional: the default enclave load policy is persistent:
set the enclave load policy as persistent:succeed!
Step1: Call sgx_qe_get_target_info:succeed!
Step2: Call create_app_report:succeed!
Step3: Call sgx_qe_get_quote_size:succeed!
Step4: Call sgx_qe_get_quote:succeed!cert_key_type = 0x5
sgx_qe_cleanup_by_policy is valid in in-proc mode only.
Clean up the enclave load policy:succeed!
QuoteGenerationSample$ ls -l quote.dat
-rw-r--r-- 1 sgxtest users 4594 Jan 28 14:31 quote.dat
You need to copy this file to the system that is your relying party. We’ll use scp to do that, but any secure file transfer option will do.
$ scp quote.dat relyingparty:.
sgxtest@relyingparty's password:
quote.dat 100% 4594 100.8KB/s 00:00
Configure the relying party system
Like the Intel SGX workload system, we’ll need to pull in the SGX SDK and the Intel SGX DCAP source code repository. The procedure is the same as it is with the Intel SGX workload system:
wget https://download.01.org/intel-sgx/latest/dcap-latest/linux/distro/ubuntu20.04-server/sgx_linux_x64_sdk_2.12.100.3.bin
chmod 755 sgx_linux_x64_sdk_2.12.100.3.bin
sudo ./sgx_linux_x64_sdk_2.12.100.3.bin --prefix=/opt/intel
source /opt/intel/sgxsdk/environment
git clone https://github.com/intel/SGXDataCenterAttestationPrimitives
The package requirements are different for the relying party. We are verifying quotes, not generating them, so we need the Intel® Software Guard Extensions ECDSA Quote Verification Library (Intel® SGX ECDSA Quote Verification Library) component as well as its development headers.
sudo apt -y install libsgx-dcap-quote-verify libsgx-dcap-quote-verify-dev
With the prerequisites in place, we can now build the sample. Change to the QuoteVerificationSample directory and run make with the same SGX_DEBUG=1 definition.
cd SGXDataCenterAttestationPrimitives/SampleCode/QuoteVerificationSample
make SGX_DEBUG=1
Verify the quote
We’re now ready to verify our quote. Run the sample code, using the -quote option to provide the path to your quote.dat file. Ours is in our home directory, so we run:
QuoteVerificationSample$ ./app -quote ~/quote.dat
Info: ECDSA quote path: /home/sgxtest/quote.dat
Trusted quote verification:
Info: get target info successfully returned.
Info: sgx_qv_set_enclave_load_policy successfully returned.
Info: sgx_qv_get_quote_supplemental_data_size successfully returned.
Info: App: sgx_qv_verify_quote successfully returned.
Info: Ecall: Verify QvE report and identity successfully returned.
Info: App: Verification completed successfully.
===========================================
Untrusted quote verification:
Info: sgx_qv_get_quote_supplemental_data_size successfully returned.
Info: App: sgx_qv_verify_quote successfully returned.
Info: App: Verification completed successfully.
The sample application performs two verifications. The first, labeled “Trusted quote verification”, is done in an Intel SGX enclave using Intel’s reference quote verification library and quote verification enclave. This approach requires that Intel SGX be enabled on the platform. The second attestation, labeled “Untrusted quote verification”, also uses Intel’s reference quote verification library, but it’s done in untrusted memory and does not require Intel SGX.
It is entirely possible that your quote will verify with a warning. As long as the verification sample reports “Verification completed with a Non-terminal result”, then your quote was valid. The most common causes of verification warnings are TCB issues. See the section titled “Quotes with TCB Issues”, below.
To demonstrate that the verification procedure works even without Intel SGX on the relying party system, you can unload the Intel SGX driver from the kernel and re-run the sample. In this instance, the trusted quote verification will fail, but the untrusted verification will succeed (though it will warn you that it could not launch any enclaves):
QuoteVerificationSample$ sudo rmmod intel_sgx
QuoteVerificationSample$ lsmod | grep sgx
QuoteVerificationSample$ ./app -quote ~/quote.dat
Info: ECDSA quote path: /home/sgxtest/quote.dat
Trusted quote verification:
Error: Can't load SampleISVEnclave. 0x2006
===========================================
Untrusted quote verification:
[load_qve ../sgx_dcap_quoteverify.cpp:209] Error, call sgx_create_enclave for QvE fail [load_qve], SGXError:2006.
[sgx_qv_get_quote_supplemental_data_size ../sgx_dcap_quoteverify.cpp:527] Error, failed to load QvE.
Info: sgx_qv_get_quote_supplemental_data_size successfully returned.
[load_qve ../sgx_dcap_quoteverify.cpp:209] Error, call sgx_create_enclave for QvE fail [load_qve], SGXError:2006.
[sgx_qv_get_quote_supplemental_data_size ../sgx_dcap_quoteverify.cpp:527] Error, failed to load QvE.
Info: App: sgx_qv_verify_quote successfully returned.
Info: App: Verification completed successfully.
The last line is the one we want: it indicates that the quote verification completed successfully.
Inspect the quote
With the quote verified, we can now inspect the quote and examine its contents. Remember that quote verifcation merely tells you whether the quote is valid, and whether there are issues with the remote enclave’s TCB. Trusting the quote is not the same as trusting the enclave: to make a trust decision about the enclave, you must examine its metadata and, at minimum, answer the following questions:
- Is this enclave one that you recognize? Specifically, do you recognize MRSIGNER, MRENCLAVE, and the vendor-specific identifiers?
- Is the debug attribute set?
- Is the quoting enclave one that you recognize? And is it up to date?
- Is the enclave’s software version up to date?
A production service may have even more stringent requirements. Ultimately, these are all policy matters; it is up to the service provider to decide which enclaves their service will trust and how that service should arrive at its decisions.
We won’t do an exhaustive examination of our demo quote, but we’ll look at some key fields that can answer a subset of the questions above. The quote structure is defined in the header file sgx_quote_3.h, and we can use the xxd tool in Linux to extract a few of its fields.
Here’s the MRENCLAVE value:
~$ xxd -s 112 -g 0 -l 16 quote.dat
00000070: da77d1e3e10c61e405442f117ae10f0e .w....a..D/.z...
And the MRSIGNER:
~$ xxd -s 176 -g 0 -l 16 quote.dat
000000b0: d412a4f07ef83892a5915fb2ab584be3 ....~.8..._..XK.
These two values tell you who signed the enclave, and which enclave it is. As a rule, a relying party should never trust an enclave or a signer that it doesn’t recognize!
This next field stores the enclave’s attributes, which is where you’ll find the DEBUG bit:
~$ xxd -s 96 -l 8 -g 0 -e ~/quote.dat
00000060: 0000000000000007 ........
In this example, bits 0, 1, and 2 have been set. The attributes are defined in the sgx_attributes.h header file.
grep SGX_FLAGS /opt/intel/sgxsdk/include/sgx_attributes.h
#define SGX_FLAGS_INITTED 0x0000000000000001ULL /* If set, then the enclave is initialized */
#define SGX_FLAGS_DEBUG 0x0000000000000002ULL /* If set, then the enclave is debug */
#define SGX_FLAGS_MODE64BIT 0x0000000000000004ULL /* If set, then the enclave is 64 bit */
#define SGX_FLAGS_PROVISION_KEY 0x0000000000000010ULL /* If set, then the enclave has access to provision key */
#define SGX_FLAGS_EINITTOKEN_KEY 0x0000000000000020ULL /* If set, then the enclave has access to EINITTOKEN key */
#define SGX_FLAGS_KSS 0x0000000000000080ULL /* If set enclave uses KSS */
From this, we can see that the DEBUG bit (bit 1 = 0x2) is set, which says that we were running a debug enclave. This result is exactly what we expect since we built the enclave in debug mode. As another rule, a production service should never trust a debuggable enclave.
Quotes with TCB Issues
It is possible for an Intel SGX quote to pass the verification process if it has TCB issues. For example, the following shows the output of the QuoteVerificationSample app when verifying a quote whose TCB is out of date:
$ ./app -quote ../QuoteGenerationSample/quote_tcbissue.dat
Info: ECDSA quote path: ../QuoteGenerationSample/quote_tcbissue.dat
Trusted quote verification:
Info: get target info successfully returned.
Info: sgx_qv_set_enclave_load_policy successfully returned.
Info: sgx_qv_get_quote_supplemental_data_size successfully returned.
Info: App: sgx_qv_verify_quote successfully returned.
Info: Ecall: Verify QvE report and identity successfully returned.
Warning: App: Verification completed with Non-terminal result: a002
===========================================
Untrusted quote verification:
Info: sgx_qv_get_quote_supplemental_data_size successfully returned.
Info: App: sgx_qv_verify_quote successfully returned.
Warning: App: Verification completed with Non-terminal result: a002
From /opt/intel/sgxsdk/include/sgx_qve_header.h, we can see that error type A000 is a verification failure, and that sub-code 0002 refers to an out of date TCB.
#ifndef SGX_QL_QV_MK_ERROR
#define SGX_QL_QV_MK_ERROR(x) (0x0000A000|(x))
#endif //SGX_QL_QV_MK_ERROR
/** Contains the possible values of the quote verification result. */
typedef enum _sgx_ql_qv_result_t
{
SGX_QL_QV_RESULT_OK = 0x0000, ///< The Quote verification passed and is at the latest TCB level
SGX_QL_QV_RESULT_MIN = SGX_QL_QV_MK_ERROR(0x0001),
SGX_QL_QV_RESULT_CONFIG_NEEDED = SGX_QL_QV_MK_ERROR(0x0001), ///< The Quote verification passed and the platform is patched to
///< the latest TCB level but additional configuration of the SGX
///< platform may be needed
SGX_QL_QV_RESULT_OUT_OF_DATE = SGX_QL_QV_MK_ERROR(0x0002), ///< The Quote is good but TCB level of the platform is out of date.
///< The platform needs patching to be at the latest TCB level
This is a non-fatal error, meaning that the quote was verified, but the system that generated it is behind on security patches.
Whether the relying party should trust a quote with TCB issues is a policy decision. Security patches and recommended BIOS configurations harden the system against known vulnerabiilties, and only the service provider can determine how much of a risk such a system presents. These decisions typically take the workload into account, as not all workloads are sensitive to specific vulnerabilities.
Wrapping up
The sample code in the Intel SGX DCAP source code repository contains two utilities that can be used to simulate a remote attestation. With them, you can generate a quote on one system, and then verify that quote on a second system both with and without Intel SGX. These two operations lie at the heart of attestation, and the samples can serve both as a guide for implementing a production service and a known-good test of your environment.