Address Range Partial Memory Mirroring

Operating System Interface Specification

Contents

Introduction
Presenting Mirrored Memory to OS—Boot Time
Runtime Hot-Added/Removed Mirrored Memory
UEFI Variables for Memory Address Range Mirroring

• Background on UEFI Variables
• UEFI Variables for Memory Address Range Mirroring
• GetVariable()
• SetVariable()
• Management Interface Flow—BIOS
• Management Interface Flow—OS

Tables

UEFI runtime services
MirrorStatus values

Introduction

The Intel® Xeon® processor family-based platform offers a RAS (Reliability Availability Serviceability) feature called Memory Mirroring. This feature allows users to configure the memory in a highly reliable mode such that system uptime can be maintained—even when a memory component is affected by an uncorrectable fault. When enabled, a trade-off between memory capacity and reliability is required. For example, user-accessible memory may be reduced to half of the total available memory.

The Intel® Xeon® processor E7 v3 product family-based platform introduces a new layer of support for partial memory mirroring called Address Range Mirroring. When Address Range Mirroring is used, the platform allows an OS to specify a subset of total available memory for mirroring (and also to optionally specify whether to mirror the range 0–4 GB). This capability allows a user to make an appropriate trade-off between non-mirrored memory range and mirrored memory range, thus optimizing total available memory while keeping highly reliable memory range available for mission-critical workloads and/or kernel space.

Legacy memory mirroring is transparent to the OS; however, address range mirroring requires a firmware-OS interface for a user to specify the desired subset of memory to mirror. To fully utilize Address Range Mirroring:

• Present partial or total mirrored memory on the platform to the OS.
• Provide the OS a method to request the amount of mirrored memory that takes effect on subsequent boots.

This specification defines the interface between firmware (BIOS/SMM) and OS for presenting blocks of memory that are mirrored. It also defines how an OS can allocate the percentage of memory (and, optionally, whether 0–4 GB should be mirrored) over a subsequent boot.

Presenting Mirrored Memory to OS—Boot Time

An existing UEFI Call GetMemoryMap() returns to the OS all the address ranges presented by the platform. Partial memory ranges that are set to be mirrored are indicated by the Attribute field in the EFI_MEMORY_DESCRIPTOR. Any mirrored memory must be associated with the following attribute.

#define EFI_MEMORY_MORE_RELIABLE     0x0000000000010000 

Although the scope of this specification is to support Memory Address Range Mirroring, the above UEFI scheme can also be used to expose Total Mirrored Memory to the OS.

Note that the UEFI call GetMemoryMap() is part of UEFI BOOT Services and hence cannot be invoked after ExitBootServices() has been executed. As such, this scheme defines the state of mirrored memory at boot time only. That is, mirroring capabilities of hot-added memory cannot be described using this scheme. Similarly, any subsequent loss of redundancy in memory after boot time is not captured by this scheme.

Runtime Hot-Added/Removed Mirrored Memory

Hot-added memory can be presented to an OS with a platform using ACPI namespace object: PNP0C80. Refer to section 5.2.21.10 of ACPI v5.1 specification for further details.

A Memory Device object contains a _CRS method (Current Resource Setting – refer to section 6.2.2 of ACPI v5.1 specification) that describes the address spaces that correspond to the memory device in question. In case of hot-plug memory, the device indicates the hot-added memory. For memory presented statically at boot, the platform represents the entire memory range throughout a single memory device using a _CRS method that comprehends non-contiguous address ranges (if needed).

Memory ranges that are mirrored can be indicated in the _CRS method by using the _ATT (Type-Specific Attribute) attribute fields of the Extended Address Space Descriptor. See section 6.4.3.5.4 of ACPI v5.1 specification for further details.

In the particular case where a change to memory redundancy status takes place (such as loss of memory-mirroring redundancy), the platform notifies the OSPM (OS Power Management) layer with a bus check event (Notify 0x00) at the corresponding memory device. The event trigger causes a reevaluation of _CRS, which results in the _ATT attribute field indicating the loss in redundancy.

Scope (\_SB){
Device (MEM0) {
Name (_HID, EISAID (“PNP0C80”))
Name (_CRS, ResourceTemplate () {
	…
})
} // MEM0
} // _SB
Scope(\_GPE)
{
Method(_L00) {
	Notify(\_SB.MEM0, 0x00) // Bus Check
}
}

Note:_ATT attribute uses the UEFI attribute now.

UEFI Variables for Memory Address Range Mirroring

Unified Extensible Firmware (UEFI) variables enable OS or system software to identify which memory blocks the platform is mirroring. They are also used to permit OS requests for mirroring on a subsequent boot.

Supported features are:

• Enable or disable mirroring of memory below 4 GB¹ for a subsequent boot.
• Specify amount of the total memory capacity (up to 50 percent) that is to be mirrored for a subsequent boot.
• Indicate whether memory below 4 GB is mirrored for current boot.
• Indicate the amount of memory mirrored on the current boot.
• Indicate status of requested memory redundancy (percent at and below 4 GB) during last boot, as well as the status of the request – SUCCESS, FAILURE, PARTIAL.

Background on UEFI Variables

This section provides a high-level overview of UEFI variables for completeness. Refer to section 7.2 of the UEFI Specification v2.4 for detailed description on using UEFI variables.

Variables are defined as key/value pairs that consist of uniquely identifying information plus attributes (the key) and arbitrary data (the value). Variables are intended for use as a means to store data that is passed between the EFI environment loaders and other applications that may be implemented in the platform and EFI OS running in the EFI environment.

The following UEFI runtime services are available for managing variables:

Table: UEFI runtime services

UEFI Variables for Memory Address Range Mirroring

Use the following GUID to associate with both the variables:

#define ADDRESS_RANGE_MIRROR_VARIABLE_GUID { 0x7b9be2e0, 0xe28a, 0x4197, 0xad, 0x3e, 0x32, 0xf0, 0x62, 0xf9, 0x46, 0x2c }

The variable data consists of the following structure:

typedef struct{

UINT8       MirrorVersion;

BOOLEAN     MirrorMemoryBelow4GB;

UINT16      MirroredAmountAbove4GB;

UINT8       MirrorStatus;

} ADDRESS_RANGE_MIRROR_VARIABLE_DATA;

• MirrorVersion should be set to 1 in the initial implementation.
• MirrorMemoryBelow4GB set to true to mirror memory below 4 GB.
• MirroredAmountAbove4GB is the amount of available memory above 4 GB that needs to be mirrored, measured in basis points (hundredths of percent – 12.75% = 1275).
• MirrorStatus indicates whether the desired parameters have been successfully applied when setting up the mirror.

Table: MirrorStatus values

In multi-socket systems, a platform is required to distribute the mirrored memory ranges such that the amount mirrored is approximately³ proportional to the amount of memory on each NUMA node. For example, on a two node machine with 64 GB on node 0 and 32 GB on node 1, a request for 12 GB of mirrored memory should be allocated with 8 GB of mirror on node 0 and 4 GB on node 1.

For example, if the total memory in the system is 48 GB, and 12 GB memory needs to be mirrored (with all of the memory below 4 GB mirrored), the amount would be:

• Total Memory⁴ = 48 GB
• Total Memory above 4 GB = 46 GB (assuming TOLM = 2 GB)
• Percentage  = 10/46 * 100 = 21.74% = 2174 basis points

Consider a 2S system with 32 GB of memory attached to socket 0 and 16 GB on socket 1. Then socket 0 should mirror 8 GB of memory and socket 1 mirror 4 GB to maintain the requested 21.74%. This ensures that the OS has an adequate amount of mirrored memory on each NUMA domain.

Size of the ADDRESS_RANGE_MIRROR_VARIABLE_DATA is calculated as follows:

#define ADDRESS_RANGE_MIRROR_VARIABLE_SIZE sizeof(ADDRESS_RANGE_MIRROR_VARIABLE_DATA)

The Variables will be associated with the following Attributes⁵:

#define ADDRESS_RANGE_MIRROR_VARIABLE_ATTRIBUTE \
(EFI_VARIABLE_NON_VOLATILE | EFI_VARIABLE_BOOTSERVICE_ACCESS | \
EFI_VARIABLE_RUNTIME_ACCESS)

Where:

#define EFI_VARIABLE_NON_VOLATILE  0x00000001
#define EFI_VARIABLE_BOOTSERVICE_ACCESS  0x00000002
#define EFI_VARIABLE_RUNTIME_ACCESS  0x00000004

The Variables are associated with the following Names:

#define ADDRESS_RANGE_MIRROR_VARIABLE_CURRENT “MirrorCurrent”

• This variable is written by the BIOS and read by the OS to communicate the current status of address range mirroring on the platform.

#define ADDRESS_RANGE_MIRROR_VARIABLE_REQUEST “MirrorRequest”

• This variable is written by the OS to request a new mirror configuration on the next boot.

GetVariable()

GetVariable() is a UEFI runtime service that is invoked using the following parameters:

Prototype

typedef
EFI_STATUS
GetVariable(
	IN CHAR16     *VariableName,
	IN EFI_GUID  *VendorGuid,
	OUT UINT32    *Attributes,
	IN OUT UINTN  *DataSize,
	OUT VOID    *Data
);