C++ Transactions for Persistent Memory Programming

Overview

Transactions are a vital part of persistent memory programming because of their ability to protect data structures from unexpected interruptions with little added code. In this article we describe what a transaction is, show the different types of transactions provided by the Persistent Memory Development Kit (PMDK), and look at the usage of transactions across multiple programs.

This article assumes you have a basic understanding of persistent memory concepts and are familiar with features of the PMDK. If not, visit the Intel® Developer Zone Persistent Memory Programming site, where you’ll find the information you need to get started.

What are Transactions?

Use of a transaction enables you to secure data structures from corruption that could happen as a result of an interruption such as a power failure. It does this by rolling back the uncommitted changes if an interruption occurs. For example, if a system crashes while performing an operation, a transaction protects the operation and rolls back to the previous state before a crash.

The PMDK provides several transactional libraries: libpmemobj, libpmemblk, and libpmemlog. Each library has several language bindings including C, C++, Java*, and Python*; more information can be found on the PMDK project website. PMDK transactions provide two ACID (atomicity, consistency, isolation, durability) properties – atomicity and durability. Here is a detailed look at what each property entails:

• Atomicity: All changes within the transaction are made durable or none of them are.
• Consistency: Data remains intact before and after interruption. This only holds true for PMDK metadata. Application data consistency is implemented within the application.
• Isolation: Each transaction is consistent with respect to the others and is implemented by ensuring proper locking within the application.
• Durability: Committed data is saved by the system such that, even in the event of a failure or system restart, the data is available in its correct state.

Using transactions, you can convert regular data structures into persistent data structures with very few code changes, which we will cover later. First, let’s look at the three types of transactions provided by the PMDK.

Three Types of Transactions

The PMDK implements three types of transaction models: manual, automatic, and closure, which are described below in more detail using the C++ language bindings.

Manual Transactions

Manual transactions must be executed inside a persistent memory pool. A pool is essentially a memory-mapped file inside persistent memory. Most importantly, each manual transaction must be manually committed when it is complete.

If you want a refresher or want to read more about pools, check out the introduction to libpmemobj tutorial at pmem.io.

Let’s look at the syntax for a manual transaction:

auto pop = pool ::open(“/path/to/poolfile”, “layout string”)
{
    transaction::manual(pop, persistent_mtx, persistent_shmtx);
    // any tasks being performed
    transaction::commit();
}
auto aborted = transaction::get_last_tx_error(); 

In the first line, a persistent memory pool is opened and referenced using the pop (pool object pointer or pool handle). Following that is a code block {...} that will contain our transaction. The constructor for the manual transaction, transaction::manual, contains the pool handle, and then accepts an arbitrary number of locks (persistent_mtx and persistent_shmtx) that are held until the end of the transaction. Inside the transaction you can place any actions that need to be completed. The updates are committed by calling transaction::commit(). Once the transaction successfully commits, calling get_last_tx_error() reports any errors.

Let’s look at a real example of a manual transaction in a code sample called PMAN. PMAN is a game of Pac-Man* designed to take advantage of persistent memory using the libpmemobj library and the sample code can be found the PMDK examples directory on GitHub. Read the article Code Sample: PMAN – A Persistent Memory Version of the Game Pac-Man for details. Let’s look at the manual transaction section of that sample:

{
    transaction::manual tx(pop);
    if (intro_p->size() == 0) {
        for (int i = 0; i < SIZE / 4; i++) {
            intro_p->push_back(
                make_persistent<intro>(i, i, DOWN));
            intro_p->push_back(make_persistent<intro>(
                SIZE - i, i, LEFT));
            intro_p->push_back(make_persistent<intro>(
                i, SIZE - i, RIGHT));
            intro_p->push_back(make_persistent<intro>(
                SIZE - i, SIZE - i, UP));
        }
    }
    transaction::commit();
}

The transaction above is protecting the process of initializing the game points and direction. In this case, intro_p is a list of objects used in the game. The transaction ensures that the game does not end up half-initialized.

Automatic Transactions

Automatic transactions are simpler than manual transactions. They act in the same way but are easier to implement because they handle all the commit/abort semantics behind the scenes. This allows you to avoid writing transaction::commit() at the end of each transaction. A great benefit of automatic transactions is that they throw an exception from the constructor if there is an abnormal event, so you don’t have to explicitly check to see if there was an interruption. These transactions are only available in a C++ compiler supporting the C++17 standard or later.

The following shows an example of automatic transactions:

auto pop = pool ::open(“/path/to/poolfile”, “layout string”)
try {
    transaction::exec_tx(pop, persistent_mtx, persistent_shmtx);
    // any tasks being performed 
} catch(..){
    // do something
} 

In this case, the transaction looks very similar to a manual one, except that we call transaction::exec_tx() directly without constructing an object.

auto aborted = transaction::get_last_tx_error()

For the above example, we use a try catch block for the transaction. This format is used to catch exceptions and, although not required, if an exception is not caught the program will end. One thing to note here is that locks are held only in the try block.

Now let’s take a look at a real usage example of an automatic transaction in the Panaconda code sample. Panaconda is a game of Snake that demonstrates persistent memory pools, pointers, and transactions. You can learn more about how it works by reading Panaconda - A Persistent Memory Version of the Game Snake. For now, let’s dive into a small example:

try {
    transaction::exec_tx(state, [&]() {
        r->get_board()->creat_dynamic_layout(i, line);
    });

} catch (transaction_error &err) {
	std::cout << err.what() << std::endl;

} catch (transaction_scope_error &tse) {
	std::cout << tse.what() << std::endl;
}

Above you can see the call to state, which is the pool handle. This transaction wraps the creation of a dynamic layout.

Closure Transactions

Closure transactions work in the same way as automatic transactions, except these transactions are available with the C++ 11 standard or later. In closure transactions you also do not need to check whether a transaction was committed or if it was interrupted because it will throw an exception if there is an error. A noticeable difference between closure compared to both manual and automatic transactions is that the locks are passed through the constructor after the main body of code is passed in. This syntax follows a functional programming style—it allows you to pass the code into it in a function-like manner. Essentially, the body of the code in closure transactions is passed in to the constructor.

auto pop = pool <root>::open(“/path/to/poolfile”, “layout string”)
transaction::exec_tx(pop, []{
    // any tasks being performed 

}, persistent_mtx, persistent_shmtx );

Now, let’s take a look at a real usage example of a closure transaction that can be found in the Simple Grep code sample. Grep is a program that scans input line by line, and then outputs the lines that match the provided pattern. You can learn in more detail about how it works reading Boost Your C++ Applications with Persistent Memory – A Simple grep Example, but for now let’s dive into a small example:

transaction::exec_tx (pop, [&] { 
    /* LOCKED TRANSACTION */
    /* allocating new files head */
    persistent_ptr<file> new_files = make_persistent<file> (filename);

    /* making the new allocation the actual head */
    new_files->set_next (files);
    files = new_files;
    nfiles = nfiles + 1;
    new_file = files.get ();
},
pmutex); /* END LOCKED TRANSACTION */

In this case the transaction is protecting the creation of a new file. The transaction is also incorporating the use of a persistent mutex to provide isolation, so multiple threads executing the above transaction won’t corrupt the data structure. Here you can see that this lock is placed after the lambda function is passed in.

Summary

The goal of this article is to introduce you to the three transaction types available in PMDK that you can implement to make your program persistent. So, let’s do a quick recap of what we learned.

Transactions are a way to secure data structures from corruption that could happen as a result of an interruption such as a power failure. There are three different types of transactions: manual, automatic, and closure. The main difference is that manual transactions must be committed at the end, while automatic and closure transactions do not. After discussing each transaction, we showed a real usage of each transaction type from the PMAN, Panaconda, and Simple Grep code samples. You can find more persistent memory programming examples in the PMDK examples repository on GitHub.