Introduction
Persistent Memory Container Storage Interface (PMEM-CSI) is a storage driver for container orchestrators like Kubernetes*. It makes Intel® Optane™ DC persistent memory available as a filesystem volume to container applications.
In this article, we provide an overview of current features, future work, and explain how to get started with PMEM-CSI.
Features
Persistent Memory Management
PMEM-CSI manages persistent memory (PMEM) through libncdtl. Optionally, it can layer the Logical Volume Manager (LVM) on top of the PMEM block devices.
A cluster administrator must choose between these two approaches and does that by deploying PMEM-CSI differently. When deploying for LVM, some initialization containers take care of setting up the PMEM namespaces and LVM before starting the driver. This is currently the default because it is more stable.
When deploying for direct usage, the driver starts immediately and then allocates PMEM when needed.
Persistency Models
In a typical CSI deployment, volumes are provided by a storage backend that is independent of a particular node. When a node goes offline, the volume can be mounted elsewhere. However, PMEM volumes are local to a node and thus can only be used on the node where they were created.
To assist application developers in getting the most out of local PMEM storage, PMEM-CSI currently supports two persistency models: cache volumes and persistent volumes.
For cache volumes, one actual volume is allocated on a certain configurable number of nodes. Each volume is independent of the others. Applications using such a cache can run on any of the nodes where the cache was created. This is useful for applications that use a pre-populated local cache when they start up.
The normal persistent volumes are also supported. Here each volume in Kubernetes corresponds to one volume on one particular node.
Future Work
Currently, PMEM-CSI is of alpha quality. It is only provided as source code. Work is in progress to address known limitations (mostly around error recovery) and to provide pre-built container images.
Another persistency model is ephemeral. In this model, a volume is allocated each time an application starts on a node and deleted immediately when it stops. This is useful for local scratch data. Supporting this for CSI drivers is currently in progress in the kubernetes-csi GitHub* repository.
Getting Started
The PMEM-CSI setup instructions explain how to download the source code and build the container image that is needed for deploying the driver. It also shows how to deploy on a cluster which has Intel Optane DC persistent memory.
If such hardware is not available, then the instructions from the end-to-end testing section of PMEM-CSI documentation can be used to bring up a virtual cluster instead. The virtual cluster will use the latest Clear Linux* OS and Kubernetes 1.13 and have four nodes, of which the three worker nodes have emulated PMEM.
The prerequisites for this are a Linux machine, QEMU, Docker*, a local Docker registry (which itself can run on Docker), and sudo rights.
To try out PMEM-CSI on the virtual cluster, follow the instructions that are printed at the end of the `make start` output.
Summary
In this article, we provided an overview of the PMEM-CSI storage driver for Kubernetes. We described the persistency models and future work, and provided links to the repository with setup instructions. We invite you to try the driver for yourself, and we welcome your contributions.
About the Authors
This article was written by Patrick Ohly, Olev Kartau, and Amarnath Valluri who are software engineers at Intel Corporation.
References
- CSI specification
- Kubernetes-CSI
- pmem.io
- Persistent Memory Programming at Intel® Developer Zone (Intel® DZ)