Provides Unmatched Performance and Scalability

The Lustre* file system is the ideal distributed, parallel file system for technical computing. Next generation storage built using Lustre software provides software-defined storage (SDS) optimized to address the key storage and data throughput challenges of technical computing.

The latest release of Lustre* software unleashes the performance and scalability of the Lustre parallel file system as a platform for organizations large and small. It allows businesses that need large-scale, high-bandwidth storage to tap into the power and scalability of Lustre—which can be supplemented by worldwide 24x7 technical support from the Lustre experts at Intel. 

The open-source advantage Lustre provides standard interfaces that promote easy integration and interoperability with existing infrastructures, and because Lustre is open-source, IT can grow datacenter storage systems over time and on budget, employing a variety of networking and hardware options without vendor lock-in.

Intel Support for Lustre Software

Enterprise-grade technical support and services

Intel is the primary developer, maintainer, and technical support provider of Lustre software. As the recognized experts in the development of Lustre software, Intel has the technical expertise and experience you need to support even the most complex Lustre configurations.

Key Features of Storage Systems Built Using Lustre Storage Software

Optimized for performance at scale

Lustre software was designed to enable fully parallel I/O across clients, servers, and storage devices. Metadata and data are stored separately to allow each to be optimized yielding maximum performance.

  • Massive data flows efficiently use a high percentage of the underlying storage and network bandwidth for low latency, high-throughput storage performance
  • Improved innovation and precision as enterprises are able to run larger and more complex simulations faster and more easily
  • Native Lustre client optimized for Intel® Xeon Phi™ coprocessors delivers data 10 times faster than a Network File System (NFS)—boosting application performance

Intel® Xeon Phi™ processor optimization

Lustre software provides support for the Intel® Xeon Phi™ Processor in both self-boot and leveraged-boot modes to provide compute nodes direct access to high-performance Lustre storage.

Scalability

Most Scalable POSIX Filesystem 

Lustre clients can read and write directly from storage servers, striping files across multiple targets. This allows both performance and volume to increase up as storage servers are added, and because Lustre provides extremely efficient hardware utilization, storage systems can scale up to thousands of storage nodes. 

Scalable metadata performance 

Remote filesystem directories can now be spread across multiple metadata servers to provide increased metadata performance as the number of metadata servers is increased.

Open Ecosystem

Storage vendor neutral

Lustre is delivered as software-defined storage (SDS) meaning that it is completely vendor neutral. Existing storage clusters can be expanded without having to purchase new hardware from the original vendor, or trying to match existing HW specifications. 

Broad OS support 

Lustre software provides broad support for enterprise-class operating systems such as RedHat Enterprise Linux, SUSE Linux Enterprise Server, and Ubuntu Server.

Windows client support

Lustre software now provides a high availability framework for providing SMB file services to Windows clients. This framework allows Lustre to be integrated into Windows enterprise networks and provide consolidated storage and identity management between Windows and Linux client systems.

Security

Kerberos authentication and encryption

Lustre software provides enhanced authentication and encryption by leveraging Kerberos to provide customers with secure authentication and encryption of information sent over the network.

Mandatory access controls with SELinux*

Lustre now supports the use of SELinux allowing file access control to be moved from user space to kernel space, and providing enforcement of client access to allowed files.

Data Management

Tiered storage

Lustre software provides modular interfaces for tiered storage, and intelligently manages the filesystem through policy driven data movement. This allows storage administrators to migrate “hot” and “cold” data between Lustre and value-optimized storage dynamically to achieve the ideal balance of performance and cost.

Filesystem snapshots

Integrated support for snapshots allow storage administrators to leverage this data protection feature to checkpoint filesystems. Snapshots can be mounted as a new namespace allowing access to previously modified or deleted files in the storage system.

Compression

Lustre and OpenZFS provide on-the-fly data compression maximizing available storage and increasing overall performance by reducing data I/O requests.

Subdirectory mounts

Clients can mount an individual directory, limiting clients from accidentally modifying or deleting files outside of the subdirectory.

Networking

Software defined networking

Lustre Networking (LNET) is abstracted from the physical hardware allowing Lustre to run on top of heterogeneous networks comprised of InfiniBand, Omni-Path, Ethernet, and other interconnects.

Optimized for Intel® Omni-Path Fabric

Intel® Omni-Path Architecture (Intel® OPA) delivers high performance fabric allowing scalability to tens of thousands of nodes and is built on enhanced IP and Intel® technology.

Dynamic network configuration

This powerful extension of the Lustre Networking (LNET) stack simplifies administrative tasks by allowing routes and parameters to be modified on the fly without requiring LNET to be stopped or restarted.

OpenZFS

Extreme capacity

OpenZFS is a 128-bit file system allowing it to address 1.84 x 1019 time more data than 64-bit filesystems such as Btrfs. The actual size limitations of OpenZFS filesystems are so large that they are unlikely to ever be encountered in actual usage.

Copy-on-write

OpenZFS provides copy-on-write mechanism, meaning when data is overwritten a new copy is written to a separate block. In the case that power is lost before the operation completes, the existing metadata will still point to the old block minimizing the impact of data loss.

Hybrid Storage Pools

OpenZFS Hybrid Storage Pools allow SSDs to be used as an Adaptive Read Cache (L2ARC) for HDD-based storage. This cache can greatly improve overall read performance of HDD-based storage pools.

RAID-Z protection against write holes

Tradition RAID suffers from “Write Holes” - multiple disks can’t be updated atomically, causing RAID volumes to become damaged during a crash or power outage. OpenZFS writes are full stripes and leverage the Copy-on-Write mechanism, eliminating “Write Holes” and providing superior data protection when compared to traditional RAID.

Coming Soon

Enhanced small file performance

Small file performance is being enhanced in Lustre by allowing small files to be stored directly on fast SSD-based metadata targets. This reduces RPCs and improves overall performance of both small files and larger sequential IO.

Project quotas

This feature allows administrators to set and enforce quotas based on individual projects providing more flexible quota use when compared with user/group quotas.

Parity declustered RAID

Parity declustered RAID for Lustre allows for a massive improvement in rebuild performance of an array, reducing the time it takes to recover from a drive fault. This improvement comes from the parallelization of rebuild operations and is especially impactful when using extremely large HDD.

Integrated storage encryption

Native data encryption for storage at rest is coming to Lustre when used with OpenZFS backed storage.

Secure delete

Integrated Storage Encryption will also provide the mechanism for secure delete by removing encryption keys from the system. This is more effective than purely overwriting data as modern filesystems and SSDs can obscure the blocks being written to, making other implementations of secure erase ineffective.

Progressive File Layouts

Progressive File Layouts in Lustre allow for file striping to expand from narrow to wide as the size grows. This enables maximum file performance to be maintained regardless of future file size.

Start Using Lustre Today

Lustre is Software-Defined Storage (SDS) capable of handling extremely large volumes of data shared concurrently across clustered servers. Storage powered by Lustre software is a breakthrough technology for addressing the Exascale and emerging high-performance data analytics challenge. 

Now, with Lustre software and support from Intel, organizations have a complete solution to meet their storage and data requirements.