The world increasingly runs on hyperscale databases. Across fields as diverse as ecommerce, genomics, and inventory management, the speed of database analysis and actionable insights seemingly scales in tandem with database size. However, with increasing database requirements, conventional DRAM-only system memory approaches have limited ability to scale in performance. This often leads organizations to consider moving to an in-memory database or a NoSQL database to keep pace with modern business. Such approaches involve tradeoffs, including potential incompatibility with existing queries and coding. Additionally, these scaling methods may prove cost-prohibitive or downright impossible due to RAM costs and capacity constraints.
Sharding helps mitigate these concerns. Sharding allows administrators to break the workload into manageable chunks that can be processed more quickly and efficiently across multiple nodes. Many years of refinement have proven that sharding is preferable to forcing the database to seek information across the storage bus in NAND or hard disk media, which can be orders of magnitude slower than system memory.
Sharding a vastly scalable database platform such as Oracle Database 21c offers numerous advantages, including having no need to rewrite code and very compelling performance scaling. Oracle sharding with Intel® Optane™ Persistent Memory (Intel® Optane™ PMem) goes a step further, delivering the additional performance benefits of having extensive data capacity directly on the memory bus. This eliminates the latencies introduced by I/O across a conventional storage bus and is ideal for workloads that need lower latency than flash media can offer. With Oracle Database 21c, SQL can run directly on database data mapped to and residing in PMem (thus removing the need for a large buffer cache). Moreover, Oracle sharding with PMem dramatically improves solution price points1 relative to an in-memory, DRAM-only solution, all while delivering data persistence benefits.
Benefits of Sharding With Intel® Optane™ Persistent Memory
Intel Optane PMem offers several advantages for relational databases. Most notably, PMem modules ship in 128, 256, and 512 GB capacities, whereas conventional DRAM DIMMs usually range from 8 GB to 64 GB. This allows PMem, which occupies the same memory slots as conventional RAM, to provide far more memory capacity per node while delivering near DRAM-level performance.
Sharding also provides numerous scale-out benefits for databases. For instance, performance gains from sharding are remarkably linear; doubling the number of shards generally doubles the performance. Sharding is inherently redundant and fault-tolerant. However, unlike common redundancy approaches such as RAID-5, one shard failure will not impact the database’s performance or availability. Similarly, shards can be upgraded individually without affecting other shards. Administrators can alter a small subset of data and observe the change’s impact before rolling out an update to the entire database.
Sharding is inherently parallelized. If a sharded database distributes across ten nodes, a query can run simultaneously on all ten systems. When the desired data is returned from a node’s system memory, the results will be faster than running the same query on a single-node database and fetching that data from significantly slower storage after a cache miss. Oracle achieves this sharded configuration with a “shared-nothing” architecture, meaning no specialized software or network adapters are needed. Oracle Database 21c is a massively parallel-processing architecture, able to shard structured and unstructured databases across up to 1,000 nodes. Note that sharding is supported specifically in Oracle Database 21c Enterprise Edition (EE) and Enterprise Edition on Oracle Exadata Database Machine (EE-Exa).
Unlike DRAM, PMem can operate in a non-volatile way. In the event of a power interruption, data persists in memory (just like with storage). In-memory data persistence then enables extremely fast reboot/restart times, since the potentially terabytes-large dataset already resides in system memory and there is no need to pull it from storage. In fact, with the dataset located entirely within memory, there is no need for SSDs or hard disks to host the database within nodes, which helps reduce capital costs and the number of components needing maintenance. Additionally, new Oracle algorithms prevent partial or inconsistent stores to PMem, further ensuring data integrity.
With in-memory data persistence, Oracle Database 21c can ensure atomicity (literally “undividable” from ancient Greek). Imagine entering an order for air travel, hotel accommodations, and car rental into a travel database. If one element of that order fails, the entire order needs to fail because all three components are needed for a successful trip. This handling of multiple data operations across multiple data stores becomes imperative with sharding. Fortunately, with non-volatile PMem and Oracle’s Data Guard services, the platform can provide ensured atomicity and what Oracle calls “absolute data consistency.”
Enabling Breakthrough Database Performance
Intel Optane PMem enables easy, cost-effective, and high-performance relational database scale-out using Oracle sharding. With PMem, users can have massive datasets with in-memory-class performance and no compromise on any traditional database capabilities while gaining memory persistence. Data is protected at multiple levels, and system recoveries occur with exceptional speed due to zero dependency on loading datasets from storage.
Fundamentally, PMem and Oracle sharding tackle the biggest challenge in hyperscale database systems: time. Queries often consume minutes and reports can take hours. By accommodating petabyte-level datasets in memory across a sharded Oracle Database deployment, these time-sinks can radically diminish, and enterprise productivity can scale accordingly.
To learn more, please visit:
- Intel Optane Persistent Memory
- Oracle Database Technologies
- Understanding Database Sharding
- Using Oracle Sharding
1Newegg pricing comparison as of April 12, 2021: https://www.newegg.com/p/0RN-0020-000M5?Description=optane%20persistent%20memory&cm_re=optane_persistent%20memory-_-0RN-0020-000M5-_-Product&quicklink=true and
https://www.newegg.com/p/pl?d=256gb+ddr4+server&N=100007952%20601281939&isdeptsrh=1