By Gavin Cohen – Head of Product Marketing
Let me address the elephant in the room (actually, it’s an elephant put there by our competitors). Can Nimble Storage, a company that has been shipping market-leading hybrid flash arrays for more than five years, really build an All Flash array that leapfrogs the competition? The answer is a resounding, Yes, absolutely! To illustrate this, let’s see how Nimble compares to EMC XtremIO and Pure Storage along the dimensions of:
- Product architecture
- Capacity and performance scalability
- Driving down the TCO
- Delivering non-stop availability.
EMC XtremIO and Pure Storage were among the first purpose-built all flash arrays to focus on reducing the cost of flash capacity through software functionality such as deduplication and compression.
At the time these two vendors designed their products, circa 2009, flash economics were very different than they are today: MLC flash cost $4/GB, eMLC flash cost $10/GB, DRAM cost $10/GB and the largest solid state drives (SSDs) were 512GB. Today, 3D-NAND flash costs $0.40/GB, DRAM costs $8/GB, and the SSDs Nimble uses are 4TB.
In building their all flash arrays, EMC and Pure Storage have taken very different architectural approaches:
XtremIO’s design goal was to build an all flash array with scale-out and deduplication. At the time it was architected, the relative price of DRAM to flash was low-enough that XtremIO built an architecture that requires a very large amount of DRAM compared to flash capacity. XtremIO leverages InfiniBand to facilitate scale-out and uses higher-endurance eMLC flash media.
- eMLC flash is 2X the cost of MLC and other flash technologies, which combined with the additional cost of InfiniBand infrastructure, makes XtremIO one of the most expensive All Flash arrays available.
- XtremIO did not build flash endurance management capabilities into its software architecture due to its choice of higher-endurance eMLC flash. This means if XtremIO ever wants to benefit from current and future generations of cost-optimized 3D-NAND (and beyond) technologies, they will need to build overprovisioning and endurance management at the SSD level – making it more complex and less efficient.
- XtremIO write performance has not increased even across multiple generations of product. This is in contrast to other All Flash arrays that have continued to increase write performance with newer generations of CPU.
Pure’s design goal was the holy grail of building an All Flash array at a lower effective cost than a legacy performance disk array (think EMC Symmetrix). This was achieved by implementing deduplication and compression, and designing an architecture suitable for lower-endurance but more cost-effective MLC flash. Similar to XtremIO, Pure’s architecture also requires significant amounts of DRAM compared to flash capacity.
- In order to support lower-endurance MLC Flash, Pure built flash endurance management and garbage collection software. However, their implementation requires a very large capacity overhead, leaving only 55%-60% of total flash capacity available for use.
- Pure optimized its architecture for reads which made sense five-plus years ago. However, InfoSight data from 7,500+ customers verifies that widespread adoption of server-side caching has resulted in real-life workloads today that are dominated 70%/30% in favor of writes. Feedback from Pure customer deployments points to a very significant performance drop-off under heavy write workloads, and also at varying deduplication rates.
By contrast, Nimble’s design goal was to leverage today’s flash economics and use the experience gained from its Adaptive Flash architecture to build an enterprise focused All Flash array that leapfrogs the competition.
In doing so, Nimble paid careful attention to focus on three of the most impactful areas to enterprise customers: capacity and performance scalability, driving down TCO, and delivering absolute availability / resiliency.
Capacity and Performance Scalability
Nimble supports scale-up (non-disruptive adding of capacity and/or performance) as well as scale-out (the ability to cluster and manage multiple arrays as a single entity with linear capacity and performance scalability). This provides the ability to independently and flexibly scale capacity and performance to meet the needs of varying customer requirements. XtremIO only supports scale-out and Pure Storage only supports scale-up.
The vast majority of real-life I/Os are at block-sizes of either 4KB or 8KB. Please read a blog post by one of our data scientists, “Storage Performance Benchmarks Are Useful – If You Read Them Carefully”, which quantifies this based on real-life measurements across thousands of customers.
As mentioned above, today’s workloads are dominated by writes and, as such, Nimble designed its architecture to to be able to sustain consistent performance under write-intensive workloads and across a wide range of varying deduplication rates.
Driving Down the TCO
There are many factors that combine to enable Nimble to deliver TCO that is 33-66% lower than its all flash competitors:
- Nimble is 9x to 31x more memory efficient than its competitors as measured by the amount of DRAM required to support maximum array capacity. This lowers the Nimble array cost and increases array scalability.
|DRAM||Max. Raw Capacity||Compared to Nimble|
|Current as at Feb 23, 2016.|
- Nimble, like Pure, uses cost-optimized 3D TLC NAND flash. Nimble has implemented a very efficient flash endurance management and garbage collection scheme that yields 20% higher usable-to-raw capacity compared to Pure (Pure 136TB raw yields 80TB usable, Nimble 136TB raw yields 101TB usable). As mentioned above, XtremIO does not implement flash endurance management due to its use of eMLC flash, but does so at twice the cost of 3D TLC NAND.
- Nimble can perform backup, disaster recovery and archiving at one-third the cost of XtremIO and Pure. This is achieved by replicating from a Nimble All Flash array to a cost-optimized Nimble Adaptive Flash array. It should be noted that there are no compromises in doing so because Adaptive Flash arrays are identically featured and can still run workloads at all flash speeds. In comparison, XtremIO and Pure require replication to a second All Flash array (of course this is also an option for Nimble). XtremIO’s replication is not native and is achieved through the use of RecoverPoint, adding cost and complexity.
- Density – a picture showing configurations required to achieve same usable capacity paints a thousand words:
- Data Reduction – Nimble has implemented inline variable-block deduplication, compression, zero-block elimination, thin provisioning and zero-block cloning that achieve similar data reduction to Pure and higher data reduction compared to XtremIO.
- Timeless Storage guarantee – Nimble provides all-inclusive software licensing, does not increase support pricing in later years, does not require “forklift” upgrades, and guarantees SSDs for a seven-plus year lifespan. Nimble also offers the option for a customer to receive a free controller upgrade after three years, and even guarantees that the controller upgrade will be at least 25% faster.
Nimble designed its architecture for six-nines of availability, and is proud to have achieved a measured availability of 99.9997% uptime across more than 7,500 customers.
InfoSight – Nimble developed InfoSight Predictive Analytics as part of its original design. Since releasing InfoSight multiple competitors (including Pure’s Pure1) have appeared, but none has been able to match the breadth and depth of InfoSight Predictive Analytics:
- Predictive Analytics requires big data – Nimble collects and processes more anonymized sensor data points every four hours than there are stars in the Milky Way. This is possible because, since the day Nimble started, a mandatory requirement of every code module check-in is to include comprehensive telemetry collection for all conceivable states. This is done in thousands of places within our code base. Sensor collection of this magnitude and scope cannot be easily retro-fitted to an existing code-base.
- Bottlenecks occur across the infrastructure stack – Sensors are collected not just from the array but also from the networks, physical servers, hosts, and VMs. All of this is correlated to perform end-to-end proactive monitoring and issue-avoidance across the stack. Further, contextual analysis techniques ensure that customers are protected from upgrade issues in the storage and outside the storage. It should be noted that over half of the issues InfoSight identifies are not due to the array itself but have root causes in other parts of the infrastructure stack, such as hosts and networking.
- Installed-base learning – Nimble has amassed what is arguably the largest repository of storage related information in the world. InfoSight data scientists leverage this data to continually improve the performance and availability of Nimble arrays, as well as to implement issue avoidance as new cases arise.
Triple+ Parity RAID – When it comes to resiliency, Nimble’s Triple+ Parity RAID allows a system to continue operating after the loss of any three simultaneous SSD failures (even within the same RAID group). Further, intra-drive parity (the “+” in Triple+) protects against additional read errors on any other SSD. Using our nomenclature, XtremIO uses Double Parity RAID and Pure uses Double+ Parity RAID (they call it RAID-3D). For XtremIO and Pure, three simultaneous failures in the same RAID group will cause irreparable data loss.
A Better Hardware Design – Nimble uses dense dual SSD carriers, but allows each SSD to be individually inserted or removed without impact to any other SSD. Pure also uses dual SSD carriers, but SSDs are not individually accessible. This means if one SSD fails, Pure has to either fail a second drive upon removal, or wait for two drive rebuilds (assuming there’s free space) before the carrier can be removed. XtremIO carriers contain only one SSD.
Application Granular Encryption and Secure Data Shredding – Nimble SmartSecure is performed at application, volume, or data-set granularity. If a data breach occurs, it can easily be contained to just that data set without having to destroy all data in the array. When data is replicated, it remains encrypted over the wire. This level of granularity also facilitates a more secure multi-tenancy operation. In contrast, XtremIO and Pure use system-wide encryption facilitated by the SSD’s Self-Encrypting Drive (SED) technology. However, on both XtremIO and Pure, if there’s a data breach, all data on the array will need to be destroyed. And replicated data needs to be decrypted, and is sent over the wire in the clear.
About That Elephant …
Getting back to the elephant in the room. As these proof points show, the experience we’ve gained in building a market-leading hybrid flash array (our Adaptive Flash array), has provided the foundation for a new generation of All Flash array. One that focuses on absolute performance and scalability, lowering the TCO by 33-66% and delivering non-stop availability. And because Nimble All Flash arrays and Adaptive Flash arrays run the same version of NimbleOS, we can now provide flash for every enterprise application in the data center – without compromise.