Since we announced our next generation StorNext 5 Appliances three weeks ago, we’ve been getting requests for more background about how we’ve achieved such significant increases in performance, scalability and flexibility. To dive deeper into how we built StorNext 5 from the ground up, I’ve tapped Skip Levens, director of technical marketing, to detail some of its core design features.
A new standard for metadata efficiency. The footprint of file metadata in StorNext has always been small, but with StorNext 5, the engineering team was able to shrink the size of metadata volumes even further. That means StorNext metadata controllers can use the space dedicated to file metadata more efficiently, allowing StorNext 5 file systems to scale to 5 billion files. That’s really important since modern workflows are more likely to combine large streaming files with smaller files like audio, proxy and content-based metadata files.
Smaller metadata means large-scale performance. More compressed metadata and better caching techniques also unleashes hidden performance in existing metadata controllers, especially in larger deployments with frequent metadata reads and writes. With StorNext 5, large-scale file creation and deletion can also occur concurrently with high-performance access. That means customers won’t need to set up a “delete window” of limited file system access when tearing down a file system, say between analysis runs or after a project is completed.
Preventing fragmentation by design. As more and more users and applications create, delete and access files in large-scale file systems, fragmentation can occur, leading to file system slowdowns. Instead of turning to a defragmentation solution, Quantum engineers virtually eliminated the problem altogether by re-architecting StorNext 5’s on-disk file structures, caching techniques, and metadata tracking capabilities. No slowdowns over time means staff won’t waste time tearing down and rebuilding volumes as they age.
Evolving beyond SAN. No single network topology is ideal for every user and every application in today’s workflows. In many cases one and 10 Gb Ethernet IP networks are fast enough and more cost effective than dedicated Fibre Channel networks, especially for smaller files that have typically performed poorly in StorNext over Fibre Channel. That all changes with StorNext 5, which delivers high performance and high efficiency over both Fibre Channel and IP networks, and offers 100% performance improvement for small file operations.
Lower latency and higher I/O. Need low latency and high I/O operations for HPC-style number crunching? StorNext 5 takes full advantage of SSD capabilities and supports InfiBand connections both between the StorNext clients and the metadata controller (MDC), and between the MDC and the storage. With StorNext 5, analysis workflows will see shorter compute runs for quicker results, which can mean more time for more runs for better overall results.
Less manual tuning to achieve highest performance. Before StorNext 5, customers who demanded highest performance would manually configure parameters like block sizes to tweak operations based on their predicted file system usage. In StorNext 5, there’s no need for manual tuning. StorNext 5 monitors system usage and will adjust operations based on parameters like the number of clients accessing the file system or number of threads open—all in real time.
Non-disruptive upgrades. Upgrading to StorNext 5 is a straightforward process. It’s backward compatible with previous versions so partner integrations will work without any changes in your environment. What’s more, after StorNext 5 is installed, you won’t have to pause file system access for the next StorNext upgrade. We’ve built StorNext 5 so future upgrades will use metadata controller failover capabilities to keep files accessible throughout the process.
About our performance benchmarks. Our engineers have made some impressive performance improvements in StorNext 5: up to 10 times faster on file creation and deletion and up to twice as fast for small file operations. These results are 100% due to software improvements—our benchmark tests compared StorNext 5 vs StorNext 4 running on the same hardware. That means new StorNext 5 appliances will show even greater performance due to next generation hardware.
So when can you get your hands on StorNext 5? StorNext 5 appliances will be made generally available later this calendar year, while software-only StorNext 5 solutions will be available in the first half of 2014.