The Virtues of Combining Locally Managed Storage with iRODS

[vc_row][vc_column][vc_column_text][mk_dropcaps style=”simple-style”]I[/mk_dropcaps] recently had the privilege of participating in this year’s annual iRODS users group meeting in Durham, NC. Aside from interacting with a fantastic group of people, I solidified some of my views on the value companies can get from iRODS deployments, and clarified precisely how iRODS and Quantum solutions complement each other. I’m definitely excited that Quantum has joined the iRODS consortium and started product testing.

The Pillars of iRODS Value: Virtualization, Discovery, Automation, Collaboration

The four pillars of iRODS value are particularly important, considering these capabilities can span across multiple groups in an enterprise or institution. As a result, organizations can implement an enterprise-wide data management strategy that can sit above departmental workflows. This makes a lot of sense, and my exposure to users who are adopting iRODS for that purpose is helping to clarify some of these use cases.

I met with one such customer in the genomics field last week who has a variety of storage silos in their shop. One silo is performance storage to support analysis and ingest. There are several NAS silos used for retention and archive. It is not unusual to find shops with multiple silos like this, because of scalability of one array or re-purposing of another. This is where having an overarching management platform such as iRODS can be very useful; it can help tie together all storage resources to enable and even automate data placement based on some useful criteria. And that is why this customer is deploying iRODS.

As the conversation progressed with this customer, we explained how Xcellis — Quantum’s Scale-out NAS product based on StorNext — has some similar capabilities. Xcellis ties together high-performance storage with lower-cost, high-capacity tiers to deliver the required cost/performance/capacity balance for any given workflow. In this case, the customer is already invested in their high-performance tier-one storage, but what they really liked about Xcellis is that it can be tuned for an active archive workflow, where there is performance storage on the front end for incoming or active data, and the back-end storage is virtually unlimited and always open ended.

In other words, as iRODS is used to determine which data goes to archive, Xcellis can act as a single archive namespace that can federate tape, object storage, disk and public cloud, so the customer can leverage whatever storage media they want now for their archive and data protection. And as their needs grow and change, they can add, grow or migrate in whatever direction serves them best.

This notion is consistent with a portion of what I presented at the iRODS UGM (diagram below). While iRODS can manage all the components in an HPC or similar workflow, given that iRODS is an open source, DIY solution, in many cases it is much more efficient to leverage iRODS for its unique value related to discovery and top tier virtualization where other more integrated solutions like Xcellis can be more easily tuned to address very specific needs for localized workflows. This can apply to the archive workflow described above as well as localized HPC workflows also included in Figure 1.

Figure 1:  Quantum Xcellis Localized Workflows under an iRODS Umbrella

Based on the feedback I’ve received from the iRODS UGM, I anticipate even more interesting deployment discussions in the weeks and months ahead. So stay tuned.[/vc_column_text][/vc_column][/vc_row]