Big Data is newish buzzword in the IT industry. Big Data can mean different things to different people but for the sake of this post, we are going to define it as “a large data set that has traditionally been difficult to manage.” If you have heard the words “Big Data”, you have probably also heard of Isilon.
Today I will quickly describe what Isilon is but mostly answer a question I am often asked, “How do we back up our Isilon cluster?” Isilon was acquired by EMC to fill the need for scale-out NAS storage. Scale-out is the type of storage where everything scales linearly: capacity, network, CPU and memory. This is different than scale-up architecture, which starts with a base NAS “head” unit, and scales capacity only. Generally, there are multiple aspects of data protection: RAID, disaster recovery, backup, etc. Isilon protects files using file level protection vs. the traditional RAID protection. But that is a post for another day. Let’s look at disaster recovery and backups.
So back to that question I am often asked, “How do we backup our Isilon cluster?” Most of the time, my answer is, you don’t. But let’s take a realistic look at big data. Let’s say you have a 250TB “Big Data” cluster; how long would that take to run a full backup against 250TB? Where would you store that 250TB? Is that cost effective? How long would that 250TB take to restore? If you were restoring from tape, realistically, would you feel confident that you could actually restore the data and it would be good data? Okay, 250TB is quit a bit of data to manage for backups and restores, but that pales in comparison to some of the Isilon clusters in production. As of today, Isilon scales to 50PB, that is just a little more than 250TB. Okay, it’s a lot more.
So, traditional backups are not very feasible for Big Data. Now what? Before we answer that, let’s look at why backup data. Agree or disagree, but we backup data for disaster recovery purposes (ie. – get your data off site) and to recover files locally if files are deleted or become corrupt. This provides us with a two-pronged approach, 1) replicate data to get your data off site and 2) schedule local snapshots for your data to protect from deletion and corruption. Isilon provides software for both of these functions, SyncIQ for replication between Isilon clusters and SnapIQ for local snapshots. Fortunately, these tasks can be scheduled or run automatically once configured, and little management is required from admins to maintain this functionally. Now imagine the amount of user interaction it takes to manage multiple petabytes of backups. If you just groaned, you know snap and replicate is a much better approach to protecting your Big Data environment.
That being said, Isilon and most NAS providers, do provide a way to backup data, in the traditional sense, by using NDMP (Network Data Management Protocol). NDMP is a protocol built to backup file data in a block fashion. It is much faster than backing up a file shares. If traditional backups are desired or required on a smaller subset of data, you are in luck. The entire cluster doesn’t need to be backed up; Isilon gives the ability to select individual folders for backup. So, if you really need to have a set of folders backed up to disk or tape, you have that option. And of course, you can backup the file shares without NDMP by pointing your backup software to the file share, but this not a very efficient way to backup data and can take a very long time.
These are some general rules I use when protection Big Data on Isilon, but these same rules can apply to just about all Big Data workloads. Traditional backups just don’t cut it for Big Data, a different approach is needed; snap and replicate.