Bdtyktl

I am trying to understand and compare storage virtualization methods, including RAID and LVM. I hope I could get some general idea and big picture for the relation between various concepts.

1. 
   

   I was wondering if various storage
   virtualization methods can be
   classified into virtualization at
   the device (disk), partition or filesystem
   levels, as following

   
   
   
   
   
   • 
     RAID belongs to virtualization at the device/disk level, which replace
     physical disks with logical/virtual
     disks.
   
   
   • 
     LVM belongs to virtualization at the partition level, which
     replaces partitions with logical/virtual
     partitionss (also called logical
     volumes).
   
   
   • There is also vitualization at the filesystem level, which
     replaces filesystems with logical/virtual
     filesystems, for example,
     Network-attached storage (NAS).
   
   
   
   



2. If my above understanding is
   correct, does virtualization at each
   level also implement virtualization
   at all lower levels? For example,
   virtualization at partition level
   also implements virtualization at
   device level, and virtualization at
   filesystem level also implements
   virtualization at both partition and
   device levels?




3. How do different levels of
   virtualization affect/determine
   their different areas of
   applications? For example, are there
   applications suitable for RAID but
   not for LVM, and for LVM but not for
   RAID?



4. 
   

   There is a Wikipedia article for
   storage virtualization, where
   there are two main categories of
   methods, block virtualization (which
   can further be classified into
   storage device-based and host-based
   and network-based) and file
   virtualization.

   
   
   

   Compare the article with my
   understanding in part 1,:

   
   
   
   
   
   • Is it correct that storage device-based block virtualization is same as virtualization at the device level. Host-based block Virtualization is same as virtualization at the partition level. File virtualization is same as virtualization at the filesystem level.
   
   
   • But in Host-based block Virtualization#Specific_examples,
     it looks like Host-based block Virtualization includes virtualization at the filesystem level? How shall one understand what is File virtualization then?
   
   
   
   


5. I would rather to single out
   network-based from block
   virtualization in the aforementioned
   Wikipedia article, because for
   storage virtualization over
   network, I think we can also
   classify the various methods into
   the levels of device, partition and
   filesystem? For example, can I say
   Storage Area Network (SAN) belongs
   to the level of device, and
   Network-attached storage (NAS) to
   the level of filesystem?

Thanks and regards!

Might as well provide an answer. Refer to @soandos 's answer for more detailed answering of your specific questions.

LVM vs RAID

RAID, as many have mentioned, is a standard of technologies in which multiple disk drives are allocated together as an array of disks, providing varying level of performance and reliability benefits. For example, RAID 0 provides the best performance one can possibly get with the harddrives, and is extremely sensitive to disk loss (one loss = essentially total loss) whereas RAID 6 provide redundancy even when rebuilding array in a one drive loss scenario. RAID array are usually seen as one single drive to the OS.

One can say that RAID is a many to one mapping.

LVM, on the other hand, allows logical "disk drives" (block device to be accurate, but anyways) to be formed by parts of different disk drives. They exist in a "many-to-many mapping" manner. While one can use LVM to accomplish what can be accomplished by RAID, LVM is actually something that can accomplish much more. For example, to add another disk drive to a RAID array it would likely be necessary to rebuild the whole array from scratch. With LVM, it is just adding a disk drive to the machine, adding the disk drive mapping to a logical volume, and using it (the actual configuration is a little bit more complicated but certainly less than rebuilding a whole array).

1. RAID is a backup technology that insure that in the case of drive failure, all data remains intact, and LVM is Logical Volume Manager that can do many things.


2. It's not.


3. That seems like an odd question. RAID is a way of separating data across drives, so that if one fails no data is lost. LVM is a volume manager that can be used to change the way a user/OS looks at all the hard drives. They have nothing to do with each other (though LVM can implement RAID 1 and RAID 0, that is not its primary focus).


4. The first means you don't have to know what physical device the data is on, and the second means that you can store for lack of a better word the links between files in a more abstract way.


5. As stated above there is no "device level" or "partition level" to talk about so no, you can't refer to them as such.

The article is not well written.

One of the biggest problems is that there are multiple layers of abstraction built into the full stack of storage, and "virtualization" is a fuzzy enough word as to be hard to definitively assign a place to put it. For a good look at the many layers of abstraction in storage, I'll point you at a blog piece I did last year (read it here for the gory details).

In marketing-speak, "Storage Virtualization" is just introducing abstraction where there previously hasn't been any. That can happen at many points depending on the market segment. But that's just marketing. Time for technical.

The storage stack (somewhat simplified):

1. Disk


2. RAID controller


3. Software RAID


4. Volume manager


5. Filesystem


6. Network filesystem


7. Network filesystem client

Disk, even old school spinning magnetic disks, do a level of virtualization. They present a logical view of the actual blocks on the platters (or storage cells for an SSD), and this has been this way since the mid 80's or so. Magnetic drives reserve a certain number of blocks for reassigning blocks that go bad, and the logical view is how this is abstracted away from the disk controller. Technologies like SMART can catch this in the act and report that the drive is "pre-fail" so you can plan your transition accordingly. This has been in place in some form since the 80's.

RAID cards provide another abstraction layer, hiding the true shape of storage from an operating system. This has been in place since the first RAID cards came out in the late 80's, and they've only gotten more complex since then. Cards with write-caches on them provide still another abstraction layer, as writes can be reported as committed before they're actually on a disk somewhere. The really fancy ones (such as those in Storage Area Network arrays) can even write to two separate disk arrays for realtime replication, and the OS is none the wiser.

Once you get into the operating system things get a lot more murky, as each does their own thing. Software RAID (md in Linux) is typically implemented as a low level storage driver that presents the logically combined storage to higher storage layers. As with the RAID cards, you can do all sorts of interesting things here. Some of the "Storage Virtualization" products you see out there are implemented at this stage.

Going higher you get to the volume managers (LVM) can provide for some seriously complex configurations. Where the next layer down aggregates disks into a single virtual volume, the volume managers can combine multiple volumes into a single bigger volume... or split a pool of volumes into an arbitrary number of volumes. Again, some of the Storage Virtualization products you see have a presence in this layer as well.

The next step up is the filesystem. This is the layer where the well known abstractions of "file" and "directory" come into existence. Some filesystems (btrfs, zfs) have volume-manager like features built into them which allows things like snapshotting, deduplication, replication to other devices, and even migration of files between storage tiers. That last bit is not in many filesystems yet, but is definitely a target for Storage Virtualization vendors.

The next step up is the network filesystem. This is things like Samba/CIFS, NetATalk/Appletalk, NFS, and others. If written the right way, these network filesystems can further abstract storage. One product I'm thinking of, Novell's Open Enterprise Server and their ShadowVolumes, takes multiple volumes on different storage (presumably differing speeds/cost) and presents them as a single volume to the network user, and then migrates files between the volumes based on usage statistics. Some of the "Storage Virtualization" appliances you can buy actually do their heavy lifting at this layer.

The last stop on our trip up the storage stack is the network filesystem client in the client machine. It is at this level that the Distributed File System (DFS) exists, which allows a single logical presentation of a filesystem to exist on multiple network filesystems. The client knows that this is a DFS share, and that specific object is a DFS link, and when following it present the specified network-share as a sub-directory of the parent directory. There have been other examples of abstraction at this level, but DFS is perhaps the most common.

One thing to keep in mind is that through all of this, each layer of the storage stack is independent of those above it. Many layers are already doing block-level abstraction, so adding one more doesn't change a whole lot. File-level abstraction has to happen near the top of the stack (for that's where the file-systems live) and the impacts lower down are highly decoupled to the point that it may not even be noticed.

At it's core "Storage Virtualization" is still mostly a marketing term for something that has been happening since the dawn of the PC era (if not earlier), only this time the new abstraction layers are happening when virtualization is the buzzword of the moment.

The one new abstraction layer I know of is something called a "Storage Router", which you'll only ever see on large Storage Area Networks. This device has several different storage arrays behind it, and presents those separate arrays as single array with multiple LUNs. The fancier ones can do interesting block-level abstractions like moving rarely used blocks to slower/cheaper storage and moving the highly used blocks to SSD layers, or handle realtime replication between storage arrays that normally wouldn't allow that kind of thing.

P.S.: RAID is not just device-level virtualization. I'm working with a storage array right now that takes slices of disks and assigns them to different RAID groups. It is working just fine (I'm doing it right now), and I have both RAID1 and RAID5 volumes on the same disk device. Lose two drives and the RAID5 volumes are toast, but the RAID1 volumes on the same disks are just fine.