In this lesson, you're going to learn
how to use logical volumes
to create a more flexible structure
for organizing data on storage devices,
because after all, partitioning is not the only way
to divide up a storage device logically.
As we discussed before,
we've made the use of physical storage devices,
but device mapping turns these physical storage devices
into virtual storage devices.
In Linux, the device mapper creates the virtual device
and passes data from that virtual device
to one or more physical devices.
DM-multipath is a feature of the Linux Kernel
that provides redundancy and improved performance
for block storage devices.
If one path fails, DM-multipath will switch
to one of the other paths that remains,
keeping that storage device available
for reading and writing.
The multipath tools package enables you
to manage DM-multipath for storage devices
and typically, you're going to find the configuration file
located at /etc/multipath.conf.
The mdadm command is a tool that's going to be used
to manage software-based RAID arrays.
In a RAID array, data is stored
across multiple physical storage devices
and those devices are combined
into a single virtual storage device.
This type of software-based RAID configuration
is an alternative to using device mapper
and the DM-multipath.
The mdadm tool enables you to create,
manage and monitor RAID arrays.
Now, before we dive too much further
into logical volumes and RAIDs,
it's important for us to take a quick detour
and do a quick review of RAIDs
from your previous A plus studies.
A RAID is a redundant array
of independent or inexpensive disks.
RAIDs are classified based on their level,
such as RAID 0, RAID 1, RAID 5, RAID 6 and RAID 10.
Each RAID has either striping, mirroring,
parity, or a combination of these features,
depending on the RAID level.
Striping refers to a technique used by RAIDs
in which multiple smaller physical disks
can be combined to logically act as a single larger disk.
For example, if I wanted to have a four terabyte hard drive
for storage in my Linux system,
but I only have two two terabyte hard drives available,
I can use striping to combine them logically
and my system is going to treat them
as a single four terabyte hard disk drive.
Mirroring, on the other hand,
is the second technique that's used in a RAID.
With mirroring, two physical hard disk drives
are going to be combined into a single logical volume.
An identical copy of everything that's put on them,
is going to be put on both drives.
This way, if a single drive fails,
the system can quickly access all the files it needs,
because there's still a fully redundant copy
on that second drive as part of that RAID.
Parity is the third technique we use in RAIDs.
Now, parity computations are going to be used
in RAID drive arrays for fault tolerance
by calculating the data in two drives
and storing the results on a different drive.
For example, in a RAID 5,
there's going to be a minimum of three disk drives
and these are going to be used
so that data can be striped across the drives
by splitting each file into two pieces,
putting one piece on one drive
and the other piece on the second drive,
and then the calculated parity gets put on a third drive.
All right, now that we've defined the term striping,
mirroring and parity, let's take a quick look
at the different types of RAIDs
that you can configure on your Linux system.
First, we have a RAID 0.
RAID 0 relies on striping.
With a RAID 0, we have two different disks
that are working together
and each of them holds half of the data.
So as you can see here,
part of the file A1 and part of the file A2
are each put on each of the drives.
So let's assume that I have a RAID 0 with my Linux system.
If I've installed it and configured it,
what does the operating system actually see?
Do they see disk zero and disk one?
Well, no.
Instead it's just going to see one logical drive
in the RAID, because both disks are working together
to create a single larger logical volume.
Now, if I'm going to copy this video
over to the new striped RAID 0 logical drive,
what do you think is going to happen?
Well, it's going to put the first piece on disk zero
and the second piece on disk one
and the third piece on disk zero,
and the fourth piece on disk one.
And it keeps doing this,
where all of the even parts and odd parts
are being split up across the two different disks.
This is what's known as striping,
because as I'm putting a piece on each of those two disks,
if you look at it, it looks like a candy stripe
or a peppermint stick,
because it's swirling around those two RAIDs,
with the evens on one side and the odds on the other.
Now, RAID 0 is a great thing
if you want to have faster speeds,
because it stripes the data across two physical drives,
but this provides no redundancy at all for your data.
If one of these two disks fail,
then you're going to lose half of the file.
And if you lose half of the file, well, guess what?
The computer isn't going to be able to read the file at all,
so really, it's like you lost the whole file.
Now, the good thing about using a stripe disk
or a RAID 0 array,
is the fact that you get this speed boost, right?
And so as you're doing things like high speed gaming
or high speed video editing,
RAID 0s can be a great setup for this,
because you can access two disks a lot faster
than you can access one disk,
because each of them is going to give you
half of the information at the same time.
Now, the other good thing about using a RAID 0,
is the fact that there is no loss of space on those disks.
If I take 2 one terabyte hard disks
and I combine them logically into a RAID 0,
this is going to appear as a single logical volume
containing two terabytes of size for us to use.
The next type of RAID we need to discuss,
is known as a RAID 1.
Whenever you hear RAID 1,
I want you to think about two words, mirror and redundancy.
When you have a RAID 1, you have a mirrored disk array.
This means that every single thing that's put on disk zero,
is also put on disk one.
So if I have a file called A
and I break it up into four pieces,
both disk zero and disk one are going to have
all of those four parts of file A.
And as you can see here, one, two, three
and four is on disk zero.
One, two, three and four is also on disk one.
So if I delete everything on disk one, guess what?
I can still access it on disk zero.
This is what we're talking about
by having a mirror disk array.
The fact is, both of these disks have an identical copy
and this gives us a great amount of redundancy.
In fact, it's fully redundant,
because we have an entire disk
that can go away and I can still operate with no problems,
because I can rely on the other disk.
There is one big downside to using a RAID 1 though,
and that's the loss of space on one of those disks,
because if I have to take two one terabyte hard disks,
I no longer get two terabytes of storage space to use,
instead I'm only getting one terabyte of space.
Why?
Because I have a mirrored copy
of every single piece of data
and that means I have to make two full copies
and it takes up twice the amount of space
to store those files.
So if you have half of the total space
being used simply for redundancy,
this means you're paying a lot more for storage costs,
because you're going to be using something like a RAID 1 here,
instead of a RAID 0
that allows you to use all of the disk space.
So if you want to get the best of both worlds,
you really might want to consider
using something like a RAID 5.
A RAID 5 is going to give you the redundancy through parity.
So when you hear RAID 5,
I want you to remember the keywords.
These are redundancy and parity.
Now, a RAID 5 is a bit different
in how it provides that redundancy though.
Instead of using a full mirror like a RAID 1 did
and taking up an entire disk worth of space,
a RAID 5 is going to use striping and a computed parity
across three or more disks.
Now, it's important to remember and to take note
that a RAID 5 can use three or more disks,
but you must have at least three disks
in order to configure a RAID 5.
Now, a RAID 5 with three disks is going to work like this.
First, we have three disks, disk zero,
disk one, and disk two.
If I want to store four different files on those drives,
I can do this by putting pieces of file A, B, C and D
on all of those drives.
Notice that B, C and D have this thing called a Bp,
a Cp and a Dp.
This is the computed parity.
So when I take file B and I put half on disk zero
and half on disk one, I then can do a calculation
and put the results of that calculation onto disk two.
Now, if I lose one of those disks,
I can use the part that I have and the parity
in order to calculate that file,
or if I have the two parts and no parity,
I can then simply recalculate the parity
and recreate that file as well.
Now, I know that sounds a little bit complicated,
but I want you to think about it this way.
Let's say I gave you two numbers and I give you an answer.
For example, two plus three equals five.
Now, if I take away any of the three numbers
and I give you the other two,
you could still figure out what the other one was, right?
Of course you could, because if I gave you two and three,
you could add them together and get five.
If I gave you two plus something equals five,
you could say, well, five minus two is three.
So the answer must be three.
Or if I gave you something plus three is five.
You could say that five minus three is two.
And therefore, you calculated
that the missing number was two.
That's what we're doing inside of a RAID 5.
We're going to be able to recompute
the missing parts of the file
by using the known parts and the stored parity
and this is how redundancy through parity
is going to be achieved.
The great thing about this,
is that it takes up less disk space
to store a computed parity
than it does to store the entire mirrored file.
So if I'm using three drives in a RAID 5,
I'm only using one third of the space for that parity.
If I use four drives,
I'm losing only one fourth of the space for that parity.
And if I'm using five drives, I'm only going to lose one fifth
of the total space for that parity.
This is a lot less than a full mirrored file.
So it's a lot more efficient to create a RAID 5
in terms of space, than using a RAID 1
using a mirrored array.
For this reason, RAID 5 is the most popular type of RAID
in use today and you're going to find it heavily used
in most server environments.
Next, let's talk about RAID 6.
What exactly is a RAID 6?
Well, six is better than five by one
and any time you hear RAID 6,
I want you to think of five plus one.
So everything that was true about a RAID 5
is still true about a RAID 6,
except we are going to have double parity in a RAID 6,
instead of single parity.
Because of this double parity,
we also need to add an extra drive
and have a minimum of at least four drives
to create a RAID 6.
Now, why would you want to use a RAID 6 instead of a RAID 5?
Well, in a RAID 5, I can only lose one drive
and keep operating that array.
In RAID 6, I can actually lose two disks and keep operating.
So if I have maybe five or 10 disk drives,
using two of them for parity
gives me a much better redundancy value
than if I just use RAID 5
with a single redundancy and single parity.
This is the main difference between a RAID 5 and a RAID 6.
Now, the last type of RAID we're going to discuss,
is known as a RAID 10.
A RAID 10 is really a RAID of RAIDs.
What I mean by this, is that I actually have two RAID 1s
and they're being placed inside a RAID 0 configuration.
If we go back to a RAID 0,
we remember this is all about speed and striping
and so as the file comes in, I'm going to stripe it
and put half to the left side and half to the right side.
But because each of that left and right side
is actually a mirrored RAID,
I have all of the odd stuff on the left side
and all of the even stuff on the right side,
and now I have two full copies.
Because of this setup, you need a minimum of four disks
to be able to create a RAID 10.
This means I'm going to lose 50% of my total disk space
to create this striped array of mirrored to arrays,
but I do get the benefit
of having two fully redundant mirrors inside my RAID 10.
RAID 10 is also pretty fast,
because you're using that RAID 0 striping technique
across two mirrored arrays.
So with RAID 10, you get the benefits of RAID 1
and the benefits of RAID 0,
but you're having to use four disks to do this.
All right, now that we've reviewed the types of RAIDs,
you might be wondering,
how do I get information about my RAID
from my Linux system and its Kernel?
Well, to do that, you're going to use /proc/mdstat.
This is a file and it's a text-based file
that contains a snapshot of the Kernel's RAID or MD state.
To read this file on your system,
simply type cat /proc/mdstat
in the command line interface and then hit ENTER.
This will display the information to your screen
about any known RAID configurations on your system.
Now, for example, on one of my systems,
I get the following returned when I enter this command.
Under personalities, the file's going to tell me
which RAIDs could be supported by my Kernel.
In this case, it says I can support a RAID 6,
a RAID 5 or a RAID 4.
Next, we see that md0,
which is the name of my array, is active.
It's configured as a RAID 5
and it's using three disk drives to make up that RAID 5.
Sda1, sdd1 and sdb1.
The numbers after each of these tells me which disk
is physically located in the array
and this is actually disk zero, which is sda1,
disk one which is sdd1, and disk two which is sdd1.
Next, we see the number of blocks supported,
the size of the chunks and the status of the algorithm
and then we can see the status of the disks in that RAID.
And this is shown as UUU_.
This means up, up, up and down.
Notice, there's that underscore
and when there's an underscore shown,
this means there's a disk that is down.
In this case, we can see the fourth disk,
which is probably going to be sdc1.
It's going to be down, because there's an underscore shown.
This explains why the Kernel supports RAID 6,
which requires four disks, but we're only using a RAID 5,
which uses three disks, because we have three disks
that are up and configured.
Next, we're going to discuss one of the major applications
of the device mapper,
which is called the logical volume manager, or LVM.
The LVM will map whole physical devices and partitions
into one or more virtual containers, known as volume groups.
Within these volume groups, are one or more logical volumes.
Ultimately, these logical volumes become the storage devices
that the system, the user and the applications
are going to interact with and work with.
With the logical volume manager, you can dynamically create,
delete and resize volumes
without having to reboot the system.
You can also map multiple logical volumes
across multiple physical devices
and you can create virtual snapshots of each logical volume
so that you can quickly and easily revert a volume
to a specific state.
The /dev/mapper directory contains all the logical volumes
on a given system that are being managed
by the logical volume manager.
Devices in this directory
are typically going to be called /dev/mapper,
the volume group name, -the logical volume name.
the logical volume manager
divides its volume management tools
into three categories based on three different components,
including physical volume tools, volume group tools
and logical volume tools.
Some logical volume managers' physical volume tools
will include things like pvscan,
which is used to scan for all the physical devices
that are being mounted and used as physical volumes.
You also have pvcreate, which is used to initialize
a driver partition to use as a physical volume.
We also have pvdisplay, which is used to list
the attributes of physical volumes.
We have pvchange, which is used to change attributes
of a physical volume.
We have pvs, which is used to display information
about physical volumes.
We have pvck, which is used to check
the metadata of physical volumes.
And we have pvremove,
which is used to remove physical volumes.
Now, we also have some volume group tools.
This includes vgscan,
which is going to scan all physical devices for volume groups.
We have vgcreate to create our volume groups.
We have vgdisplay to list out attributes
of our volume groups.
We have vgchange to change the attributes
of our volume groups.
We have vgs to display information about our volume groups.
We have vgck to check the metadata of those volume groups.
We have vgrename to rename a volume group.
We have vgreduce to remove physical volumes
from a volume group to reduce its size.
We have vgextend, which allows us to add physical volumes
to the volume groups.
We have vgmerge, where we can merge
two volume groups together.
We have vgsplit, which can be used to split
a volume group into two groups
and we have vgremove,
where we can remove those volume groups.
When we get down to our logical volume tools,
we have things like lvscan,
which scans all the physical devices for logical volumes.
We have lvcreate to create logical volumes
in a volume group.
We have lvdisplay to list the attributes of logical volumes.
Lvchange is going to be used to change attributes
of the volumes.
Lvs will display information about logical volumes
and lvrename is used to rename the logical volumes.
If you use lvreduce,
that'll reduce the size of your logical volumes.
Lvextend will extend the size of your logical volumes.
Lvresize will resize the logical volumes
and finally, lvremove will remove the logical volumes.
Now, you may be wondering,
do I have to memorize that long list of tools
that Jason just read out?
And the answer is no, because you always have the help files
and the man files at your system
when you're doing this in the real world.
For the exam, they're not going to ask you
each and every one of these commands,
but it is something I wanted to introduce you to
so you can get comfortable with the idea
of how to work with logical volumes
inside the logical volume manager.