In this lesson we're going to discuss hashing.
Now in network security, integrity is usually provided
through the use of a hashing function.
Hashing is a one-way cryptographic function
that takes an input and produces
a unique message digest as its output.
Because this function is one way
there is no way to determine the original message
based on the message hash or hash digest
that's being outputted by this algorithm.
The resulting message digest will then act
like a digital fingerprint for the original file.
Another unique thing about a hash digest
is that they're always going to be the same length
regardless of how long your input is.
Whether I input a file containing one word
or a file containing 1 million words,
the output will always be the same length
based on the hashing algorithm you chose.
Now, when it comes to hashing, there are a few
main algorithms that you need to be aware of,
including MD5, the SHA family, RIPEMD, and HMAC.
First we have MD5, which is by far the most popular
and widely used hashing algorithm over the past few decades.
Now, the MD5 algorithm is going to create 128-bit
hash value that is unique to the input file.
Unfortunately, because the hash value being output
is only 128 bits long, it can only create
a limited number of unique values, and this can lead
to two files having the exact same resulting hash digest,
which we call a collision.
Now, due to the limited number of unique hash values
associated with MD5, a newer family of algorithms
was created called the Secure Hash Algorithm family, or SHA.
Now the SHA family, or S-H-A includes SHA-1 SHA-2 and SHA-3
which are all families of hashing algorithms.
Now, SHA-1 for example, creates 160-bit hash digest
which significantly reduces the number of collisions
that can occur over MD5 since MD5 only at 128 bits instead.
SHA-2 is a family of hash functions
that contains longer hash digests.
This includes SHA-224, SHA-256,
SHA-348, and SHA-512 hashing functions.
And each of those has a digest between 224 bits
up to 512 bits, as their name suggests.
Now, each version of SHA is going to perform a different number
of rounds of mathematical computations to create
their hash digest, anywhere from 64 to 80 rounds,
but you really don't need to know the specific number
of rounds for each version of SHA.
Now, SHA-3 is a newer family of hash functions
and its hash digest can go between 224 bits and 512 bits
just like SHA-2 could.
Now, the biggest thing here though with SHA-3
is that it has a major increase in security
because it uses 120 rounds of computations
to create its message digest for each unique file
instead of the 64 to 80 rounds used by SHA-2.
Now, the third algorithm we have to talk about
is the RIPEMD hashing algorithm.
Now, RIPEMD stands for the Race Integrity Primitive
Evaluation Message Digest, or R-I-P-E-M-D,
and it comes in 160-bit, 256-bit and 320-bit versions.
Now with the 160-bit version this is going to be the one
that is most commonly used and you'll hear
people talking about this as RIPEMD-160.
Now RIPEMD-160 is considered to be an open source
hashing algorithm that was created as a competitor
to the SHA family of hashes, but it really hasn't gained
the same level of popularity that the SHA family has.
Now, the fourth and final one
we need to talk about is HMAC.
HMAC is the hash-based message
authentication code, or H-M-A-C.
Now, HMAC is used to check the integrity
of a message and provides some level of assurance
that its authenticity is real.
Now, HMAC will actually be paired with other algorithms
to do the work and so it'll be called
something like HMAC-MD5 or HMAC-SHA1,
or HMAC-SHA256, depending on the underlying hash
that's going to be used with HMAC.
Now, one of the most common applications of hashes
is for us to create a digital signature
to ensure the integrity of a file or message
as it's being transferred over our network.
A digital signature is created by hashing a file
and then taking the resulting hash digest
and encrypting it using a private key.
So, if I want to send an email that's going to be a couple
of pages in length, and I want to digitally sign it
to make sure you know that nothing was changed
inside of that email, I can run that email message
through a hashing algorithm like SHA-1, and then take
the resulting 160-bit hash digest from the SHA-1 algorithm,
and then I can encrypt that using my private key.
When I send the email over to you
I'm going to attach the resulting encrypted hash,
which we call the digital signature, along with the message
and then this is going to prove the integrity of the message
and it creates non-repudiation.
When your system receives my email, it's going to decrypt
the digital signature using my public key
which is going to provide you
with that original 160-bit hash digest.
Now, your system takes that multiple page email
and runs it through its own version of the SHA-1 algorithm.
Then it compares your message digest that you calculated
with the one I sent as part of my digital signature.
If those two things match, then you can be assured
that the email was not modified in transit between my system
and yours, and this provides you with the integrity check.
Now, since I also encrypted that SHA-1 digest
with my private key, and only I have my private key,
this also assures you that the person
who sent the message is the only person
who could have sent that message, and this provides you
with non-repudiation for that email.
This non-repudiation means I can't claim
that I didn't send the email to you,
because I'm the only one who could have,
because I'm the only person in the world
who has a copy of my private key.
Now, to practically use digital signatures,
you need to use an algorithm for that, and we're going to
either use the digital security algorithm known as DSA,
the Rivest-Shamir-Adleman cipher, or RSA,
or the elliptic curve cryptography version
of either DSA or SHA.
Now, the federal government has decided to use
the Digital Security Standard, which they call DSS,
and this relies upon 160-bit message digest
created by the Digital Security algorithm.
Now, most commercial entities though, will rely on RSA
instead because it tends to be faster
and can be used for digital signatures,
encryption, and key distribution.
Digital signatures have also expanded
beyond just emails, too.
Code signing of our files relies upon the digital signature
for a program or file being added to that file.
For example, if I create a new mobile application
and I want to put it into an app store like Google Play,
or the Apple App Store the installer file
will have to be digitally signed
and this is what we refer to as code signing.
Now, every developer must register with Apple or Google
and they'll receive their own private key.
Just as the email example I provided earlier
the application file will be hashed and then that hash
will be encrypted using the developer's private key.
This is what we call code signing,
and it ensures the installer hasn't been modified
or corrupted since the developer actually published it.
Now, I know we covered a lot here already
but before we close out this lesson
I want to provide you with a quick demonstration
of how drastically the hash digest will change
if the source text or file is changed even a little bit.
To do this, I'm going to use an online MD5
hash digest generator to create two slightly different
inputs that will result in a very large
difference in their outputs.
On the left, I have an online MD5 hash generator
and on the right I have a text file of the US Constitution.
Now, what I'm going to do is I'm going to copy this entire file
of the text file of the US Constitution
and place it into the box.
And immediately what is going to end up happening,
if I go up to here to the top, I have the entire text here.
And we have this hash string down here at the bottom.
Now, the hash string at the bottom
is unique to that Constitution
and the word and the order that those are in.
If I change even one letter in the Constitution
that hash is going to be drastically changed,
and I'm going to demonstrate that to you right now.
So if I go here, there's the word defence with a C-E,
which is the British spelling.
But as Americans we don't spell it that way anymore.
We spell it as D-E-F-E-N-S-E.
So if I change the C to an S,
watch that hash value as it changes.
I'm going to do that in 3, 2, 1, change.
Notice a vast difference in that hash.
That's why one letter makes all the difference
when you're dealing with hashes.
Because of the way that these are computed, they come
in one way and they come out a different way,
and they come out the same way every time
based on what's input.
So let me show you another example of this.
Let's go ahead and say I have the line
1, 2, 3, 4, 5, 6, 7, 8, 9, 0
and I'm going to treat each line as a separate hash now.
if I went and said, let's do it
as 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
they're vastly different because I shifted the position.
You'd probably expect that.
But what if I did 1, 2, 3, 4, 5, 6, 7, 8, 9, 0?
They're both the same now, and I added a dot.
Well, that's going to change the bottom one drastically.
If I go back, it's the same.
If I add a space, it changes it again.
Every time you add any differences,
whether it's one bit or one character,
it's going to drastically change that MD5 output.
And that's why MD5s are considered
a way to verify integrity.
It verifies that the original input
was never changed in transit.
Because the hashes match, there could be no changes.
Now, are there any cases where two different things
will give you the same hash?
Yes, this is called a collision.
They do occur because an MD5 only uses 128 bits
to represent that hash value,
that is going to give us a limited number of choices.
But there are unlimited number of inputs.
So when you have two that give you the same hash value,
that is called a collision, and that is a bad thing.
So, because MD5 only has 128 bits,
it does have more collisions than something
like SHA-1, or SHA-256, or SHA-512,
because as you extend that space of what that unique
hash value can be you have less collisions.
And that's why most people have moved to SHA-1, SHA-256,
or SHA-512 as we've moved forward into the future.
So remember, hashing is a one-way cryptographic function
that takes an input and produces
a unique message digest as its output.
Because this function is one way,
there is no way to determine the original message
based on the message hash digest.
Because this function is considered to be a one-way
algorithm, there is no way to undo it and then determine
the original message based solely on the message hash
or hash digest that you received
as an output from that algorithm.
This makes the hash digest or resulting message digest
act like a digital fingerprint
for our original files that we're sending around.
There are also a few commonly used hashing algorithms
that you need to be aware of, including MD5,
the SHA family, RIPEMD, and HMAC.
The most secure of these is going to be the SHA family
of algorithms, and specifically SHA-256 or SHA-3.
Finally, remember that digital signatures
are basically just an encrypted hash digest
that's going to be attached to a given file.
This digital signature is created by hashing a file
and then taking the resulting hash digest
and encrypting it with the signer's private key.