1 00:00:00,020 --> 00:00:02,790 In this lesson we're going to discuss hashing. 2 00:00:02,790 --> 00:00:05,340 Now in network security, integrity is usually provided 3 00:00:05,340 --> 00:00:07,470 through the use of a hashing function. 4 00:00:07,470 --> 00:00:09,660 Hashing is a one-way cryptographic function 5 00:00:09,660 --> 00:00:11,130 that takes an input and produces 6 00:00:11,130 --> 00:00:13,800 a unique message digest as its output. 7 00:00:13,800 --> 00:00:15,420 Because this function is one way 8 00:00:15,420 --> 00:00:17,610 there is no way to determine the original message 9 00:00:17,610 --> 00:00:19,980 based on the message hash or hash digest 10 00:00:19,980 --> 00:00:22,410 that's being outputted by this algorithm. 11 00:00:22,410 --> 00:00:24,420 The resulting message digest will then act 12 00:00:24,420 --> 00:00:27,120 like a digital fingerprint for the original file. 13 00:00:27,120 --> 00:00:29,100 Another unique thing about a hash digest 14 00:00:29,100 --> 00:00:30,960 is that they're always going to be the same length 15 00:00:30,960 --> 00:00:33,090 regardless of how long your input is. 16 00:00:33,090 --> 00:00:34,920 Whether I input a file containing one word 17 00:00:34,920 --> 00:00:36,870 or a file containing 1 million words, 18 00:00:36,870 --> 00:00:38,880 the output will always be the same length 19 00:00:38,880 --> 00:00:41,370 based on the hashing algorithm you chose. 20 00:00:41,370 --> 00:00:43,230 Now, when it comes to hashing, there are a few 21 00:00:43,230 --> 00:00:45,090 main algorithms that you need to be aware of, 22 00:00:45,090 --> 00:00:50,090 including MD5, the SHA family, RIPEMD, and HMAC. 23 00:00:50,130 --> 00:00:53,310 First we have MD5, which is by far the most popular 24 00:00:53,310 --> 00:00:56,880 and widely used hashing algorithm over the past few decades. 25 00:00:56,880 --> 00:00:59,760 Now, the MD5 algorithm is going to create 128-bit 26 00:00:59,760 --> 00:01:02,550 hash value that is unique to the input file. 27 00:01:02,550 --> 00:01:04,920 Unfortunately, because the hash value being output 28 00:01:04,920 --> 00:01:07,500 is only 128 bits long, it can only create 29 00:01:07,500 --> 00:01:10,050 a limited number of unique values, and this can lead 30 00:01:10,050 --> 00:01:13,350 to two files having the exact same resulting hash digest, 31 00:01:13,350 --> 00:01:15,180 which we call a collision. 32 00:01:15,180 --> 00:01:17,130 Now, due to the limited number of unique hash values 33 00:01:17,130 --> 00:01:19,830 associated with MD5, a newer family of algorithms 34 00:01:19,830 --> 00:01:24,000 was created called the Secure Hash Algorithm family, or SHA. 35 00:01:24,000 --> 00:01:29,000 Now the SHA family, or S-H-A includes SHA-1 SHA-2 and SHA-3 36 00:01:29,040 --> 00:01:31,530 which are all families of hashing algorithms. 37 00:01:31,530 --> 00:01:35,280 Now, SHA-1 for example, creates 160-bit hash digest 38 00:01:35,280 --> 00:01:37,080 which significantly reduces the number of collisions 39 00:01:37,080 --> 00:01:42,080 that can occur over MD5 since MD5 only at 128 bits instead. 40 00:01:42,240 --> 00:01:44,220 SHA-2 is a family of hash functions 41 00:01:44,220 --> 00:01:46,290 that contains longer hash digests. 42 00:01:46,290 --> 00:01:49,160 This includes SHA-224, SHA-256, 43 00:01:49,160 --> 00:01:52,440 SHA-348, and SHA-512 hashing functions. 44 00:01:52,440 --> 00:01:55,410 And each of those has a digest between 224 bits 45 00:01:55,410 --> 00:01:58,620 up to 512 bits, as their name suggests. 46 00:01:58,620 --> 00:02:00,870 Now, each version of SHA is going to perform a different number 47 00:02:00,870 --> 00:02:03,180 of rounds of mathematical computations to create 48 00:02:03,180 --> 00:02:06,570 their hash digest, anywhere from 64 to 80 rounds, 49 00:02:06,570 --> 00:02:08,220 but you really don't need to know the specific number 50 00:02:08,220 --> 00:02:10,500 of rounds for each version of SHA. 51 00:02:10,500 --> 00:02:13,080 Now, SHA-3 is a newer family of hash functions 52 00:02:13,080 --> 00:02:17,730 and its hash digest can go between 224 bits and 512 bits 53 00:02:17,730 --> 00:02:19,380 just like SHA-2 could. 54 00:02:19,380 --> 00:02:21,210 Now, the biggest thing here though with SHA-3 55 00:02:21,210 --> 00:02:23,310 is that it has a major increase in security 56 00:02:23,310 --> 00:02:25,980 because it uses 120 rounds of computations 57 00:02:25,980 --> 00:02:28,680 to create its message digest for each unique file 58 00:02:28,680 --> 00:02:32,130 instead of the 64 to 80 rounds used by SHA-2. 59 00:02:32,130 --> 00:02:34,110 Now, the third algorithm we have to talk about 60 00:02:34,110 --> 00:02:36,630 is the RIPEMD hashing algorithm. 61 00:02:36,630 --> 00:02:39,900 Now, RIPEMD stands for the Race Integrity Primitive 62 00:02:39,900 --> 00:02:43,470 Evaluation Message Digest, or R-I-P-E-M-D, 63 00:02:43,470 --> 00:02:48,240 and it comes in 160-bit, 256-bit and 320-bit versions. 64 00:02:48,240 --> 00:02:51,150 Now with the 160-bit version this is going to be the one 65 00:02:51,150 --> 00:02:53,040 that is most commonly used and you'll hear 66 00:02:53,040 --> 00:02:56,833 people talking about this as RIPEMD-160. 67 00:02:56,833 --> 00:03:00,660 Now RIPEMD-160 is considered to be an open source 68 00:03:00,660 --> 00:03:02,940 hashing algorithm that was created as a competitor 69 00:03:02,940 --> 00:03:05,970 to the SHA family of hashes, but it really hasn't gained 70 00:03:05,970 --> 00:03:09,060 the same level of popularity that the SHA family has. 71 00:03:09,060 --> 00:03:10,260 Now, the fourth and final one 72 00:03:10,260 --> 00:03:12,630 we need to talk about is HMAC. 73 00:03:12,630 --> 00:03:14,550 HMAC is the hash-based message 74 00:03:14,550 --> 00:03:17,430 authentication code, or H-M-A-C. 75 00:03:17,430 --> 00:03:19,260 Now, HMAC is used to check the integrity 76 00:03:19,260 --> 00:03:21,390 of a message and provides some level of assurance 77 00:03:21,390 --> 00:03:23,550 that its authenticity is real. 78 00:03:23,550 --> 00:03:25,920 Now, HMAC will actually be paired with other algorithms 79 00:03:25,920 --> 00:03:27,300 to do the work and so it'll be called 80 00:03:27,300 --> 00:03:31,464 something like HMAC-MD5 or HMAC-SHA1, 81 00:03:31,464 --> 00:03:35,160 or HMAC-SHA256, depending on the underlying hash 82 00:03:35,160 --> 00:03:37,020 that's going to be used with HMAC. 83 00:03:37,020 --> 00:03:39,210 Now, one of the most common applications of hashes 84 00:03:39,210 --> 00:03:40,950 is for us to create a digital signature 85 00:03:40,950 --> 00:03:42,990 to ensure the integrity of a file or message 86 00:03:42,990 --> 00:03:45,270 as it's being transferred over our network. 87 00:03:45,270 --> 00:03:47,670 A digital signature is created by hashing a file 88 00:03:47,670 --> 00:03:49,470 and then taking the resulting hash digest 89 00:03:49,470 --> 00:03:51,690 and encrypting it using a private key. 90 00:03:51,690 --> 00:03:53,850 So, if I want to send an email that's going to be a couple 91 00:03:53,850 --> 00:03:56,010 of pages in length, and I want to digitally sign it 92 00:03:56,010 --> 00:03:57,570 to make sure you know that nothing was changed 93 00:03:57,570 --> 00:04:00,060 inside of that email, I can run that email message 94 00:04:00,060 --> 00:04:02,790 through a hashing algorithm like SHA-1, and then take 95 00:04:02,790 --> 00:04:06,660 the resulting 160-bit hash digest from the SHA-1 algorithm, 96 00:04:06,660 --> 00:04:09,570 and then I can encrypt that using my private key. 97 00:04:09,570 --> 00:04:11,100 When I send the email over to you 98 00:04:11,100 --> 00:04:13,170 I'm going to attach the resulting encrypted hash, 99 00:04:13,170 --> 00:04:16,170 which we call the digital signature, along with the message 100 00:04:16,170 --> 00:04:18,510 and then this is going to prove the integrity of the message 101 00:04:18,510 --> 00:04:20,850 and it creates non-repudiation. 102 00:04:20,850 --> 00:04:23,430 When your system receives my email, it's going to decrypt 103 00:04:23,430 --> 00:04:25,650 the digital signature using my public key 104 00:04:25,650 --> 00:04:26,483 which is going to provide you 105 00:04:26,483 --> 00:04:29,250 with that original 160-bit hash digest. 106 00:04:29,250 --> 00:04:31,890 Now, your system takes that multiple page email 107 00:04:31,890 --> 00:04:35,220 and runs it through its own version of the SHA-1 algorithm. 108 00:04:35,220 --> 00:04:38,190 Then it compares your message digest that you calculated 109 00:04:38,190 --> 00:04:40,980 with the one I sent as part of my digital signature. 110 00:04:40,980 --> 00:04:43,260 If those two things match, then you can be assured 111 00:04:43,260 --> 00:04:46,110 that the email was not modified in transit between my system 112 00:04:46,110 --> 00:04:49,020 and yours, and this provides you with the integrity check. 113 00:04:49,020 --> 00:04:51,510 Now, since I also encrypted that SHA-1 digest 114 00:04:51,510 --> 00:04:54,390 with my private key, and only I have my private key, 115 00:04:54,390 --> 00:04:55,950 this also assures you that the person 116 00:04:55,950 --> 00:04:57,930 who sent the message is the only person 117 00:04:57,930 --> 00:05:00,240 who could have sent that message, and this provides you 118 00:05:00,240 --> 00:05:02,490 with non-repudiation for that email. 119 00:05:02,490 --> 00:05:04,590 This non-repudiation means I can't claim 120 00:05:04,590 --> 00:05:06,000 that I didn't send the email to you, 121 00:05:06,000 --> 00:05:07,650 because I'm the only one who could have, 122 00:05:07,650 --> 00:05:09,150 because I'm the only person in the world 123 00:05:09,150 --> 00:05:11,850 who has a copy of my private key. 124 00:05:11,850 --> 00:05:13,890 Now, to practically use digital signatures, 125 00:05:13,890 --> 00:05:16,230 you need to use an algorithm for that, and we're going to 126 00:05:16,230 --> 00:05:18,957 either use the digital security algorithm known as DSA, 127 00:05:18,957 --> 00:05:21,990 the Rivest-Shamir-Adleman cipher, or RSA, 128 00:05:21,990 --> 00:05:23,880 or the elliptic curve cryptography version 129 00:05:23,880 --> 00:05:25,560 of either DSA or SHA. 130 00:05:25,560 --> 00:05:27,240 Now, the federal government has decided to use 131 00:05:27,240 --> 00:05:29,910 the Digital Security Standard, which they call DSS, 132 00:05:29,910 --> 00:05:32,880 and this relies upon 160-bit message digest 133 00:05:32,880 --> 00:05:35,100 created by the Digital Security algorithm. 134 00:05:35,100 --> 00:05:37,830 Now, most commercial entities though, will rely on RSA 135 00:05:37,830 --> 00:05:39,810 instead because it tends to be faster 136 00:05:39,810 --> 00:05:41,190 and can be used for digital signatures, 137 00:05:41,190 --> 00:05:43,500 encryption, and key distribution. 138 00:05:43,500 --> 00:05:45,120 Digital signatures have also expanded 139 00:05:45,120 --> 00:05:46,800 beyond just emails, too. 140 00:05:46,800 --> 00:05:49,440 Code signing of our files relies upon the digital signature 141 00:05:49,440 --> 00:05:52,290 for a program or file being added to that file. 142 00:05:52,290 --> 00:05:54,900 For example, if I create a new mobile application 143 00:05:54,900 --> 00:05:57,330 and I want to put it into an app store like Google Play, 144 00:05:57,330 --> 00:05:59,490 or the Apple App Store the installer file 145 00:05:59,490 --> 00:06:00,840 will have to be digitally signed 146 00:06:00,840 --> 00:06:03,330 and this is what we refer to as code signing. 147 00:06:03,330 --> 00:06:05,820 Now, every developer must register with Apple or Google 148 00:06:05,820 --> 00:06:07,980 and they'll receive their own private key. 149 00:06:07,980 --> 00:06:10,140 Just as the email example I provided earlier 150 00:06:10,140 --> 00:06:12,570 the application file will be hashed and then that hash 151 00:06:12,570 --> 00:06:15,240 will be encrypted using the developer's private key. 152 00:06:15,240 --> 00:06:17,040 This is what we call code signing, 153 00:06:17,040 --> 00:06:19,080 and it ensures the installer hasn't been modified 154 00:06:19,080 --> 00:06:22,170 or corrupted since the developer actually published it. 155 00:06:22,170 --> 00:06:24,000 Now, I know we covered a lot here already 156 00:06:24,000 --> 00:06:25,650 but before we close out this lesson 157 00:06:25,650 --> 00:06:27,330 I want to provide you with a quick demonstration 158 00:06:27,330 --> 00:06:30,030 of how drastically the hash digest will change 159 00:06:30,030 --> 00:06:33,180 if the source text or file is changed even a little bit. 160 00:06:33,180 --> 00:06:35,430 To do this, I'm going to use an online MD5 161 00:06:35,430 --> 00:06:38,070 hash digest generator to create two slightly different 162 00:06:38,070 --> 00:06:40,080 inputs that will result in a very large 163 00:06:40,080 --> 00:06:41,400 difference in their outputs. 164 00:06:41,400 --> 00:06:44,550 On the left, I have an online MD5 hash generator 165 00:06:44,550 --> 00:06:48,780 and on the right I have a text file of the US Constitution. 166 00:06:48,780 --> 00:06:52,410 Now, what I'm going to do is I'm going to copy this entire file 167 00:06:52,410 --> 00:06:54,360 of the text file of the US Constitution 168 00:06:54,360 --> 00:06:56,670 and place it into the box. 169 00:06:56,670 --> 00:06:59,370 And immediately what is going to end up happening, 170 00:06:59,370 --> 00:07:02,850 if I go up to here to the top, I have the entire text here. 171 00:07:02,850 --> 00:07:06,090 And we have this hash string down here at the bottom. 172 00:07:06,090 --> 00:07:07,410 Now, the hash string at the bottom 173 00:07:07,410 --> 00:07:09,840 is unique to that Constitution 174 00:07:09,840 --> 00:07:11,910 and the word and the order that those are in. 175 00:07:11,910 --> 00:07:15,180 If I change even one letter in the Constitution 176 00:07:15,180 --> 00:07:17,400 that hash is going to be drastically changed, 177 00:07:17,400 --> 00:07:19,260 and I'm going to demonstrate that to you right now. 178 00:07:19,260 --> 00:07:22,230 So if I go here, there's the word defence with a C-E, 179 00:07:22,230 --> 00:07:23,850 which is the British spelling. 180 00:07:23,850 --> 00:07:26,430 But as Americans we don't spell it that way anymore. 181 00:07:26,430 --> 00:07:29,790 We spell it as D-E-F-E-N-S-E. 182 00:07:29,790 --> 00:07:31,410 So if I change the C to an S, 183 00:07:31,410 --> 00:07:33,750 watch that hash value as it changes. 184 00:07:33,750 --> 00:07:37,860 I'm going to do that in 3, 2, 1, change. 185 00:07:37,860 --> 00:07:41,250 Notice a vast difference in that hash. 186 00:07:41,250 --> 00:07:44,220 That's why one letter makes all the difference 187 00:07:44,220 --> 00:07:45,540 when you're dealing with hashes. 188 00:07:45,540 --> 00:07:47,700 Because of the way that these are computed, they come 189 00:07:47,700 --> 00:07:50,850 in one way and they come out a different way, 190 00:07:50,850 --> 00:07:52,710 and they come out the same way every time 191 00:07:52,710 --> 00:07:54,060 based on what's input. 192 00:07:54,060 --> 00:07:56,100 So let me show you another example of this. 193 00:07:56,100 --> 00:07:58,350 Let's go ahead and say I have the line 194 00:07:58,350 --> 00:08:01,110 1, 2, 3, 4, 5, 6, 7, 8, 9, 0 195 00:08:01,110 --> 00:08:04,380 and I'm going to treat each line as a separate hash now. 196 00:08:04,380 --> 00:08:06,120 if I went and said, let's do it 197 00:08:06,120 --> 00:08:09,750 as 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 198 00:08:09,750 --> 00:08:12,300 they're vastly different because I shifted the position. 199 00:08:12,300 --> 00:08:13,800 You'd probably expect that. 200 00:08:13,800 --> 00:08:18,480 But what if I did 1, 2, 3, 4, 5, 6, 7, 8, 9, 0? 201 00:08:18,480 --> 00:08:21,570 They're both the same now, and I added a dot. 202 00:08:21,570 --> 00:08:24,360 Well, that's going to change the bottom one drastically. 203 00:08:24,360 --> 00:08:26,130 If I go back, it's the same. 204 00:08:26,130 --> 00:08:28,680 If I add a space, it changes it again. 205 00:08:28,680 --> 00:08:31,020 Every time you add any differences, 206 00:08:31,020 --> 00:08:33,419 whether it's one bit or one character, 207 00:08:33,419 --> 00:08:36,539 it's going to drastically change that MD5 output. 208 00:08:36,539 --> 00:08:38,640 And that's why MD5s are considered 209 00:08:38,640 --> 00:08:40,200 a way to verify integrity. 210 00:08:40,200 --> 00:08:42,210 It verifies that the original input 211 00:08:42,210 --> 00:08:44,190 was never changed in transit. 212 00:08:44,190 --> 00:08:47,400 Because the hashes match, there could be no changes. 213 00:08:47,400 --> 00:08:50,880 Now, are there any cases where two different things 214 00:08:50,880 --> 00:08:52,560 will give you the same hash? 215 00:08:52,560 --> 00:08:54,720 Yes, this is called a collision. 216 00:08:54,720 --> 00:08:58,680 They do occur because an MD5 only uses 128 bits 217 00:08:58,680 --> 00:09:01,110 to represent that hash value, 218 00:09:01,110 --> 00:09:03,420 that is going to give us a limited number of choices. 219 00:09:03,420 --> 00:09:06,030 But there are unlimited number of inputs. 220 00:09:06,030 --> 00:09:08,520 So when you have two that give you the same hash value, 221 00:09:08,520 --> 00:09:11,430 that is called a collision, and that is a bad thing. 222 00:09:11,430 --> 00:09:14,370 So, because MD5 only has 128 bits, 223 00:09:14,370 --> 00:09:16,500 it does have more collisions than something 224 00:09:16,500 --> 00:09:21,240 like SHA-1, or SHA-256, or SHA-512, 225 00:09:21,240 --> 00:09:24,390 because as you extend that space of what that unique 226 00:09:24,390 --> 00:09:27,210 hash value can be you have less collisions. 227 00:09:27,210 --> 00:09:30,900 And that's why most people have moved to SHA-1, SHA-256, 228 00:09:30,900 --> 00:09:34,380 or SHA-512 as we've moved forward into the future. 229 00:09:34,380 --> 00:09:37,620 So remember, hashing is a one-way cryptographic function 230 00:09:37,620 --> 00:09:39,030 that takes an input and produces 231 00:09:39,030 --> 00:09:41,430 a unique message digest as its output. 232 00:09:41,430 --> 00:09:43,170 Because this function is one way, 233 00:09:43,170 --> 00:09:45,330 there is no way to determine the original message 234 00:09:45,330 --> 00:09:47,070 based on the message hash digest. 235 00:09:47,070 --> 00:09:49,230 Because this function is considered to be a one-way 236 00:09:49,230 --> 00:09:51,990 algorithm, there is no way to undo it and then determine 237 00:09:51,990 --> 00:09:54,510 the original message based solely on the message hash 238 00:09:54,510 --> 00:09:56,070 or hash digest that you received 239 00:09:56,070 --> 00:09:58,650 as an output from that algorithm. 240 00:09:58,650 --> 00:10:01,410 This makes the hash digest or resulting message digest 241 00:10:01,410 --> 00:10:02,820 act like a digital fingerprint 242 00:10:02,820 --> 00:10:05,340 for our original files that we're sending around. 243 00:10:05,340 --> 00:10:07,350 There are also a few commonly used hashing algorithms 244 00:10:07,350 --> 00:10:09,630 that you need to be aware of, including MD5, 245 00:10:09,630 --> 00:10:13,200 the SHA family, RIPEMD, and HMAC. 246 00:10:13,200 --> 00:10:15,420 The most secure of these is going to be the SHA family 247 00:10:15,420 --> 00:10:19,770 of algorithms, and specifically SHA-256 or SHA-3. 248 00:10:19,770 --> 00:10:21,750 Finally, remember that digital signatures 249 00:10:21,750 --> 00:10:23,850 are basically just an encrypted hash digest 250 00:10:23,850 --> 00:10:25,920 that's going to be attached to a given file. 251 00:10:25,920 --> 00:10:28,470 This digital signature is created by hashing a file 252 00:10:28,470 --> 00:10:30,360 and then taking the resulting hash digest 253 00:10:30,360 --> 00:10:32,760 and encrypting it with the signer's private key.