1 1 00:00:00,450 --> 00:00:03,030 Traffic spikes, in this lesson, 2 2 00:00:03,030 --> 00:00:04,920 we're going to talk about traffic spikes 3 3 00:00:04,920 --> 00:00:06,690 and some of the things that may cause them 4 4 00:00:06,690 --> 00:00:08,100 because this could be an indicator 5 5 00:00:08,100 --> 00:00:10,320 or compromise within your network. 6 6 00:00:10,320 --> 00:00:12,090 Now, when I talk about a traffic spike, 7 7 00:00:12,090 --> 00:00:15,360 this is really any sharp increase in connection request 8 8 00:00:15,360 --> 00:00:17,820 in comparison with a given baseline. 9 9 00:00:17,820 --> 00:00:19,800 Now, I know this is a very generic statement 10 10 00:00:19,800 --> 00:00:21,660 but it's important that we start out generically 11 11 00:00:21,660 --> 00:00:23,250 because by default, 12 12 00:00:23,250 --> 00:00:26,160 a traffic spike doesn't have to be a bad thing. 13 13 00:00:26,160 --> 00:00:28,170 For instance, let's take a look here. 14 14 00:00:28,170 --> 00:00:29,730 Let's say you're looking at your logs 15 15 00:00:29,730 --> 00:00:32,619 and you start seeing that traffic was around 100 16 16 00:00:32,619 --> 00:00:37,080 or 200 or 300 for the first 10 or 15 minutes after 1:00 AM. 17 17 00:00:37,080 --> 00:00:41,280 But then it jumps up at around 1:15 to 1:25 18 18 00:00:41,280 --> 00:00:42,990 and during that time you reach a peak 19 19 00:00:42,990 --> 00:00:45,390 of about 800 megabits per second. 20 20 00:00:45,390 --> 00:00:48,510 Then it drops down again to 200, 100, 21 21 00:00:48,510 --> 00:00:50,790 kind of stays there through the rest of the night. 22 22 00:00:50,790 --> 00:00:54,210 Now, is this malicious, are you under attack? 23 23 00:00:54,210 --> 00:00:57,180 Well, maybe, we don't have enough information. 24 24 00:00:57,180 --> 00:01:00,210 All we know right now is there is something anomalous. 25 25 00:01:00,210 --> 00:01:01,350 This could be normal. 26 26 00:01:01,350 --> 00:01:04,170 For instance, on my server, we run our backups 27 27 00:01:04,170 --> 00:01:06,900 that go from our servers to offsite cloud storage 28 28 00:01:06,900 --> 00:01:08,550 between 1:00 and 2:00 AM. 29 29 00:01:08,550 --> 00:01:10,560 And so if I saw this, I would then look 30 30 00:01:10,560 --> 00:01:12,540 and see where was the traffic going, 31 31 00:01:12,540 --> 00:01:14,370 and if it was going to my backup server, 32 32 00:01:14,370 --> 00:01:15,570 this would be normal 33 33 00:01:15,570 --> 00:01:17,850 because it's going from my servers to a backup server 34 34 00:01:17,850 --> 00:01:20,280 which shows a large export of data 35 35 00:01:20,280 --> 00:01:23,040 between 1:00 and 2:00 AM on my servers. 36 36 00:01:23,040 --> 00:01:24,990 Now, if it happened at 1:00 in the afternoon, 37 37 00:01:24,990 --> 00:01:27,630 that would be different, but I expected it this time. 38 38 00:01:27,630 --> 00:01:29,070 And so it really does depend 39 39 00:01:29,070 --> 00:01:30,840 what is causing that traffic spike 40 40 00:01:30,840 --> 00:01:32,730 and that's an important thing to consider. 41 41 00:01:32,730 --> 00:01:34,320 Now, one of the things that could cause one 42 42 00:01:34,320 --> 00:01:37,080 of these traffic spikes that would be malicious in nature 43 43 00:01:37,080 --> 00:01:39,330 would be something like a denial of service attack 44 44 00:01:39,330 --> 00:01:41,580 or a Distributed Denial of Service attack, 45 45 00:01:41,580 --> 00:01:43,710 which is much more likely these days. 46 46 00:01:43,710 --> 00:01:44,543 Now, when you deal 47 47 00:01:44,543 --> 00:01:46,650 with a Distributed Denial of Service attack, 48 48 00:01:46,650 --> 00:01:48,270 this is when you're going to have lots 49 49 00:01:48,270 --> 00:01:50,190 of different hosts all trying to connect 50 50 00:01:50,190 --> 00:01:52,950 to your server at once to waste your resources, 51 51 00:01:52,950 --> 00:01:54,750 whether that's network resources, 52 52 00:01:54,750 --> 00:01:56,790 processor resources, or memory. 53 53 00:01:56,790 --> 00:02:00,570 Essentially, a Distributed Denial of Service attack or DDoS 54 54 00:02:00,570 --> 00:02:03,450 is an attack that uses multiple compromised hosts, 55 55 00:02:03,450 --> 00:02:06,060 usually bots or zombies inside of a botnet, 56 56 00:02:06,060 --> 00:02:08,370 to overwhelm a service with request 57 57 00:02:08,370 --> 00:02:10,080 or with response traffic. 58 58 00:02:10,080 --> 00:02:11,190 And by doing that, 59 59 00:02:11,190 --> 00:02:13,710 we can essentially take your machine offline 60 60 00:02:13,710 --> 00:02:16,110 by attacking it and overwhelming it. 61 61 00:02:16,110 --> 00:02:17,910 Now, one of my favorite examples to use 62 62 00:02:17,910 --> 00:02:18,990 with a denial of service 63 63 00:02:18,990 --> 00:02:20,910 or Distributed Denial of Service attack 64 64 00:02:20,910 --> 00:02:22,560 is thinking about kids. 65 65 00:02:22,560 --> 00:02:24,180 For instance, if you're a parent, 66 66 00:02:24,180 --> 00:02:26,670 you may have one or two or three kids. 67 67 00:02:26,670 --> 00:02:29,250 When I had one kid and they would look up and say, 68 68 00:02:29,250 --> 00:02:32,520 mommy or daddy, I need help, it wasn't that big of a deal. 69 69 00:02:32,520 --> 00:02:34,740 You could stop what you're doing, help the child 70 70 00:02:34,740 --> 00:02:36,630 and then go back to what you're doing. 71 71 00:02:36,630 --> 00:02:38,190 Now over time, if you have more kids, 72 72 00:02:38,190 --> 00:02:40,560 say you have two kids, it becomes a little harder. 73 73 00:02:40,560 --> 00:02:42,690 And so mommy might get a little more overwhelmed 74 74 00:02:42,690 --> 00:02:44,280 because now she's got two kids 75 75 00:02:44,280 --> 00:02:45,870 who are trying to get her attention at all times, 76 76 00:02:45,870 --> 00:02:49,290 going, mommy, mommy, mommy and that can become overwhelming. 77 77 00:02:49,290 --> 00:02:53,010 Now, if I add three or four or five kids, or say 20 kids 78 78 00:02:53,010 --> 00:02:54,780 'cause you're an elementary school teacher, 79 79 00:02:54,780 --> 00:02:56,880 you can really get overwhelmed quickly. 80 80 00:02:56,880 --> 00:02:58,170 Now, that's the difference. 81 81 00:02:58,170 --> 00:03:00,480 When you have one kid, that's a denial of service. 82 82 00:03:00,480 --> 00:03:02,220 Yes, you had to stop what you're doing 83 83 00:03:02,220 --> 00:03:04,980 and take care of the child, but as soon as you're done, 84 84 00:03:04,980 --> 00:03:06,360 you can go right back to what you're doing 85 85 00:03:06,360 --> 00:03:07,980 and you can recover quickly. 86 86 00:03:07,980 --> 00:03:09,990 Now, if you had two or three or five 87 87 00:03:09,990 --> 00:03:13,470 or 10 or 20 kids doing this, you can become overwhelmed 88 88 00:03:13,470 --> 00:03:14,850 and it would take you a lot more time 89 89 00:03:14,850 --> 00:03:16,530 to satisfy all of their needs 90 90 00:03:16,530 --> 00:03:18,450 and you basically can just shut down 91 91 00:03:18,450 --> 00:03:20,460 because there's too much going on. 92 92 00:03:20,460 --> 00:03:22,860 Well, that's what our servers do as well. 93 93 00:03:22,860 --> 00:03:25,230 Now, A DDoS can really overwhelm 94 94 00:03:25,230 --> 00:03:27,180 even the most well defended networks 95 95 00:03:27,180 --> 00:03:28,830 through the sheer volume of traffic 96 96 00:03:28,830 --> 00:03:30,780 that they're going to be exposing you to. 97 97 00:03:30,780 --> 00:03:32,340 This means you have to come up 98 98 00:03:32,340 --> 00:03:34,830 with good mitigation strategies and be prepared for them 99 99 00:03:34,830 --> 00:03:37,800 because even if you have all the best security measures, 100 100 00:03:37,800 --> 00:03:40,350 most DDoS traffic comes at you 101 101 00:03:40,350 --> 00:03:42,390 and it looks like legitimate traffic. 102 102 00:03:42,390 --> 00:03:44,670 So your servers want to try to answer it. 103 103 00:03:44,670 --> 00:03:46,020 Now, one of the other big problems we have 104 104 00:03:46,020 --> 00:03:49,500 with DDoSs these days is that anyone can do them. 105 105 00:03:49,500 --> 00:03:52,380 So it used to be you had to actually be a developer 106 106 00:03:52,380 --> 00:03:55,470 and create this botnet and you'd have to buy resources 107 107 00:03:55,470 --> 00:03:58,650 or hack other people's machines and add them to your botnet. 108 108 00:03:58,650 --> 00:04:01,740 Well, nowadays, people actually sell botnets. 109 109 00:04:01,740 --> 00:04:02,790 So you can actually go in 110 110 00:04:02,790 --> 00:04:05,670 for the low price of 34.99 per month, 111 111 00:04:05,670 --> 00:04:08,190 you can have access to the one month Diamond Plan 112 112 00:04:08,190 --> 00:04:10,140 which will give you 3,600 seconds, 113 113 00:04:10,140 --> 00:04:13,230 which is about six minutes of time to use this DDoS. 114 114 00:04:13,230 --> 00:04:15,390 You can have two concurrent streams going at once, 115 115 00:04:15,390 --> 00:04:16,920 you can get a total network traffic 116 116 00:04:16,920 --> 00:04:19,260 of about 220 gigabits per second 117 117 00:04:19,260 --> 00:04:20,580 and all the tools are included 118 118 00:04:20,580 --> 00:04:22,110 and you get 24, seven support. 119 119 00:04:22,110 --> 00:04:23,340 I mean, who wouldn't want that deal 120 120 00:04:23,340 --> 00:04:25,350 for 34.99 per month, right? 121 121 00:04:25,350 --> 00:04:26,790 This is the idea of how cheap 122 122 00:04:26,790 --> 00:04:28,890 and inexpensive these botnets have gotten 123 123 00:04:28,890 --> 00:04:31,350 and it really takes no skill at all to use them. 124 124 00:04:31,350 --> 00:04:32,850 So anyone can. 125 125 00:04:32,850 --> 00:04:34,680 So your network has to be prepared. 126 126 00:04:34,680 --> 00:04:36,960 Now, as I said before, not all surges 127 127 00:04:36,960 --> 00:04:38,730 in traffic mean you have a botnet, 128 128 00:04:38,730 --> 00:04:41,640 but if you do have a large unexpected surge in traffic 129 129 00:04:41,640 --> 00:04:44,160 from internet hosts, this could be the indication 130 130 00:04:44,160 --> 00:04:47,670 of an ongoing DDoS attack against you using a botnet. 131 131 00:04:47,670 --> 00:04:50,610 Now, it's not a clear indication, but it is an indication 132 132 00:04:50,610 --> 00:04:53,190 and so you'd have to back that up with other factors. 133 133 00:04:53,190 --> 00:04:55,170 Now, what are some of those other factors? 134 134 00:04:55,170 --> 00:04:57,330 Well, you might start seeing an excessive number 135 135 00:04:57,330 --> 00:04:59,940 of time weight connections in your load balancer 136 136 00:04:59,940 --> 00:05:02,310 or inside your web server state table. 137 137 00:05:02,310 --> 00:05:04,200 If you see that, plus you have high numbers 138 138 00:05:04,200 --> 00:05:06,390 of HTTP 503 errors, 139 139 00:05:06,390 --> 00:05:08,460 which is service unavailable log events, 140 140 00:05:08,460 --> 00:05:11,940 this could indicate that a DDoS attack is occurring. 141 141 00:05:11,940 --> 00:05:14,700 Now, those are great indications that you are the victim 142 142 00:05:14,700 --> 00:05:17,550 of a DDoS attack that people are pointing at you 143 143 00:05:17,550 --> 00:05:19,710 but how do you know you're not part of the problem 144 144 00:05:19,710 --> 00:05:21,600 and you don't have hosts compromising your network 145 145 00:05:21,600 --> 00:05:22,890 that are part of this problem 146 146 00:05:22,890 --> 00:05:24,570 that are attacking somebody else? 147 147 00:05:24,570 --> 00:05:26,460 Well, if you happen to see a large amount 148 148 00:05:26,460 --> 00:05:29,220 of outbound traffic from your network, this could indicate 149 149 00:05:29,220 --> 00:05:31,260 that your network contains victimized hosts 150 150 00:05:31,260 --> 00:05:33,990 that are being used in a DDoS against other people. 151 151 00:05:33,990 --> 00:05:36,150 And so you may want to figure out where those hosts are 152 152 00:05:36,150 --> 00:05:37,110 and get them cleaned up 153 153 00:05:37,110 --> 00:05:39,540 because you don't want victims on your network, right? 154 154 00:05:39,540 --> 00:05:40,373 So now that we've talked 155 155 00:05:40,373 --> 00:05:42,750 about DDOS attacks and what they look like coming at us 156 156 00:05:42,750 --> 00:05:44,190 or going against somebody else, 157 157 00:05:44,190 --> 00:05:47,400 we have to talk about how do you measure a DDoS attack? 158 158 00:05:47,400 --> 00:05:48,930 Well, the most common way to measure it 159 159 00:05:48,930 --> 00:05:51,420 is how much bandwidth is being consumed. 160 160 00:05:51,420 --> 00:05:53,280 So you'll do it in megabytes per second 161 161 00:05:53,280 --> 00:05:56,550 or gigabytes per second, or even terabytes per second. 162 162 00:05:56,550 --> 00:05:58,230 Now, when we do this, this is all known 163 163 00:05:58,230 --> 00:05:59,520 as bandwidth consumption. 164 164 00:05:59,520 --> 00:06:01,230 And bandwidth consumption can be measured 165 165 00:06:01,230 --> 00:06:03,630 as the value of bites that were sent or received 166 166 00:06:03,630 --> 00:06:06,300 or as a percentage of your link utilization. 167 167 00:06:06,300 --> 00:06:08,640 For example, if you're reading a news article 168 168 00:06:08,640 --> 00:06:11,280 about a recent Distributed Denial of Service attack, 169 169 00:06:11,280 --> 00:06:12,660 usually it'll say something like this 170 170 00:06:12,660 --> 00:06:16,110 was a 1.5 terabit per second attack, and that's talking 171 171 00:06:16,110 --> 00:06:18,420 about the amount of bites sent are received. 172 172 00:06:18,420 --> 00:06:20,970 Now if you're looking at internally on your organization, 173 173 00:06:20,970 --> 00:06:23,070 you'll know what your total link size is. 174 174 00:06:23,070 --> 00:06:26,130 And so if you have a one gigabit per second connection 175 175 00:06:26,130 --> 00:06:27,810 from your server out to the internet 176 176 00:06:27,810 --> 00:06:30,780 and you see that you've been using 800 megabits per second, 177 177 00:06:30,780 --> 00:06:33,450 you could say that was an 80% link utilization 178 178 00:06:33,450 --> 00:06:34,620 based on this attack. 179 179 00:06:34,620 --> 00:06:36,390 And so both those are ways you can measure it 180 180 00:06:36,390 --> 00:06:39,360 from the outside looking in or the inside looking out. 181 181 00:06:39,360 --> 00:06:41,490 In addition to a standard DDoS attack, 182 182 00:06:41,490 --> 00:06:44,640 there's another kind out there called a DRDoS, 183 183 00:06:44,640 --> 00:06:46,770 and this is a distributed reflection denial 184 184 00:06:46,770 --> 00:06:48,030 of service attack. 185 185 00:06:48,030 --> 00:06:50,280 Now, back in security, plus you probably learned 186 186 00:06:50,280 --> 00:06:52,890 about reflection attacks or amplification attacks 187 187 00:06:52,890 --> 00:06:54,870 and that's really what we're talking about here. 188 188 00:06:54,870 --> 00:06:57,780 A DRDoS attack is a network-based attack 189 189 00:06:57,780 --> 00:07:00,480 where the attacker dramatically increases the bandwidth sent 190 190 00:07:00,480 --> 00:07:02,430 to the victim during the attack 191 191 00:07:02,430 --> 00:07:05,310 by implementing an amplification factor. 192 192 00:07:05,310 --> 00:07:07,380 A DRDoS attack can occur 193 193 00:07:07,380 --> 00:07:10,170 when the adversary spoofs the victim's IP address 194 194 00:07:10,170 --> 00:07:13,620 and tries to open up connections with multiple servers. 195 195 00:07:13,620 --> 00:07:14,910 Now, by doing that, 196 196 00:07:14,910 --> 00:07:17,670 all those servers try to respond back to them. 197 197 00:07:17,670 --> 00:07:20,040 We talked about this back in security plus as well 198 198 00:07:20,040 --> 00:07:22,680 when we talk about the three-way handshake that occurs. 199 199 00:07:22,680 --> 00:07:24,480 When you're trying to connect to a server, 200 200 00:07:24,480 --> 00:07:26,130 you send a SYN packet. 201 201 00:07:26,130 --> 00:07:28,170 The server then will hold a space for you 202 202 00:07:28,170 --> 00:07:29,970 and send back a SYN ACK. 203 203 00:07:29,970 --> 00:07:33,030 It now has resources reserved to handle your request, 204 204 00:07:33,030 --> 00:07:36,480 and then you acknowledge that by sending an ACK message. 205 205 00:07:36,480 --> 00:07:37,920 And then you have a two-way communication 206 206 00:07:37,920 --> 00:07:39,750 that will occur after that three-way handshake. 207 207 00:07:39,750 --> 00:07:43,470 So SYN, SYN ACK, ACK, and then start communicating. 208 208 00:07:43,470 --> 00:07:47,010 Well, it's like I said hi to you and you said hi to me 209 209 00:07:47,010 --> 00:07:47,910 and then I ignored you 210 210 00:07:47,910 --> 00:07:49,920 and you're waiting for me to say something back. 211 211 00:07:49,920 --> 00:07:51,660 You won't hang up the phone until I say something 212 212 00:07:51,660 --> 00:07:53,100 but I give you silence. 213 213 00:07:53,100 --> 00:07:55,980 That's essentially what's happening inside of this attack. 214 214 00:07:55,980 --> 00:07:58,560 Now, you don't have to just use TCP to do this, though. 215 215 00:07:58,560 --> 00:08:00,840 There's lots of different protocols you can use. 216 216 00:08:00,840 --> 00:08:02,610 For instance, one of the most common ones 217 217 00:08:02,610 --> 00:08:05,190 that used to be used was the ICMP attack 218 218 00:08:05,190 --> 00:08:07,890 where you would flood a server with ICMP requests, 219 219 00:08:07,890 --> 00:08:09,300 which are ping requests. 220 220 00:08:09,300 --> 00:08:12,000 Essentially, the attacker would send a ping request 221 221 00:08:12,000 --> 00:08:14,010 to the broadcast of a subnet. 222 222 00:08:14,010 --> 00:08:16,560 Then that subnet has all of the machines on it 223 223 00:08:16,560 --> 00:08:18,360 responding back to that server. 224 224 00:08:18,360 --> 00:08:22,767 This way, I send out one ping, I get back five, 10, 50, 225 225 00:08:22,767 --> 00:08:25,770 100 computers responding to that server. 226 226 00:08:25,770 --> 00:08:29,340 Notice here, the ping request, the source IP is spoofed. 227 227 00:08:29,340 --> 00:08:32,970 It's saying, Hey, I'm coming from 10.1.1.2 228 228 00:08:32,970 --> 00:08:34,980 but that's actually the IP of the server. 229 229 00:08:34,980 --> 00:08:37,260 So when all those hosts on the subnet try to respond 230 230 00:08:37,260 --> 00:08:39,930 to that ping request using an echo reply, 231 231 00:08:39,930 --> 00:08:41,370 all of those replies go back 232 232 00:08:41,370 --> 00:08:44,010 to the server instead of to the original attacker. 233 233 00:08:44,010 --> 00:08:47,370 This is the idea of an amplification and this one uses ICMP. 234 234 00:08:47,370 --> 00:08:48,203 Now, like I said, 235 235 00:08:48,203 --> 00:08:50,520 there are lots of different amplification attacks out there 236 236 00:08:50,520 --> 00:08:52,080 using different protocols. 237 237 00:08:52,080 --> 00:08:53,580 You might use DNS. 238 238 00:08:53,580 --> 00:08:56,040 Now, DNS is a good one that attackers use as well 239 239 00:08:56,040 --> 00:08:58,500 because they can send out a bogus DNS query 240 240 00:08:58,500 --> 00:09:01,560 and then that is only a small request for information 241 241 00:09:01,560 --> 00:09:04,680 but the server has to give back a lot of information. 242 242 00:09:04,680 --> 00:09:06,150 For instance, if the request was, 243 243 00:09:06,150 --> 00:09:08,460 tell me what you know about diontraining.com, 244 244 00:09:08,460 --> 00:09:10,020 the DNS server can come back and say, 245 245 00:09:10,020 --> 00:09:11,220 I know lots about that. 246 246 00:09:11,220 --> 00:09:13,890 I know what its mail server is, and here is its IP. 247 247 00:09:13,890 --> 00:09:16,440 I know that it has this CNAME and that CNAME 248 248 00:09:16,440 --> 00:09:19,170 and this SPF record and this TXT record 249 249 00:09:19,170 --> 00:09:21,780 and it will give you all that information back, 250 250 00:09:21,780 --> 00:09:23,310 and that can overwhelm the server 251 251 00:09:23,310 --> 00:09:25,530 if I send out a lot of these DNS queries 252 252 00:09:25,530 --> 00:09:27,660 and all those responses are coming back to it 253 253 00:09:27,660 --> 00:09:30,930 and we're just taking up resources, that's the idea here. 254 254 00:09:30,930 --> 00:09:32,670 Another amplification attack can be used 255 255 00:09:32,670 --> 00:09:34,320 with Network Time Protocol. 256 256 00:09:34,320 --> 00:09:36,870 Now, the reason why using NTP is so effective 257 257 00:09:36,870 --> 00:09:40,380 is because a single NTP request will generate a response 258 258 00:09:40,380 --> 00:09:42,930 from the server for the last 600 machines 259 259 00:09:42,930 --> 00:09:44,820 that that server has contacted. 260 260 00:09:44,820 --> 00:09:47,460 And so if I make a request to the NTP server, 261 261 00:09:47,460 --> 00:09:50,580 it's going to send back 600 times what I asked for. 262 262 00:09:50,580 --> 00:09:53,370 And so that can really overwhelm people very quickly. 263 263 00:09:53,370 --> 00:09:57,510 Now remember, as I said before, all of this is indications. 264 264 00:09:57,510 --> 00:10:00,060 It doesn't prove there's a DDoS attack 265 265 00:10:00,060 --> 00:10:01,950 but it is good indications of it. 266 266 00:10:01,950 --> 00:10:04,080 Now, when we start dealing with bandwidth consumption 267 267 00:10:04,080 --> 00:10:07,800 and traffic spikes, this can indicate a DDoS attack 268 268 00:10:07,800 --> 00:10:10,950 but it can also be indicative of many other types too. 269 269 00:10:10,950 --> 00:10:13,470 And we'll talk about those as we go through this section 270 270 00:10:13,470 --> 00:10:14,790 and the rest of the course. 271 271 00:10:14,790 --> 00:10:17,190 So I want you to keep that in mind as we go through. 272 272 00:10:17,190 --> 00:10:18,930 Just because you see a traffic spike 273 273 00:10:18,930 --> 00:10:21,990 doesn't mean it's a Distributed Denial of Service attack. 274 274 00:10:21,990 --> 00:10:23,460 Now, the other part of this is just 275 275 00:10:23,460 --> 00:10:25,290 because you see large amounts of bandwidth 276 276 00:10:25,290 --> 00:10:27,810 or traffic spikes being used doesn't mean it 277 277 00:10:27,810 --> 00:10:29,490 was malicious to begin with. 278 278 00:10:29,490 --> 00:10:31,500 For example, if you're running a website, 279 279 00:10:31,500 --> 00:10:32,820 your website can crash 280 280 00:10:32,820 --> 00:10:35,220 under the normal unexpected server load 281 281 00:10:35,220 --> 00:10:36,720 if that load increases 282 282 00:10:36,720 --> 00:10:39,390 because your website becomes popular too quickly. 283 283 00:10:39,390 --> 00:10:41,550 Now, I've seen this happen to a lot of smaller companies 284 284 00:10:41,550 --> 00:10:44,220 but I've also seen this happen to big companies. 285 285 00:10:44,220 --> 00:10:45,900 A couple of years ago, I was working 286 286 00:10:45,900 --> 00:10:48,690 at a security operations center, and at the time, 287 287 00:10:48,690 --> 00:10:51,090 United Airlines had reported they were having issues 288 288 00:10:51,090 --> 00:10:51,923 with their servers 289 289 00:10:51,923 --> 00:10:54,600 and they were having problems all over the US. 290 290 00:10:54,600 --> 00:10:55,920 Now, when this happened, I don't remember 291 291 00:10:55,920 --> 00:10:57,930 if they actually were under attack or not 292 292 00:10:57,930 --> 00:11:00,150 but at the same time, a couple hours later, 293 293 00:11:00,150 --> 00:11:02,700 we started seeing the wallstreetjournal.com 294 294 00:11:02,700 --> 00:11:04,380 and it dropped offline. 295 295 00:11:04,380 --> 00:11:05,850 Now everyone started thinking, 296 296 00:11:05,850 --> 00:11:07,950 oh no, Wall Street Journal's being attacked. 297 297 00:11:07,950 --> 00:11:09,960 And they all thought there was a denial of service going on 298 298 00:11:09,960 --> 00:11:11,760 or Distributed Denial of Service. 299 299 00:11:11,760 --> 00:11:13,560 And when they went through and started looking at it, 300 300 00:11:13,560 --> 00:11:14,970 that isn't what happened. 301 301 00:11:14,970 --> 00:11:15,990 What happened was 302 302 00:11:15,990 --> 00:11:18,630 because the United Airlines had this system crash, 303 303 00:11:18,630 --> 00:11:20,460 which was some kind of a configuration issue, 304 304 00:11:20,460 --> 00:11:22,710 all the news outlets started talking about it 305 305 00:11:22,710 --> 00:11:24,960 and Wall Street Journal was one of those outlets. 306 306 00:11:24,960 --> 00:11:26,610 Because of that, people were going 307 307 00:11:26,610 --> 00:11:28,230 to Wall Street Journal to learn more 308 308 00:11:28,230 --> 00:11:29,400 about this issue with United 309 309 00:11:29,400 --> 00:11:31,830 because if it was really that they were hacked or attacked, 310 310 00:11:31,830 --> 00:11:33,300 that was going to be a huge deal 311 311 00:11:33,300 --> 00:11:35,790 because this was grounding flights all over America. 312 312 00:11:35,790 --> 00:11:37,650 And so this became a big news story. 313 313 00:11:37,650 --> 00:11:39,420 Well, it became such a big news story 314 314 00:11:39,420 --> 00:11:41,820 that so many people were going to the wallstreetjournal.com 315 315 00:11:41,820 --> 00:11:45,090 at the same time that it actually took down their servers. 316 316 00:11:45,090 --> 00:11:47,730 And so it wasn't a malicious denial of service attack 317 317 00:11:47,730 --> 00:11:51,570 or a malicious DDoS but it was a huge traffic spike 318 318 00:11:51,570 --> 00:11:52,920 that was unexpected. 319 319 00:11:52,920 --> 00:11:55,770 And their site became way too popular way too quickly 320 320 00:11:55,770 --> 00:11:58,050 and actually took them down for about an hour. 321 321 00:11:58,050 --> 00:11:59,760 Now, there's actually a name for this effect 322 322 00:11:59,760 --> 00:12:02,100 and it's called the Slashdot Effect. 323 323 00:12:02,100 --> 00:12:03,900 This is also known as slashdotting. 324 324 00:12:03,900 --> 00:12:05,940 Now, this is causing a website to crash 325 325 00:12:05,940 --> 00:12:09,630 when a smaller website can become very popular very quickly 326 326 00:12:09,630 --> 00:12:12,090 due to exposure on some kind of a social sharing site, 327 327 00:12:12,090 --> 00:12:15,000 like Slashdot, Reddit or Twitter, or in the case 328 328 00:12:15,000 --> 00:12:16,830 of the Wall Street Journal, at the time, it was being shown 329 329 00:12:16,830 --> 00:12:18,840 on things like Fox News and CNN 330 330 00:12:18,840 --> 00:12:20,250 and people were going to that website 331 331 00:12:20,250 --> 00:12:22,110 to try to learn more information about it. 332 332 00:12:22,110 --> 00:12:24,090 Regardless, the effect was the same. 333 333 00:12:24,090 --> 00:12:26,760 Way too many legitimate users all going to the site 334 334 00:12:26,760 --> 00:12:29,730 at the same time, making it very popular very quickly 335 335 00:12:29,730 --> 00:12:32,820 and the server couldn't handle it, and it ended up crashing. 336 336 00:12:32,820 --> 00:12:34,860 Now, this usually happens with smaller websites 337 337 00:12:34,860 --> 00:12:36,810 'cause their architecture isn't designed 338 338 00:12:36,810 --> 00:12:40,200 in an elastic manner to handle an exceptionally high load. 339 339 00:12:40,200 --> 00:12:43,410 For instance, if my company became very popular overnight 340 340 00:12:43,410 --> 00:12:45,810 and we went from the couple of hundred thousand users 341 341 00:12:45,810 --> 00:12:47,640 that we have on a daily basis up 342 342 00:12:47,640 --> 00:12:50,040 to a couple of million users on a daily basis, 343 343 00:12:50,040 --> 00:12:51,300 that could actually shut us down 344 344 00:12:51,300 --> 00:12:52,980 because we would grow too quickly. 345 345 00:12:52,980 --> 00:12:54,630 And so we are always looking at our load 346 346 00:12:54,630 --> 00:12:57,300 and expanding as needed based on our user loads, 347 347 00:12:57,300 --> 00:12:58,770 but we don't have a server 348 348 00:12:58,770 --> 00:13:00,630 that can handle millions and millions of users 349 349 00:13:00,630 --> 00:13:02,700 because that would be prohibitively expensive 350 350 00:13:02,700 --> 00:13:04,440 for a small company like us. 351 351 00:13:04,440 --> 00:13:07,260 And so we keep an eye and we expand as we grow 352 352 00:13:07,260 --> 00:13:09,450 but that can also be a downfall for us 353 353 00:13:09,450 --> 00:13:11,400 because if we had 10 million people show up 354 354 00:13:11,400 --> 00:13:13,860 at our site tomorrow, it would shut us down. 355 355 00:13:13,860 --> 00:13:16,650 And so we put different mitigations and protections in place 356 356 00:13:16,650 --> 00:13:18,180 to be able to help prevent us 357 357 00:13:18,180 --> 00:13:20,190 from getting taken down by a huge surge 358 358 00:13:20,190 --> 00:13:22,830 of traffic from something like a DDoS attack. 359 359 00:13:22,830 --> 00:13:25,170 Now, those precautions won't help us 360 360 00:13:25,170 --> 00:13:26,190 from legitimate traffic. 361 361 00:13:26,190 --> 00:13:28,200 So if we became popular overnight like that, 362 362 00:13:28,200 --> 00:13:29,700 we would still get taken down. 363 363 00:13:29,700 --> 00:13:31,170 So we have to keep an eye on that 364 364 00:13:31,170 --> 00:13:32,550 and we're constantly looking at our logs 365 365 00:13:32,550 --> 00:13:35,040 to make sure we grow with our user base. 366 366 00:13:35,040 --> 00:13:37,470 So that brings us to the concept of mitigation. 367 367 00:13:37,470 --> 00:13:40,230 So how can we mitigate a DDoS attack? 368 368 00:13:40,230 --> 00:13:41,790 Well, there are five different ways 369 369 00:13:41,790 --> 00:13:43,050 we're going to talk about. 370 370 00:13:43,050 --> 00:13:46,380 The first, you want to conduct real-time log analysis 371 371 00:13:46,380 --> 00:13:48,750 to identify patterns of suspicious traffic 372 372 00:13:48,750 --> 00:13:51,690 and redirect it to a black hole or a sinkhole. 373 373 00:13:51,690 --> 00:13:52,860 Now, we've talked about black holes 374 374 00:13:52,860 --> 00:13:54,660 and sinkholes previously in this course, 375 375 00:13:54,660 --> 00:13:55,590 and the whole idea here 376 376 00:13:55,590 --> 00:13:57,930 is we have all this malicious traffic coming in 377 377 00:13:57,930 --> 00:14:00,030 from a DDoS, instead of it trying to go 378 378 00:14:00,030 --> 00:14:02,640 into our network and process it, we just want to dump it 379 379 00:14:02,640 --> 00:14:04,800 to this black hole so we don't have to respond to it 380 380 00:14:04,800 --> 00:14:06,750 and we don't waste processing time on it. 381 381 00:14:06,750 --> 00:14:08,100 Next, we want to make sure 382 382 00:14:08,100 --> 00:14:11,040 that we're using geolocation and IP reputation data 383 383 00:14:11,040 --> 00:14:14,010 so we can redirect or ignore suspicious traffic. 384 384 00:14:14,010 --> 00:14:16,890 Let's pretend I rent a restaurant here in the local area. 385 385 00:14:16,890 --> 00:14:18,330 If I run a local restaurant 386 386 00:14:18,330 --> 00:14:20,490 and I have a website to take orders for that, 387 387 00:14:20,490 --> 00:14:21,720 do I need to allow traffic 388 388 00:14:21,720 --> 00:14:24,570 from Russia and China and the Middle East 389 389 00:14:24,570 --> 00:14:27,750 and Europe and any place outside of my local area? 390 390 00:14:27,750 --> 00:14:30,960 Well, no, so in that case, I can use geolocation, say, well, 391 391 00:14:30,960 --> 00:14:33,150 this person's trying to connect to me from California, 392 392 00:14:33,150 --> 00:14:33,990 and since California 393 393 00:14:33,990 --> 00:14:36,180 is way too far out of my delivery radius, 394 394 00:14:36,180 --> 00:14:38,520 they probably don't really want to order food from me 395 395 00:14:38,520 --> 00:14:40,620 so I can just drop that person 396 396 00:14:40,620 --> 00:14:42,750 because they're not my real customer anyway. 397 397 00:14:42,750 --> 00:14:43,950 So that's the idea of redirecting 398 398 00:14:43,950 --> 00:14:46,860 or ignoring any suspicious traffic using geolocation 399 399 00:14:46,860 --> 00:14:48,660 or IP reputational data. 400 400 00:14:48,660 --> 00:14:50,490 People who have already been known to do bad things 401 401 00:14:50,490 --> 00:14:53,100 with that IP address, we can block them as well. 402 402 00:14:53,100 --> 00:14:54,330 Now, the third thing we can do 403 403 00:14:54,330 --> 00:14:56,700 is we can aggressively close slower connections 404 404 00:14:56,700 --> 00:14:59,850 by reducing the timeouts on the affected servers. 405 405 00:14:59,850 --> 00:15:02,400 So let's say my server was getting under load 406 406 00:15:02,400 --> 00:15:04,200 from this DDoS attack, 407 407 00:15:04,200 --> 00:15:06,060 one of the things I could do is actually go in there 408 408 00:15:06,060 --> 00:15:08,640 and change my timeouts, which by default 409 409 00:15:08,640 --> 00:15:11,370 is generally around two to three minutes. 410 410 00:15:11,370 --> 00:15:14,640 So if I have a three minute timeout or 180 seconds, 411 411 00:15:14,640 --> 00:15:15,870 I can actually reduce that maybe down 412 412 00:15:15,870 --> 00:15:19,620 to 60 seconds or 30 seconds or even 10 seconds. 413 413 00:15:19,620 --> 00:15:21,840 And that way if you can't connect within 10 seconds, 414 414 00:15:21,840 --> 00:15:23,100 the connection will be closed. 415 415 00:15:23,100 --> 00:15:24,420 And if you're a legitimate person, 416 416 00:15:24,420 --> 00:15:25,560 you'll probably come back. 417 417 00:15:25,560 --> 00:15:28,290 If you are a Distributed Denial of Service and you're a bot, 418 418 00:15:28,290 --> 00:15:30,360 you might go away and go attack somebody else 419 419 00:15:30,360 --> 00:15:32,910 and hopefully we can ride out that DDoS. 420 420 00:15:32,910 --> 00:15:35,040 The fourth item is we want to use caching 421 421 00:15:35,040 --> 00:15:37,650 and backend infrastructure to offload processing 422 422 00:15:37,650 --> 00:15:38,850 to other servers. 423 423 00:15:38,850 --> 00:15:41,520 If I have one server trying to do all of the load, 424 424 00:15:41,520 --> 00:15:43,380 it can quickly become overwhelmed. 425 425 00:15:43,380 --> 00:15:46,500 So instead, we want to use things like proxies and caches 426 426 00:15:46,500 --> 00:15:48,060 and other backend infrastructure 427 427 00:15:48,060 --> 00:15:50,700 to help offload a lot of that processing to other servers 428 428 00:15:50,700 --> 00:15:53,550 or even use a CDN where we can distribute that load 429 429 00:15:53,550 --> 00:15:55,230 across multiple servers 430 430 00:15:55,230 --> 00:15:57,300 and that way we can suffer through that attack 431 431 00:15:57,300 --> 00:15:59,700 and keep serving our legitimate customers. 432 432 00:15:59,700 --> 00:16:01,380 And then our fifth and final one 433 433 00:16:01,380 --> 00:16:04,950 is we can utilize enterprise DDoS protection services, 434 434 00:16:04,950 --> 00:16:07,650 such as things like CloudFlare or Alchemy. 435 435 00:16:07,650 --> 00:16:09,060 These are commercial providers 436 436 00:16:09,060 --> 00:16:11,100 and they sit in front of your site. 437 437 00:16:11,100 --> 00:16:13,440 So when you actually request to go to my site, 438 438 00:16:13,440 --> 00:16:15,240 it goes first to CloudFlare. 439 439 00:16:15,240 --> 00:16:18,060 CloudFlare checks you, sees if you're part of a DDoS, 440 440 00:16:18,060 --> 00:16:20,490 and if you are they black hole and sinkhole you. 441 441 00:16:20,490 --> 00:16:22,860 If you're not, they redirect you into our site 442 442 00:16:22,860 --> 00:16:24,360 and you get your service. 443 443 00:16:24,360 --> 00:16:26,520 That's the way these type of services work. 444 444 00:16:26,520 --> 00:16:28,740 Now, these services are commercial services. 445 445 00:16:28,740 --> 00:16:30,780 There is a monthly fee for them, and depending 446 446 00:16:30,780 --> 00:16:33,510 on how big your site is and how much protection you need, 447 447 00:16:33,510 --> 00:16:35,370 you'll pay more or less money. 448 448 00:16:35,370 --> 00:16:36,810 They are really good to use though 449 449 00:16:36,810 --> 00:16:39,780 especially if you're a smaller site or a large enterprise 450 450 00:16:39,780 --> 00:16:40,890 because as a smaller site, 451 451 00:16:40,890 --> 00:16:44,250 you can't afford to run your own DDoS protection. 452 452 00:16:44,250 --> 00:16:45,450 And as a larger site, 453 453 00:16:45,450 --> 00:16:46,980 you may not want to have the headache of it. 454 454 00:16:46,980 --> 00:16:48,480 And so you'd rather outsource this 455 455 00:16:48,480 --> 00:16:51,240 to the experts like CloudFlare and Alchemy. 456 456 00:16:51,240 --> 00:16:54,570 So with all that said about Distributed Denial of Service 457 457 00:16:54,570 --> 00:16:57,720 and traffic spikes, what is our goal as a company? 458 458 00:16:57,720 --> 00:17:00,420 Well, your goal as a company and a network defender 459 459 00:17:00,420 --> 00:17:03,180 should be to survive the DDoS attack. 460 460 00:17:03,180 --> 00:17:06,240 Most of the time a DDoS is going to be short-lived. 461 461 00:17:06,240 --> 00:17:08,790 They're not sustained for long periods of time, 462 462 00:17:08,790 --> 00:17:11,670 for hours or days or weeks. 463 463 00:17:11,670 --> 00:17:13,890 Generally, they're just a few minutes. 464 464 00:17:13,890 --> 00:17:16,380 As you saw earlier when I talked about buying a DDoS 465 465 00:17:16,380 --> 00:17:19,050 and DDoS for hire, they were giving you something 466 466 00:17:19,050 --> 00:17:22,440 like a six minute DDoS for that $35 per month. 467 467 00:17:22,440 --> 00:17:25,261 And so generally you're going to see these things last five, 468 468 00:17:25,261 --> 00:17:28,440 10, 15, 20 minutes and then they'll move on. 469 469 00:17:28,440 --> 00:17:30,960 They'll go on to another target or something else. 470 470 00:17:30,960 --> 00:17:32,070 Now, in some cases, 471 471 00:17:32,070 --> 00:17:34,740 there have been sustained DDoS that have gone on 472 472 00:17:34,740 --> 00:17:37,350 for several hours or even as long as a day. 473 473 00:17:37,350 --> 00:17:39,780 But in general, you can just survive 474 474 00:17:39,780 --> 00:17:42,540 by getting through it, mitigating what you can, 475 475 00:17:42,540 --> 00:17:44,460 closing down those connection cycles, 476 476 00:17:44,460 --> 00:17:46,230 offloading things to other servers, 477 477 00:17:46,230 --> 00:17:49,260 utilizing other protections, using black holes and sinkholes 478 478 00:17:49,260 --> 00:17:52,110 and other things like that to at least mitigate it down, 479 479 00:17:52,110 --> 00:17:54,720 try to keep serving your users the best you can 480 480 00:17:54,720 --> 00:17:57,720 until the attack stops and you can go on with your business.