1 1 00:00:00,010 --> 00:00:01,730 NetFlow data. 2 2 00:00:01,730 --> 00:00:04,460 In this lesson, we're going to talk about NetFlow data 3 3 00:00:04,460 --> 00:00:06,600 and how it's used to conduct traffic flow analysis 4 4 00:00:06,600 --> 00:00:08,000 within our networks. 5 5 00:00:08,000 --> 00:00:10,009 In order to best monitor traffic in our network, 6 6 00:00:10,009 --> 00:00:13,636 we can either use full packet capture or NetFlow data. 7 7 00:00:13,636 --> 00:00:15,320 Now, as you might've guessed, 8 8 00:00:15,320 --> 00:00:18,080 packet captures can take up a lot of storage space 9 9 00:00:18,080 --> 00:00:19,970 and they can grow quickly in size. 10 10 00:00:19,970 --> 00:00:22,390 For example, if I'm conducting a full packet capture 11 11 00:00:22,390 --> 00:00:23,830 on my home network each day, 12 12 00:00:23,830 --> 00:00:25,700 I would need several gigabytes of storage 13 13 00:00:25,700 --> 00:00:27,110 just for my small family, 14 14 00:00:27,110 --> 00:00:29,920 because every single packet that goes in or out of my house 15 15 00:00:29,920 --> 00:00:31,620 would be captured and logged. 16 16 00:00:31,620 --> 00:00:33,520 Every video game my son is playing online, 17 17 00:00:33,520 --> 00:00:35,330 every YouTube video he watches, 18 18 00:00:35,330 --> 00:00:37,490 every Netflix show my wife is binging. 19 19 00:00:37,490 --> 00:00:39,640 All of that will be captured bit-by-bit 20 20 00:00:39,640 --> 00:00:41,980 inside of that full packet capture. 21 21 00:00:41,980 --> 00:00:44,400 Now, a full packet capture, or FPC, 22 22 00:00:44,400 --> 00:00:46,510 is going to capture the entire packet. 23 23 00:00:46,510 --> 00:00:48,520 This includes the header and the payload 24 24 00:00:48,520 --> 00:00:51,320 for all the traffic that's entering or leaving your network. 25 25 00:00:51,320 --> 00:00:53,910 As I said, this would be a ton of information 26 26 00:00:53,910 --> 00:00:56,100 and quickly eat up all of our storage. 27 27 00:00:56,100 --> 00:00:59,240 Now, because full packet capture takes up so much space, 28 28 00:00:59,240 --> 00:01:02,400 we often don't collect it in a lot of organizations. 29 29 00:01:02,400 --> 00:01:06,096 Most businesses and organizations instead will use NetFlow. 30 30 00:01:06,096 --> 00:01:09,300 Now, NetFlow data, and other similar protocols like that, 31 31 00:01:09,300 --> 00:01:12,360 are used to conduct something known as flow analysis. 32 32 00:01:12,360 --> 00:01:14,550 Flow analysis will rely on a flow collector 33 33 00:01:14,550 --> 00:01:16,220 as a means of recording metadata 34 34 00:01:16,220 --> 00:01:18,390 and statistics about network traffic 35 35 00:01:18,390 --> 00:01:20,450 instead of recording each and every frame 36 36 00:01:20,450 --> 00:01:21,620 or every single packet 37 37 00:01:21,620 --> 00:01:23,560 that's going in or out of our network. 38 38 00:01:23,560 --> 00:01:25,800 This allows us to use flow analysis tools 39 39 00:01:25,800 --> 00:01:27,570 that provide network traffic statistics 40 40 00:01:27,570 --> 00:01:29,260 sampled by the collector. 41 41 00:01:29,260 --> 00:01:31,370 Now, by doing this, we can capture information 42 42 00:01:31,370 --> 00:01:33,980 about the traffic flow instead of the data contained 43 43 00:01:33,980 --> 00:01:35,450 within that data flow. 44 44 00:01:35,450 --> 00:01:38,090 And this saves us a lot of storage space. 45 45 00:01:38,090 --> 00:01:40,120 Now with NetFlow and flow analysis, 46 46 00:01:40,120 --> 00:01:41,500 we're not going to have the contents of 47 47 00:01:41,500 --> 00:01:43,230 what's going over the network like we would 48 48 00:01:43,230 --> 00:01:44,810 with a full packet capture, 49 49 00:01:44,810 --> 00:01:47,130 but we can still gather a lot of metadata 50 50 00:01:47,130 --> 00:01:49,350 and information about the network traffic 51 51 00:01:49,350 --> 00:01:51,390 that's helpful to us in our monitoring. 52 52 00:01:51,390 --> 00:01:53,600 This information is stored inside a database 53 53 00:01:53,600 --> 00:01:55,570 and it can be queried later by different tools 54 54 00:01:55,570 --> 00:01:57,830 to produce different reports and graphs. 55 55 00:01:57,830 --> 00:01:59,730 Now, the great thing about flow analysis is 56 56 00:01:59,730 --> 00:02:02,040 it's going to allow us to highlight trends and patterns 57 57 00:02:02,040 --> 00:02:04,100 in the traffic being generated by our network. 58 58 00:02:04,100 --> 00:02:05,630 And this becomes really useful 59 59 00:02:05,630 --> 00:02:07,570 in our network performance monitoring. 60 60 00:02:07,570 --> 00:02:09,390 Flow analysis will allow us to get alerts 61 61 00:02:09,390 --> 00:02:11,140 based on different anomalies we might see 62 62 00:02:11,140 --> 00:02:12,630 and different patterns or triggers 63 63 00:02:12,630 --> 00:02:15,070 that are outside of our expected baselines. 64 64 00:02:15,070 --> 00:02:17,300 These tools also have a visualization component 65 65 00:02:17,300 --> 00:02:18,720 that allows us to quickly create 66 66 00:02:18,720 --> 00:02:20,440 a map of different network connections 67 67 00:02:20,440 --> 00:02:23,130 and the associated flow patterns over those connections. 68 68 00:02:23,130 --> 00:02:24,760 By identifying different traffic patterns 69 69 00:02:24,760 --> 00:02:27,500 that might reveal bad behavior, malware and transit, 70 70 00:02:27,500 --> 00:02:29,750 tunneling, or other bad things out there, 71 71 00:02:29,750 --> 00:02:31,280 we're going to be able to quickly respond 72 72 00:02:31,280 --> 00:02:33,470 to these potential problems or incidents. 73 73 00:02:33,470 --> 00:02:35,570 Now, there are a few different tools we can use 74 74 00:02:35,570 --> 00:02:37,610 when dealing with traffic flow analysis. 75 75 00:02:37,610 --> 00:02:39,780 This includes things like, NetFlow, Zeek, 76 76 00:02:39,780 --> 00:02:41,950 and the Multi Router Traffic Grapher. 77 77 00:02:41,950 --> 00:02:44,060 Let's take a look at each of these for a moment. 78 78 00:02:44,060 --> 00:02:45,800 First, we have NetFlow. 79 79 00:02:45,800 --> 00:02:47,860 NetFlow is a Cisco develop means of reporting 80 80 00:02:47,860 --> 00:02:50,363 network flow information to a structured database. 81 81 00:02:50,363 --> 00:02:53,070 NetFlow is actually one of the first data flow analyzers 82 82 00:02:53,070 --> 00:02:55,130 that was created out there, and eventually, 83 83 00:02:55,130 --> 00:02:56,690 it became basically the standard that 84 84 00:02:56,690 --> 00:03:00,370 everyone started to use under the term, IPFIX, 85 85 00:03:00,370 --> 00:03:02,960 or IP Flow Information Export. 86 86 00:03:02,960 --> 00:03:06,230 Now, NetFlow allows us to define a particular traffic flow 87 87 00:03:06,230 --> 00:03:07,480 based on different packets that 88 88 00:03:07,480 --> 00:03:09,210 share the same characteristics. 89 89 00:03:09,210 --> 00:03:11,320 For example, if we want to identify packets 90 90 00:03:11,320 --> 00:03:13,430 with the same source and destination IP, 91 91 00:03:13,430 --> 00:03:15,698 this could signify there's a session between those two hosts 92 92 00:03:15,698 --> 00:03:18,020 and it should be considered one data flow 93 93 00:03:18,020 --> 00:03:19,790 that we can collect information on. 94 94 00:03:19,790 --> 00:03:21,470 Now, when you look at NetFlow data, 95 95 00:03:21,470 --> 00:03:23,450 you can capture information about the packets 96 96 00:03:23,450 --> 00:03:25,240 that are going over these devices, 97 97 00:03:25,240 --> 00:03:27,550 like the network protocol interface that's being used, 98 98 00:03:27,550 --> 00:03:29,560 the version and type of IP being used, 99 99 00:03:29,560 --> 00:03:31,480 the source and destination IP address, 100 100 00:03:31,480 --> 00:03:34,950 the source and destination port, or the IP type of service. 101 101 00:03:34,950 --> 00:03:37,420 All of this information can be gathered using NetFlow 102 102 00:03:37,420 --> 00:03:39,310 and then analyzed and displayed visually 103 103 00:03:39,310 --> 00:03:40,890 using our different tools. 104 104 00:03:40,890 --> 00:03:43,680 For example, here you can see that I'm using SolarWinds 105 105 00:03:43,680 --> 00:03:46,810 as a tool to show the NetFlow data of a network. 106 106 00:03:46,810 --> 00:03:48,570 But you could also review this data 107 107 00:03:48,570 --> 00:03:51,601 in a text-based environment using NetFlow exports themself. 108 108 00:03:51,601 --> 00:03:53,470 In this graphical environment though, 109 109 00:03:53,470 --> 00:03:55,130 it becomes really easy to see that 110 110 00:03:55,130 --> 00:03:57,110 there are 15 different traffic flows. 111 111 00:03:57,110 --> 00:03:59,110 And if I expand the 15th data flow, 112 112 00:03:59,110 --> 00:04:01,200 we can see the source and destination IP, 113 113 00:04:01,200 --> 00:04:03,300 the source port, the destination port, 114 114 00:04:03,300 --> 00:04:05,580 some basic information about that data flow, 115 115 00:04:05,580 --> 00:04:08,630 but we're not seeing the content of any of those packets 116 116 00:04:08,630 --> 00:04:10,350 that were part of this data flow. 117 117 00:04:10,350 --> 00:04:11,490 For us to be able to do that, 118 118 00:04:11,490 --> 00:04:13,600 we would have to have a full packet capture, 119 119 00:04:13,600 --> 00:04:15,870 but here we only captured the metadata 120 120 00:04:15,870 --> 00:04:18,337 or the information about those traffic flows. 121 121 00:04:18,337 --> 00:04:21,393 Now, if you want to be able to have the best of both worlds, 122 122 00:04:21,393 --> 00:04:23,520 you can use something like Zeek. 123 123 00:04:23,520 --> 00:04:25,070 Now, Zeek is a hybrid tool 124 124 00:04:25,070 --> 00:04:27,720 that passively monitors your network like a sniffer, 125 125 00:04:27,720 --> 00:04:30,560 but it's only going to log full packet captures 126 126 00:04:30,560 --> 00:04:32,800 based on data of potential interest. 127 127 00:04:32,800 --> 00:04:35,040 Essentially, Zeek is going to sample the data 128 128 00:04:35,040 --> 00:04:37,500 going across the network, just like NetFlow does, 129 129 00:04:37,500 --> 00:04:39,870 but when Zeek finds something that it deems interesting, 130 130 00:04:39,870 --> 00:04:42,140 based on the parameters and rules you've configured, 131 131 00:04:42,140 --> 00:04:44,800 it's going to log the entire packet for that part 132 132 00:04:44,800 --> 00:04:46,870 and then send it over to our cybersecurity analyst 133 133 00:04:46,870 --> 00:04:48,510 for further investigation. 134 134 00:04:48,510 --> 00:04:50,330 This method helps us reduce our storage 135 135 00:04:50,330 --> 00:04:51,480 and processing requirements, 136 136 00:04:51,480 --> 00:04:53,970 and it gives us the ability to have all this data 137 137 00:04:53,970 --> 00:04:55,650 in a single database. 138 138 00:04:55,650 --> 00:04:57,460 Now, one of the great things about Zeek is that 139 139 00:04:57,460 --> 00:04:59,690 it performs normalization of this data as well, 140 140 00:04:59,690 --> 00:05:02,010 and then stores it as either a tab-delimited, 141 141 00:05:02,010 --> 00:05:05,555 or JavaScript Object Notation, or JSON-formatted text file. 142 142 00:05:05,555 --> 00:05:07,030 This allows you to use it with 143 143 00:05:07,030 --> 00:05:09,050 lots of other different cybersecurity tools 144 144 00:05:09,050 --> 00:05:11,400 and different network monitoring tools as well. 145 145 00:05:11,400 --> 00:05:14,120 For example, now that I have this normalized data, 146 146 00:05:14,120 --> 00:05:17,010 I can import that data into another tool for visualization, 147 147 00:05:17,010 --> 00:05:18,670 searching and analysis. 148 148 00:05:18,670 --> 00:05:21,280 Here, I've imported my Zeek logs into Splunk, 149 149 00:05:21,280 --> 00:05:23,430 and then I can have my cybersecurity analyst 150 150 00:05:23,430 --> 00:05:26,637 search for specific information during a potential incident. 151 151 00:05:26,637 --> 00:05:29,110 Now, the third tool we have to talk about is 152 152 00:05:29,110 --> 00:05:32,209 MRTG, or the Multi Router Traffic Grapher. 153 153 00:05:32,209 --> 00:05:34,790 The Multi Router Traffic Grapher is a tool 154 154 00:05:34,790 --> 00:05:37,350 that's used to create graphs to show network traffic flows 155 155 00:05:37,350 --> 00:05:39,140 going through our network interfaces 156 156 00:05:39,140 --> 00:05:40,660 on different routers and switches 157 157 00:05:40,660 --> 00:05:42,830 and it does this by pulling these appliances, 158 158 00:05:42,830 --> 00:05:46,140 using SNMP, the simple network management protocol. 159 159 00:05:46,140 --> 00:05:48,567 So, what is useful about a visualization like this? 160 160 00:05:48,567 --> 00:05:51,260 Well, you're going to be able to start seeing patterns emerge 161 161 00:05:51,260 --> 00:05:53,280 that may be outside of your baseline. 162 162 00:05:53,280 --> 00:05:55,040 For example, here in the top graph, 163 163 00:05:55,040 --> 00:05:56,750 you could see a big spike in traffic 164 164 00:05:56,750 --> 00:05:58,780 between 2:00 AM and 4:00 AM. 165 165 00:05:58,780 --> 00:05:59,770 Is that normal? 166 166 00:05:59,770 --> 00:06:01,580 Well, maybe, and maybe not, 167 167 00:06:01,580 --> 00:06:04,020 but it's something we should further investigate and analyze 168 168 00:06:04,020 --> 00:06:05,880 because we're seeing this big spike occur 169 169 00:06:05,880 --> 00:06:07,760 between 2:00 AM and 4:00 AM. 170 170 00:06:07,760 --> 00:06:09,280 And that might be something normal, 171 171 00:06:09,280 --> 00:06:10,580 like doing offsite backups, 172 172 00:06:10,580 --> 00:06:12,140 or it could be something malicious. 173 173 00:06:12,140 --> 00:06:14,370 If it was the case of something that was normal, 174 174 00:06:14,370 --> 00:06:15,690 like an offsite backup, 175 175 00:06:15,690 --> 00:06:17,620 you're going to see this big spike in traffic 176 176 00:06:17,620 --> 00:06:19,200 because we're sending a backup copy of 177 177 00:06:19,200 --> 00:06:22,590 all of our data offsite to a cloud provider facility. 178 178 00:06:22,590 --> 00:06:24,330 That might be a reasonable explanation. 179 179 00:06:24,330 --> 00:06:26,090 And in that case, I wouldn't need to worry 180 180 00:06:26,090 --> 00:06:27,990 because I would see that every single night 181 181 00:06:27,990 --> 00:06:29,700 and I'd be used to seeing it. 182 182 00:06:29,700 --> 00:06:30,910 Now, on the other hand, 183 183 00:06:30,910 --> 00:06:33,000 maybe that server has been infected with malware 184 184 00:06:33,000 --> 00:06:35,000 and every day at 2-4:00 AM, 185 185 00:06:35,000 --> 00:06:38,420 it's going to send all of the data back to the bad actors 186 186 00:06:38,420 --> 00:06:40,540 while all my administrators are at home sleeping. 187 187 00:06:40,540 --> 00:06:42,630 This is considered data exfiltration 188 188 00:06:42,630 --> 00:06:44,160 as part of an attack campaign, 189 189 00:06:44,160 --> 00:06:46,150 that's something you want to be on the lookout for. 190 190 00:06:46,150 --> 00:06:47,680 Now, just looking at this graphic, 191 191 00:06:47,680 --> 00:06:49,650 I don't know which of these two cases it is. 192 192 00:06:49,650 --> 00:06:51,320 Is this something normal, like a backup, 193 193 00:06:51,320 --> 00:06:52,910 or is this something malicious? 194 194 00:06:52,910 --> 00:06:54,820 But if you know your organization 195 195 00:06:54,820 --> 00:06:56,250 and you know your baselines, 196 196 00:06:56,250 --> 00:06:58,740 now you can look at this graph and identify 197 197 00:06:58,740 --> 00:07:00,790 what should be investigated based on seeing 198 198 00:07:00,790 --> 00:07:02,970 that spike between 2:00 AM and 4:00 AM, 199 199 00:07:02,970 --> 00:07:03,803 and then figuring out where 200 200 00:07:03,803 --> 00:07:06,540 that additional traffic flow is going, and why. 201 201 00:07:06,540 --> 00:07:08,630 If we suspected something was malicious here, 202 202 00:07:08,630 --> 00:07:10,460 like somebody exfiltrating our data, 203 203 00:07:10,460 --> 00:07:12,010 then we might set up a network sniffer 204 204 00:07:12,010 --> 00:07:13,820 in front of our file server and see 205 205 00:07:13,820 --> 00:07:16,510 what traffic is leaving the network and where it's going. 206 206 00:07:16,510 --> 00:07:19,200 Then, based on that, we can have an instant response 207 207 00:07:19,200 --> 00:07:21,140 on our hands and do our cleanup. 208 208 00:07:21,140 --> 00:07:23,070 Now at this point, we just don't know 209 209 00:07:23,070 --> 00:07:24,470 if this is malicious or not, 210 210 00:07:24,470 --> 00:07:26,150 but we do know it's something different 211 211 00:07:26,150 --> 00:07:28,660 and something that is outside of the normal baseline 212 212 00:07:28,660 --> 00:07:30,490 as indicated by that big spike. 213 213 00:07:30,490 --> 00:07:32,150 So, it's important for us to investigate it 214 214 00:07:32,150 --> 00:07:33,650 for the health of our network.