-: NetFlow data. 2 2
In this lesson, we're going to talk about NetFlow data 3 3
and how it's used to conduct traffic flow analysis 4 4
within our networks. 5 5
In order to best monitor traffic in our network, 6 6
we can either use full packet capture or NetFlow data. 7 7
Now, as you might've guessed, 8 8
packet captures can take up a lot of storage space 9 9
and they can grow quickly in size. 10 10
For example, if I'm conducting a full packet capture 11 11
on my home network each day, 12 12
I would need several gigabytes of storage 13 13
just for my small family, 14 14
because every single packet that goes in or out of my house 15 15
would be captured and logged. 16 16
Every video game my son is playing online, 17 17
every YouTube video he watches, 18 18
every Netflix show my wife is binging. 19 19
All of that will be captured bit-by-bit 20 20
inside of that full packet capture. 21 21
Now, a full packet capture, or FPC, 22 22
is going to capture the entire packet. 23 23
This includes the header and the payload 24 24
for all the traffic that's entering or leaving your network. 25 25
As I said, this would be a ton of information 26 26
and quickly eat up all of our storage. 27 27
Now, because full packet capture takes up so much space, 28 28
we often don't collect it in a lot of organizations. 29 29
Most businesses and organizations instead will use NetFlow. 30 30
Now, NetFlow data, and other similar protocols like that, 31 31
are used to conduct something known as flow analysis. 32 32
Flow analysis will rely on a flow collector 33 33
as a means of recording metadata 34 34
and statistics about network traffic 35 35
instead of recording each and every frame 36 36
or every single packet 37 37
that's going in or out of our network. 38 38
This allows us to use flow analysis tools 39 39
that provide network traffic statistics 40 40
sampled by the collector. 41 41
Now, by doing this, we can capture information 42 42
about the traffic flow instead of the data contained 43 43
within that data flow. 44 44
And this saves us a lot of storage space. 45 45
Now with NetFlow and flow analysis, 46 46
we're not going to have the contents of 47 47
what's going over the network like we would 48 48
with a full packet capture, 49 49
but we can still gather a lot of metadata 50 50
and information about the network traffic 51 51
that's helpful to us in our monitoring. 52 52
This information is stored inside a database 53 53
and it can be queried later by different tools 54 54
to produce different reports and graphs. 55 55
Now, the great thing about flow analysis is 56 56
it's going to allow us to highlight trends and patterns 57 57
in the traffic being generated by our network. 58 58
And this becomes really useful 59 59
in our network performance monitoring. 60 60
Flow analysis will allow us to get alerts 61 61
based on different anomalies we might see 62 62
and different patterns or triggers 63 63
that are outside of our expected baselines. 64 64
These tools also have a visualization component 65 65
that allows us to quickly create 66 66
a map of different network connections 67 67
and the associated flow patterns over those connections. 68 68
By identifying different traffic patterns 69 69
that might reveal bad behavior, malware and transit, 70 70
tunneling, or other bad things out there, 71 71
we're going to be able to quickly respond 72 72
to these potential problems or incidents. 73 73
Now, there are a few different tools we can use 74 74
when dealing with traffic flow analysis. 75 75
This includes things like, NetFlow, Zeek, 76 76
and the Multi Router Traffic Grapher. 77 77
Let's take a look at each of these for a moment. 78 78
First, we have NetFlow. 79 79
NetFlow is a Cisco develop means of reporting 80 80
network flow information to a structured database. 81 81
NetFlow is actually one of the first data flow analyzers 82 82
that was created out there, and eventually, 83 83
it became basically the standard that 84 84
everyone started to use under the term, IPFIX, 85 85
or IP Flow Information Export. 86 86
Now, NetFlow allows us to define a particular traffic flow 87 87
based on different packets that 88 88
share the same characteristics. 89 89
For example, if we want to identify packets 90 90
with the same source and destination IP, 91 91
this could signify there's a session between those two hosts 92 92
and it should be considered one data flow 93 93
that we can collect information on. 94 94
Now, when you look at NetFlow data, 95 95
you can capture information about the packets 96 96
that are going over these devices, 97 97
like the network protocol interface that's being used, 98 98
the version and type of IP being used, 99 99
the source and destination IP address, 100 100
the source and destination port, or the IP type of service. 101 101
All of this information can be gathered using NetFlow 102 102
and then analyzed and displayed visually 103 103
using our different tools. 104 104
For example, here you can see that I'm using SolarWinds 105 105
as a tool to show the NetFlow data of a network. 106 106
But you could also review this data 107 107
in a text-based environment using NetFlow exports themself. 108 108
In this graphical environment though, 109 109
it becomes really easy to see that 110 110
there are 15 different traffic flows. 111 111
And if I expand the 15th data flow, 112 112
we can see the source and destination IP, 113 113
the source port, the destination port, 114 114
some basic information about that data flow, 115 115
but we're not seeing the content of any of those packets 116 116
that were part of this data flow. 117 117
For us to be able to do that, 118 118
we would have to have a full packet capture, 119 119
but here we only captured the metadata 120 120
or the information about those traffic flows. 121 121
Now, if you want to be able to have the best of both worlds, 122 122
you can use something like Zeek. 123 123
Now, Zeek is a hybrid tool 124 124
that passively monitors your network like a sniffer, 125 125
but it's only going to log full packet captures 126 126
based on data of potential interest. 127 127
Essentially, Zeek is going to sample the data 128 128
going across the network, just like NetFlow does, 129 129
but when Zeek finds something that it deems interesting, 130 130
based on the parameters and rules you've configured, 131 131
it's going to log the entire packet for that part 132 132
and then send it over to our cybersecurity analyst 133 133
for further investigation. 134 134
This method helps us reduce our storage 135 135
and processing requirements, 136 136
and it gives us the ability to have all this data 137 137
in a single database. 138 138
Now, one of the great things about Zeek is that 139 139
it performs normalization of this data as well, 140 140
and then stores it as either a tab-delimited, 141 141
or JavaScript Object Notation, or JSON-formatted text file. 142 142
This allows you to use it with 143 143
lots of other different cybersecurity tools 144 144
and different network monitoring tools as well. 145 145
For example, now that I have this normalized data, 146 146
I can import that data into another tool for visualization, 147 147
searching and analysis. 148 148
Here, I've imported my Zeek logs into Splunk, 149 149
and then I can have my cybersecurity analyst 150 150
search for specific information during a potential incident. 151 151
Now, the third tool we have to talk about is 152 152
MRTG, or the Multi Router Traffic Grapher. 153 153
The Multi Router Traffic Grapher is a tool 154 154
that's used to create graphs to show network traffic flows 155 155
going through our network interfaces 156 156
on different routers and switches 157 157
and it does this by pulling these appliances, 158 158
using SNMP, the simple network management protocol. 159 159
So, what is useful about a visualization like this? 160 160
Well, you're going to be able to start seeing patterns emerge 161 161
that may be outside of your baseline. 162 162
For example, here in the top graph, 163 163
you could see a big spike in traffic 164 164
between 2:00 AM and 4:00 AM. 165 165
Is that normal? 166 166
Well, maybe, and maybe not, 167 167
but it's something we should further investigate and analyze 168 168
because we're seeing this big spike occur 169 169
between 2:00 AM and 4:00 AM. 170 170
And that might be something normal, 171 171
like doing offsite backups, 172 172
or it could be something malicious. 173 173
If it was the case of something that was normal, 174 174
like an offsite backup, 175 175
you're going to see this big spike in traffic 176 176
because we're sending a backup copy of 177 177
all of our data offsite to a cloud provider facility. 178 178
That might be a reasonable explanation. 179 179
And in that case, I wouldn't need to worry 180 180
because I would see that every single night 181 181
and I'd be used to seeing it. 182 182
Now, on the other hand, 183 183
maybe that server has been infected with malware 184 184
and every day at 2-4:00 AM, 185 185
it's going to send all of the data back to the bad actors 186 186
while all my administrators are at home sleeping. 187 187
This is considered data exfiltration 188 188
as part of an attack campaign, 189 189
that's something you want to be on the lookout for. 190 190
Now, just looking at this graphic, 191 191
I don't know which of these two cases it is. 192 192
Is this something normal, like a backup, 193 193
or is this something malicious? 194 194
But if you know your organization 195 195
and you know your baselines, 196 196
now you can look at this graph and identify 197 197
what should be investigated based on seeing 198 198
that spike between 2:00 AM and 4:00 AM, 199 199
and then figuring out where 200 200
that additional traffic flow is going, and why. 201 201
If we suspected something was malicious here, 202 202
like somebody exfiltrating our data, 203 203
then we might set up a network sniffer 204 204
in front of our file server and see 205 205
what traffic is leaving the network and where it's going. 206 206
Then, based on that, we can have an instant response 207 207
on our hands and do our cleanup. 208 208
Now at this point, we just don't know 209 209
if this is malicious or not, 210 210
but we do know it's something different 211 211
and something that is outside of the normal baseline 212 212
as indicated by that big spike. 213 213
So, it's important for us to investigate it 214 214
for the health of our network.