Instructor: Vulnerability reports. 2 2
In this lesson, 3 3
we're going to dig into those vulnerability reports 4 4
and understand a little bit more about them. 5 5
Now, before we do that, 6 6
we have to remember that a vulnerability report 7 7
that is not validated is essentially useless. 8 8
If I run the scanning tool, and I take that report, 9 9
and I hand it into my boss, 10 10
that doesn't do my boss any good 11 11
because that report has not been validated. 12 12
Nobody has looked through it to see, 13 13
is it actually accurate? 14 14
All we've done is trust the system. 15 15
And our job as analysts is to look at these reports 16 16
and validate them. 17 17
Now, when we do that, 18 18
we are looking to look at each of those things 19 19
that are found in that report 20 20
and identify them as one of four categories. 21 21
Are they a true positive, a false positive, a true negative, 22 22
or a false negative? 23 23
Now, you may not understand what all of these are, 24 24
but if you've taken Security+, 25 25
these concepts should be familiar with you. 26 26
Now, for many students, this is an area that confuses them. 27 27
They don't understand the difference 28 28
between these four categories, 29 29
and it's really important that you understand them. 30 30
Now, I covered this in-depth in Security+, 31 31
but I'm going to go through it again here 32 32
because it really is an important concept. 33 33
In fact, on the exam, 34 34
they may give you some issues, 35 35
some things from a security report, 36 36
and ask you to classify them as true positive, 37 37
false positive, true negative, or false negative. 38 38
And if you don't understand the concepts, 39 39
you're not going to be able to get that right. 40 40
So let's talk about them. 41 41
First, true positive. 42 42
What is a true positive? 43 43
Well, this is an alert that matches a vulnerability 44 44
and that vulnerability actually exists 45 45
on the system you scanned. 46 46
So this is something that really happened. 47 47
So what I like to think about 48 48
when I think about these four categories is 49 49
I like to think about pregnancy, 50 50
because if you're married 51 51
and you've tried to have kids before, 52 52
you probably have gone through taking a pregnancy test 53 53
at some point in your life, 54 54
either you or your spouse has. 55 55
For me, it was my spouse who had to take them. 56 56
But the same concept is there. 57 57
And so if we look at that, 58 58
it makes it really easy to understand. 59 59
For instance, I have two children. 60 60
That means at least twice in my life, 61 61
my wife had taken a pregnancy test 62 62
and she showed me the results, 63 63
and it was positive. 64 64
The test said it was positive, that means it was positive. 65 65
And then she actually was pregnant. 66 66
She had a baby inside of her. 67 67
So therefore it was a true positive. 68 68
That's the idea of a true positive. 69 69
So in terms of cybersecurity, 70 70
if you have some bad thing on the network 71 71
and your sensor detected that bad thing and reported on it, 72 72
that's a true positive. 73 73
That's what we want to see. 74 74
We want our tools to be giving us true positives. 75 75
Those are the most helpful for us 76 76
because we know there's a bad thing, 77 77
and we now know it really exists. 78 78
Now, the next one we have is what's called a false positive. 79 79
This is when an alert matches a vulnerability 80 80
and reports on it, 81 81
but when you look at that system, 82 82
the vulnerability doesn't exist. 83 83
So going back to my pregnancy example, 84 84
my wife thought it would be really funny 85 85
when she was taking the pregnancy test 86 86
to have me take one, too. 87 87
And so she had me do the little urine test, 88 88
and of course it came back negative, right? 89 89
Well, if it showed up positive, 90 90
then that would mean it was a false positive. 91 91
Because I'm a guy, 92 92
there's no way that I can be pregnant, right? 93 93
I am just not going to have that capability 94 94
because I don't have the body parts to do that. 95 95
And so that would be a false positive 96 96
if I had a test that showed positive, 97 97
but it was my test because I'm a guy. 98 98
So going back to our computers, 99 99
if we're thinking about something, 100 100
I scan a computer, 101 101
and it says, 102 102
"You have a Windows vulnerability on this system." 103 103
And I look at that system, 104 104
and it's a Linux or a Macintosh system. 105 105
Well, those systems aren't running Windows operating system. 106 106
So therefore I can't have a Windows vulnerability 107 107
on a Mac system. 108 108
So that would be a false positive. 109 109
We can mark that off in our system and say, 110 110
"Ignore that. 111 111
We understand it's not really real." 112 112
That's the idea of a false positive. 113 113
Now, when you're dealing with false positives, 114 114
the problem with them is that they are very time-consuming 115 115
because I now have to go investigate them 116 116
and I start wasting a lot of resources. 117 117
So if I had 50 things pop up 118 118
that told me there was Windows vulnerabilities 119 119
across my network, 120 120
and I start going and looking at each of those machines 121 121
and then I find out there's no Windows on that machine, 122 122
that's a false positive 123 123
and I wasted time looking at each of those 50 things. 124 124
Now, what can you do 125 125
if you start getting a lot of false positives? 126 126
Well, one of the things you can do is 127 127
you can start adjusting your scans 128 128
to a more appropriate scope. 129 129
Maybe I'm going to have one scope of all my Windows machines, 130 130
and another scope for all my Linux machines, 131 131
and another scope for all my Mac machines. 132 132
That way I can quickly identify false positives 133 133
based on operating system 134 134
because of the use cases of those areas. 135 135
Another thing you can do is create a new baseline 136 136
for a heuristic scan. 137 137
If you're using a heuristic scanning engine, 138 138
a lot of those will generate more false positives. 139 139
Signature-based tends to be a little bit more accurate. 140 140
So if you're getting a lot of false positive 141 141
and you're using heuristics, 142 142
go ahead and create a new baseline 143 143
because obviously your baseline is so out of date 144 144
from what you're currently doing, 145 145
it's creating a lot of false positives for you. 146 146
Another thing you might want to do is 147 147
add the application to an exception list. 148 148
For instance, if there's a particular application 149 149
on my system that is throwing up that report 150 150
and giving me that flag, 151 151
I might say, 152 152
"Hey, I understand I'm running Microsoft Word 153 153
on this particular Mac machine, 154 154
but it doesn't mean I'm running Word for Windows." 155 155
So I can put an exception in there 156 156
to ignore that report every time it comes up. 157 157
Another thing you might find is 158 158
that the vulnerability really does exist on that system, 159 159
but that vulnerability isn't really exploitable. 160 160
Now what do I mean by this? 161 161
Maybe I really do have Windows on that system 162 162
because I'm running a virtual machine, 163 163
but there's a firewall in place 164 164
that no inbound or outbound connections can get to it. 165 165
So therefore it's protected, 166 166
and therefore we can say that thing is not exploitable, 167 167
and we can move on. 168 168
That's just a silly example, but you get the idea here. 169 169
Now, when you're dealing with this 170 170
and you start dealing with exceptions 171 171
because you have a vulnerability that exists 172 172
but isn't exploitable 173 173
or an application that you want to do exceptions on, 174 174
this falls under the idea of exception management. 175 175
Now, exception management is a defined process 176 176
to closely monitor systems 177 177
that cannot be patched or remediated 178 178
and must be exempted from those scans. 179 179
Now, the reason we have exception management is 180 180
because otherwise you would be buried 181 181
in these false positives all of the time. 182 182
There are things that would require a valid exception, 183 183
but, again, a thinking analyst needs to go through them, 184 184
figure out if they should be an exception, 185 185
and then follow the process 186 186
to get them added to that exception list. 187 187
The next category we have is 188 188
what's known as a true negative. 189 189
This occurs when an alert is not generated 190 190
because there is no matching vulnerability on that system. 191 191
So going back to my pregnancy example, 192 192
I am a man. 193 193
If I take one of those tests, I expect it to be negative. 194 194
If the test comes back negative, that is a true negative. 195 195
I have a negative result and I am truly not pregnant 196 196
because I'm a man and I'm not pregnant. 197 197
Same thing on cybersecurity. 198 198
For instance, if I scan Windows vulnerabilities 199 199
against my Mac system, 200 200
I expect those results to come back negative. 201 201
There should be nothing found in that report 202 202
because if I'm scanning a Mac system 203 203
for Windows vulnerabilities, 204 204
I should find nothing 205 205
because there is no Windows environment there 206 206
for it to scan. 207 207
That would be a true negative. 208 208
That's a good thing. 209 209
Now, the other side of this is 210 210
what we call a false negative. 211 211
This is something that's actually really scary for us 212 212
in the cybersecurity world. 213 213
When we talk about a false negative, 214 214
this is an alert that is not generated 215 215
even though there's a matching vulnerability on the system. 216 216
So if I go back to my pregnancy example, 217 217
this would be a really sad case. 218 218
There's a couple who's been trying to have a baby. 219 219
They take a pregnancy test and it comes back negative. 220 220
And they're really sad 221 221
because they think they're not pregnant. 222 222
And they've been trying for months and months. 223 223
Couple more weeks go by, 224 224
and then we find out that the woman actually is pregnant. 225 225
Well, she actually was pregnant the whole time, 226 226
but the test didn't generate a result for it 227 227
because it didn't detect it. 228 228
That's what we're talking about here with a false negative. 229 229
She got a negative result, 230 230
but she was actually pregnant the whole time. 231 231
That is a false negative. 232 232
Now, when we talk about this in the cybersecurity world, 233 233
we talk about false negatives, 234 234
meaning that there is a potential vulnerability 235 235
in that system 236 236
or there's a missing patch that hasn't been installed 237 237
and it's not being identified during scanning. 238 238
That's what makes these so dangerous. 239 239
The problem here is you don't know you're vulnerable 240 240
because your report says you're not vulnerable, 241 241
but yet there's vulnerabilities 242 242
that really do exist on the system. 243 243
So this is actually probably the worst of these four cases. 244 244
So something to keep in mind 245 245
as you're going through and working 246 246
as a cybersecurity analyst. 247 247
Now, what can you do to mitigate the threat 248 248
of these false negatives? 249 249
Well, one of the things you can do is run repeated scans. 250 250
Maybe your scanner was having a bad day 251 251
and it just didn't catch it. 252 252
So maybe you want to run another scan 253 253
and it catches it the second time. 254 254
Or you might use different scan types, 255 255
or even a different scanner. 256 256
So maybe I tried scanning it with Nessus 257 257
and it didn't pick it up, 258 258
but now I'm going to scan it with Qualys, 259 259
and that one did pick it up, 260 260
or something of that nature. 261 261
Another thing you might want to do is 262 262
use different sensitivities. 263 263
Maybe you are running it in safe mode. 264 264
Now you're going to run it in not safe mode, 265 265
which will allow you to do more exploitation there 266 266
and find additional vulnerabilities 267 267
that weren't found the first time. 268 268
Another thing you can do is use a different scanner, 269 269
like I said. 270 270
We switch from Nessus to Openvas, or Openvas to Qualys, 271 271
or Qualys to Nikto, 272 272
or whatever we want to do. 273 273
We can use a different scanner 274 274
and that might give us a better result. 275 275
Now, we've done all of our scanning. 276 276
We understand our four types of classifications. 277 277
Let's start validating our scan reports. 278 278
When you run a scan, you're going to get a scan report, 279 279
and they look something like this. 280 280
They'll give you a nice report with an executive summary, 281 281
different vulnerabilities by host, 282 282
and then details on each of those vulnerabilities 283 283
so you can look at them and then validate them. 284 284
So once we have our report, we have to validate that report. 285 285
And there are four things we're going to do to validate it. 286 286
First, we're going to reconcile the results. 287 287
Second, we're going to correlate results with other sources. 288 288
Third, we're going to compare them to best practices. 289 289
And fourth, we're going to identify exceptions. 290 290
So first, we want to reconcile these results 291 291
because the scanners can misinterpret the information 292 292
that they're receiving from their probes. 293 293
Again, this is the idea 294 294
of making sure we identify things as truly positive, 295 295
false positives, truly negative, or false negatives, 296 296
and being able to categorize these things 297 297
as we reconcile those results. 298 298
That is a big part of a cybersecurity analyst job. 299 299
Second, we want to correlate the scan results 300 300
with other data sources. 301 301
And we do this by reviewing related system and network logs. 302 302
So if I went and did a scan that says, 303 303
"Hey, you have a vulnerability. 304 304
Port 80 is open on this workstation," 305 305
I'm going to going to go look at that workstation 306 306
and verify if port 80 is open. 307 307
And if it is open, is it supposed to be open, 308 308
because maybe they are running a web server there 309 309
and they were allowed to do that. 310 310
If they were, 311 311
then that might be something we want to add an exception to. 312 312
Now, the third thing we want to do is 313 313
compare the results to best practices. 314 314
This allows to determine if there's a high priority 315 315
or a low risk associated with this particular finding. 316 316
Sometimes the findings 317 317
in your report are going to be informational. 318 318
It might say something like, 319 319
"The best practice is 320 320
not to run web servers from a workstation. 321 321
You should only do this from a server-based environment 322 322
like Windows Server 2019." 323 323
If that's the case, 324 324
you're going to have to look at that and say, 325 325
"Yeah, I understand that, 326 326
but my organization wants to run this web server 327 327
on Windows 10." 328 328
And if that's what they want to do 329 329
and they've accepted the risk, 330 330
then that's an acceptable thing to do. 331 331
Just make sure you note it. 332 332
And that brings us to our fourth category, 333 333
which is identifying exceptions. 334 334
Any findings whose risk has been accepted 335 335
or transferred by the organization can be added 336 336
to your exception list. 337 337
This way, those same things don't show up week after week 338 338
and month after month. 339 339
This will allow your reports to be smaller in size, 340 340
quicker to reconcile in the future, 341 341
and you're not bringing up the same issues to management 342 342
over and over again 343 343
that they've already made decisions on.