Instructor: Rule and query writing. 2 2
Now, as you're going through all of your data, 3 3
it comes to you in lots of different forms, 4 4
and if you're just looking through log data, 5 5
it's not going to be that useful to you. 6 6
So instead, we write rules to do correlation, 7 7
or we write queries to search information 8 8
to get the data out that we need. 9 9
Now, to do this, we can do this using either 10 10
correlation rules or search queries. 11 11
Let's talk about what each of these are. 12 12
First, what is correlation? 13 13
Well, correlation is the interpretation of the relationship 14 14
between individual data points to diagnose incidents 15 15
of significance to your security team. 16 16
So if I told you that it's 32, that means nothing to you, 17 17
but if I say it's 32 degrees, 18 18
that tells you a little bit more. 19 19
If I say it's 32 degrees and it's lowering, 20 20
that tells you even more, because that's telling you 21 21
that the weather is getting colder and we're going to have 22 22
ice soon, because it's going to start freezing. 23 23
These are the ideas of taking different data points 24 24
and giving it context and correlation. 25 25
Now, to do this in a SIEM, 26 26
we're going to use a SIEM correlation rule. 27 27
This is a statement that matches certain conditions 28 28
as expressed using logical expressions. 29 29
We can use things like and and or or operators, 30 30
like the matches, less than, greater than, 31 31
and contains symbols. 32 32
Now, once we have that logic, we can create a rule using it. 33 33
For example, let's say I wanted to create a rule 34 34
that could be created to send an alert 35 35
if multiple user log-in failures occur within one hour 36 36
from a single account. 37 37
What might that look like? 38 38
Well, we're going to make it look like something like this. 39 39
We're going to have Error.LogonFailure > 3, 40 40
so if more than three log-in failures, 41 41
and the second condition, log-inFailure.User, 42 42
which says there was a log-in failure for this user, 43 43
and the duration was less than one hour. 44 44
So what this says is this rule for this SIEM 45 45
says if I have three log-in failures, 46 46
from one user within one hour, send an alert. 47 47
Now, that works great, and this is what we call 48 48
a correlation rule, but correlation rules 49 49
depend on normalized data. 50 50
If you have data from all sorts of different datasets 51 51
and it's not parsed and normalized first, 52 52
you can't compare it, and so we need to make sure 53 53
that normalization happens. 54 54
Now again, that normalization, that parsing, 55 55
is going to help you get context. 56 56
For instance, what if I had this log file 57 57
which has all these different IP addresses? 58 58
Is that useful? 59 59
Well, maybe or maybe not. 60 60
I need to correlate it and I need to normalize it first, 61 61
so I understand the context of it. 62 62
By knowing that these were IP addresses from a firewall 63 63
that says what's going in or out of my network, 64 64
that's helpful. 65 65
Knowing if they're statically or dynamically assigned, 66 66
that's helpful. 67 67
Making sure the time of them matches all the other systems 68 68
using UTC across my network, that's helpful. 69 69
All of that correlates together to give me more information. 70 70
Now, when we create these correlation rules, 71 71
one of the important things to remember 72 72
is that correlation rules match data 73 73
as it's ingested into your SIEM, 74 74
and this requires the data to stay in memory 75 75
as persistent state data while you're trying to process it. 76 76
So, for the example I had earlier of the three 77 77
log-in failures within one hour, that means I have to 78 78
maintain all the log-in data for up to one hour in memory. 79 79
That stores a lot of information on my server in memory, 80 80
and that can end up slowing down the system 81 81
or causing it to crash if I have a large network. 82 82
So there's got to be a better way to do these 83 83
long-term searches across information. 84 84
And there is! 85 85
It's known as a SIEM query. 86 86
Now, a SIEM query is going to extract records from among all 87 87
the data stored for review or to show it as a visualization. 88 88
Now, when we do this, we are going to go through the data store 89 89
on the SIEM and find all the matching entries. 90 90
Now, one of the big differences is with 91 91
the correlation rules, it will flag an alert immediately. 92 92
For the SIEM query, it waits until you run the query 93 93
and look for that information. 94 94
Now, when I look at a query, 95 95
there's a couple of things we're going to look at. 96 96
We are going to select some field 97 97
where some conditions exist, sorted by some kind of field. 98 98
And so what does this look like? 99 99
Well, let's return to the last example we just had. 100 100
Let's say we want select the user field where 101 101
the error log-in failure is greater than three 102 102
and the log-in failure for that user and the duration 103 103
is less than one hour. 104 104
If all those conditions are met, 105 105
we are going to select that user, where they're going to provide 106 106
that sorted list to us based on the date that it happened 107 107
and the time that it happened. 108 108
Now, this can go back minutes, hours, months, or even years. 109 109
It depends on what you set your filters for. 110 110
In this particular example, 111 111
I am searching the entire database. 112 112
Everything that's in my SIEM for as far back as it goes. 113 113
And again, this is just another way to do this, 114 114
and this is a query as opposed to a rule, 115 115
because the rule is sitting there inside the where, 116 116
but the query is what I'm searching, 117 117
which is the where part of this, where I'm searching it, 118 118
all of those users, and how I'm going to display it, 119 119
what I'm sorting it by.