Instructor: Proxy logs. 2 2
In this lesson we're going to discuss proxy logs. 3 3
Now going back to your earlier certification studies, 4 4
you should remember that a proxy server 5 5
acts as a gateway between you and the internet. 6 6
Basically, it's an intermediary server, 7 7
that separates the end users 8 8
from the websites they're trying to browse. 9 9
Proxy servers are going to provide varying levels 10 10
of functionality, security, and privacy, 11 11
depending on your use case, your needs, 12 12
and your company policy. 13 13
Now, the first type of proxy server we have 14 14
is what's known as a forward proxy. 15 15
Now, a forward proxy is a server that mediates 16 16
the communications between a client and another server. 17 17
And it can filter or modify communications, 18 18
and provide caching services to improve performance. 19 19
Now basically, a forward proxy is going to act on behalf 20 20
of your internal host or workstation, 21 21
and forward their HTTP request to the intended destination. 22 22
Basically, when I'm sitting at work, I'm on my computer, 23 23
it's going to go from me to the proxy server, 24 24
and then from the proxy server, 25 25
out to the website I want to connect to. 26 26
Now, proxies can be classified in one of two ways. 27 27
They can either be non-transparent, or transparent. 28 28
When I talk about a non-transparent proxy, 29 29
this is a server that redirects requests 30 30
and responses from the client's configured, 31 31
with the proxy address and their port. 32 32
Now, these non-transparent proxies 33 33
are ones where you know there's a proxy being there. 34 34
If you go into your internet settings of your browser, 35 35
you're going to see that there is a proxy configured, 36 36
and that your traffic is going from you to that proxy, 37 37
and from that proxy out to the internet. 38 38
Now on the other hand, we have transparent proxies. 39 39
These are forced, or intercepting proxies. 40 40
This is when a server redirects requests and responses, 41 41
without the clients being explicitly configured to use it. 42 42
Now, this is often used if you want to make sure 43 43
your employees aren't turning off your proxy, 44 44
because this is done at a network layer, 45 45
so all the machines are going to go through 46 46
this transparent proxy, 47 47
not just the ones that are configured for it. 48 48
Now, an analysis of your proxy logs 49 49
can reveal a lot of information for you, 50 50
about the exact nature of these different web requests. 51 51
This includes websites that the users visit, 52 52
and the contents of each request. 53 53
By using a proxy server and logging all this information, 54 54
you can actually understand 55 55
exactly what your users are doing, 56 56
which websites are going to, and how long they spent there. 57 57
Now, a proxy is going to use different log formats 58 58
depending on the proxy you're using, 59 59
but one of the most common is the common log format. 60 60
This is the same one used by web servers. 61 61
We took a look at this earlier 62 62
when we looked at the Microsoft logs on the firewall lesson. 63 63
Here again, you see the information 64 64
in that last couple of lines. 65 65
This tells you the date, the time, the destination, 66 66
the type of request it was, what port it went to, 67 67
what the action was, and things like that. 68 68
This is the same type of log file 69 69
that we talked about when we talked about Windows firewalls. 70 70
Inside this, you'll see information 71 71
like the user ID of the client 72 72
when it's authenticated to the proxy, 73 73
the request method that was being used, 74 74
like a get or a post request. 75 75
It'll tell you the status code, 76 76
such as status code 200, or 403, or 404, 77 77
and the size in bytes, 78 78
and the type of information that was returned to the client. 79 79
Whether it was a text file, 80 80
or a picture, or something like that. 81 81
Now, proxies that are set to intercept or block traffic, 82 82
can also record the rule that the request matched, 83 83
to determine the employee's intent. 84 84
So if somebody is going out and trying to access something, 85 85
you can determine was there malicious intent, 86 86
or harmless intent. 87 87
Let's take a look at an example here. 88 88
Here on the screen you can see a Squid access table. 89 89
This comes from a Squid logging program, 90 90
which is a proxy server. 91 91
Now here, if we start from the bottom up, 92 92
we go from the oldest entries to the newest entries. 93 93
As we look through these logs, 94 94
what do we think is happening here? 95 95
Well, as we start looking, we start seeing this website, 96 96
515web.net, and you can see there was a code 200. 97 97
If you remember back to our HTTP lesson, 98 98
code 200 means successful attempt. 99 99
So this was an okay, permitted access action, 100 100
at this particular time. 101 101
Now, about seven minutes later, they tried again, 102 102
and instead they went to www.515web.net. 103 103
And again, this was allowed. 104 104
Now about 90 seconds after that, 105 105
we see they tried to go back to that same website, 106 106
www.515web.net, and you'll notice here, there's a 403 error. 107 107
What happened that's different? 108 108
Well, between that 90 seconds, 109 109
somebody released a policy update, 110 110
and it told the proxy server, for now on, 111 111
that website is considered something that's not allowed, 112 112
and so it's being blocked by the proxy. 113 113
And you can see that there with TCP_denied as the status, 114 114
and a 403 error. 115 115
This is just some of the information you can find 116 116
when looking through these proxy logs. 117 117
This particular snippet is only showing 118 118
a couple of the columns, you're not seeing all the content, 119 119
and the mime data inside of those requests, 120 120
I just showed you the status and the address, 121 121
and the IP address and the date for this particular example. 122 122
Now, another type of proxy we have, 123 123
is known as a reverse proxy, 124 124
and this is a type of proxy server that protects servers 125 125
from direct contact with the client request. 126 126
The idea with a reverse proxy 127 127
is to provide for protocol specific inbound traffic. 128 128
When we get a request from the public internet, 129 129
it goes to the proxy server first. 130 130
Then, that proxy server can create the appropriate request 131 131
to the internal server, whether it's a mail server, 132 132
or a web server, or something like that, get that request, 133 133
and then send it back to the external client. 134 134
This means the external client 135 135
never directly touches your servers, 136 136
it only touches your proxy server, 137 137
and this can help protect things from malicious traffic. 138 138
Now the other great thing about doing this, 139 139
is you get a great source of logs 140 140
right at that reverse proxy, 141 141
that you can analyze for indicators of attack or compromise, 142 142
such as malicious code and HTTP request headers or URLs. 143 143
This way, you have a single point to check all those logs, 144 144
because everything is going in through that proxy, 145 145
anything from the outside internet coming into your network 146 146
to be able to touch one of your internal servers, 147 147
so this is a great way to do this. 148 148
Now here on the screen, you could see another example 149 149
of a Squid log from a proxy server. 150 150
This one is coming from a reverse proxy. 151 151
Notice, everything coming in from the IPs are external IPs, 152 152
every destination is an internal server, 153 153
and the proxy here is in the middle, 154 154
and it's the one that's seeing the information, 155 155
and passing those requests back and forth, 156 156
from the external side over to the internal server. 157 157
That way your internal servers only touch your proxy server, 158 158
they do not touch the external client directly, 159 159
and this gives us that single point 160 160
where we can start looking at all of the status messages, 161 161
and start doing statistical analysis 162 162
of those different response codes, 163 163
to be able to identify any suspicious trends, 164 164
or anomalous deviations from the baseline traffic.