1 1 00:00:00,330 --> 00:00:02,520 Data normalization. 2 2 00:00:02,520 --> 00:00:05,100 Data normalization is really important to us, 3 3 00:00:05,100 --> 00:00:06,900 especially when we're collecting everything 4 4 00:00:06,900 --> 00:00:08,970 into a central repository. 5 5 00:00:08,970 --> 00:00:10,800 When we collect everything into a central repository 6 6 00:00:10,800 --> 00:00:13,740 like a SIEM, we have to realize that our security data 7 7 00:00:13,740 --> 00:00:16,860 comes from numerous sources across our organization 8 8 00:00:16,860 --> 00:00:18,780 and all of these use different formats 9 9 00:00:18,780 --> 00:00:20,670 and different ways of storing data. 10 10 00:00:20,670 --> 00:00:23,310 So what we have to do is we have to normalize that. 11 11 00:00:23,310 --> 00:00:24,780 When we talk about normalization, 12 12 00:00:24,780 --> 00:00:27,540 this is a process where the data is going to be reformatted 13 13 00:00:27,540 --> 00:00:29,940 or restructured to facilitate the scanning 14 14 00:00:29,940 --> 00:00:33,570 and analysis process later on during the intelligence cycle. 15 15 00:00:33,570 --> 00:00:35,430 Now, when you start thinking about your SIEM data 16 16 00:00:35,430 --> 00:00:37,500 where does this data come from? 17 17 00:00:37,500 --> 00:00:39,390 Well, it can come from three different places 18 18 00:00:39,390 --> 00:00:40,740 across our network. 19 19 00:00:40,740 --> 00:00:42,300 For instance, here on the screen, you can see 20 20 00:00:42,300 --> 00:00:45,330 all sorts of different places that data is coming from. 21 21 00:00:45,330 --> 00:00:47,880 We have it coming from a HIDS or an EDR. 22 22 00:00:47,880 --> 00:00:49,260 We have it coming from agents. 23 23 00:00:49,260 --> 00:00:51,240 We have it coming over the syslog protocol. 24 24 00:00:51,240 --> 00:00:53,460 We have it coming from span ports and tap ports. 25 25 00:00:53,460 --> 00:00:56,550 And we aggregate all that information into a SIEM. 26 26 00:00:56,550 --> 00:00:57,660 Now, we're going to break that down 27 27 00:00:57,660 --> 00:00:59,310 as we go through this lesson. 28 28 00:00:59,310 --> 00:01:01,860 First, we have agent-based information. 29 29 00:01:01,860 --> 00:01:04,170 Now, agent-based collection comes from systems 30 30 00:01:04,170 --> 00:01:07,410 like host-based intrusion detection systems, or EDRs. 31 31 00:01:07,410 --> 00:01:09,210 It comes from agents that are actually installed 32 32 00:01:09,210 --> 00:01:10,890 on a host or on a server, 33 33 00:01:10,890 --> 00:01:13,590 and all that data can then be sent back to a SIEM. 34 34 00:01:13,590 --> 00:01:15,360 When dealing with agent-based collection, 35 35 00:01:15,360 --> 00:01:17,970 we have an agent service that's installed on each host 36 36 00:01:17,970 --> 00:01:21,840 or server to log, filter, aggregate, and normalize the data 37 37 00:01:21,840 --> 00:01:24,360 on that host or server before we send it up 38 38 00:01:24,360 --> 00:01:26,820 to the SIEM server for analysis and storage. 39 39 00:01:26,820 --> 00:01:28,500 Now, the second way we can deal with things 40 40 00:01:28,500 --> 00:01:31,140 is when we deal with a listener or a collector. 41 41 00:01:31,140 --> 00:01:32,610 When we have a listener or collector, 42 42 00:01:32,610 --> 00:01:35,790 this is going to send information over the syslog protocol. 43 43 00:01:35,790 --> 00:01:36,810 And as you can see here, 44 44 00:01:36,810 --> 00:01:40,320 we have syslog being used inside our extranet and our VPN 45 45 00:01:40,320 --> 00:01:42,570 to send that data back to the SIEM server. 46 46 00:01:42,570 --> 00:01:44,490 Now, as we look at this, a listener/collector 47 47 00:01:44,490 --> 00:01:47,190 is going to be a host that's configured to push updates 48 48 00:01:47,190 --> 00:01:51,630 to the SIEM server using a protocol like syslog or SNMP. 49 49 00:01:51,630 --> 00:01:54,450 It can use either of these protocols or another protocol. 50 50 00:01:54,450 --> 00:01:57,090 Either way, it's still considered a listener and collector. 51 51 00:01:57,090 --> 00:01:59,970 In my diagram I just showed you, we were using syslog. 52 52 00:01:59,970 --> 00:02:02,100 Now, the next one we have is sensors. 53 53 00:02:02,100 --> 00:02:03,690 And here you see I have those TAPs 54 54 00:02:03,690 --> 00:02:05,940 and the SPAN ports across the switch. 55 55 00:02:05,940 --> 00:02:07,713 That's going to allow me to collect network data. 56 56 00:02:07,713 --> 00:02:09,510 Now, when I'm dealing with the sensor, 57 57 00:02:09,510 --> 00:02:11,820 this is going to allow my SIEM to collect packet capture 58 58 00:02:11,820 --> 00:02:14,040 and traffic flow data from different sniffers 59 59 00:02:14,040 --> 00:02:16,800 and sensors that are positioned across your network. 60 60 00:02:16,800 --> 00:02:18,690 And by doing this, we get a good mixture 61 61 00:02:18,690 --> 00:02:21,510 across all of our different devices because now we have 62 62 00:02:21,510 --> 00:02:23,550 all of these things working together 63 63 00:02:23,550 --> 00:02:26,160 pulling data from hosts, from network equipment, 64 64 00:02:26,160 --> 00:02:28,350 from packet capture, from servers, 65 65 00:02:28,350 --> 00:02:29,580 and it brings all that data 66 66 00:02:29,580 --> 00:02:31,920 to the centralized server where it's aggregated. 67 67 00:02:31,920 --> 00:02:33,660 That gives me all the data aggregated 68 68 00:02:33,660 --> 00:02:34,920 so I can look through it. 69 69 00:02:34,920 --> 00:02:37,050 This data is aggregated across the network 70 70 00:02:37,050 --> 00:02:39,960 from multiple sources in multiple formats. 71 71 00:02:39,960 --> 00:02:41,970 Now, because of that, that does lead to a problem though 72 72 00:02:41,970 --> 00:02:43,170 because it does come to us 73 73 00:02:43,170 --> 00:02:45,420 in all those different multiple formats. 74 74 00:02:45,420 --> 00:02:48,330 Now, we have formats like proprietary binary formats, 75 75 00:02:48,330 --> 00:02:51,450 tab-separated formats, comma-separated values, 76 76 00:02:51,450 --> 00:02:55,650 database log storage, syslog format, SNMP format, 77 77 00:02:55,650 --> 00:02:59,040 XML format, JSON format, or text-based format. 78 78 00:02:59,040 --> 00:03:01,410 All sorts of data from all different sensors 79 79 00:03:01,410 --> 00:03:03,090 and systems are all coming 80 80 00:03:03,090 --> 00:03:05,580 to this one SIEM to aggregate it all. 81 81 00:03:05,580 --> 00:03:08,340 So we need to parse it and normalize it. 82 82 00:03:08,340 --> 00:03:11,100 Parsing and normalization is used to interpret the data, 83 83 00:03:11,100 --> 00:03:12,480 which is what we call parsing, 84 84 00:03:12,480 --> 00:03:15,000 from all these different formats, and then standardize them 85 85 00:03:15,000 --> 00:03:18,180 into a single format for later analysis and processing. 86 86 00:03:18,180 --> 00:03:20,610 And this is one of the major functions of your SIEM 87 87 00:03:20,610 --> 00:03:22,260 in addition to just collecting all that data 88 88 00:03:22,260 --> 00:03:25,050 is making sure it's all in a format that we can look at. 89 89 00:03:25,050 --> 00:03:26,280 Now, the way it does this is 90 90 00:03:26,280 --> 00:03:28,350 by using connectors or plug-ins. 91 91 00:03:28,350 --> 00:03:30,510 Now, a connector or plug-in is a piece of software 92 92 00:03:30,510 --> 00:03:32,010 that's designed to provide parsing 93 93 00:03:32,010 --> 00:03:35,220 and normalization functions to a particular SIEM. 94 94 00:03:35,220 --> 00:03:37,230 As I said previously when talking about Splunk, 95 95 00:03:37,230 --> 00:03:39,540 there are tons of different connectors 96 96 00:03:39,540 --> 00:03:41,400 that are built into the Splunk framework, 97 97 00:03:41,400 --> 00:03:44,130 so you can pretty much get data from everything. 98 98 00:03:44,130 --> 00:03:45,990 And depending on the system you're dealing with, 99 99 00:03:45,990 --> 00:03:47,820 that can be one of your deciding factors 100 100 00:03:47,820 --> 00:03:49,440 on which SIEM you're going to use. 101 101 00:03:49,440 --> 00:03:52,230 Which one has the widest variety of plug-ins and connectors 102 102 00:03:52,230 --> 00:03:53,730 that will work with all the other systems 103 103 00:03:53,730 --> 00:03:55,530 you have on your network? 104 104 00:03:55,530 --> 00:03:57,270 Now, the last problem we have to deal with 105 105 00:03:57,270 --> 00:03:59,220 is what we call synchronization. 106 106 00:03:59,220 --> 00:04:00,210 So now that we've figured out 107 107 00:04:00,210 --> 00:04:02,460 the problem of parsing and normalization, 108 108 00:04:02,460 --> 00:04:04,800 there is one more problem that we have to solve, 109 109 00:04:04,800 --> 00:04:06,330 and that's time. 110 110 00:04:06,330 --> 00:04:09,000 There is time coming from all these different systems. 111 111 00:04:09,000 --> 00:04:10,680 And depending on your organization, 112 112 00:04:10,680 --> 00:04:13,380 you may not just be located in one time zone. 113 113 00:04:13,380 --> 00:04:14,670 For instance, in my company, 114 114 00:04:14,670 --> 00:04:17,220 we operate in four different time zones. 115 115 00:04:17,220 --> 00:04:19,410 We operate in the east coast of the United States, 116 116 00:04:19,410 --> 00:04:22,050 we operate in Puerto Rico, which is Atlantic Standard Time, 117 117 00:04:22,050 --> 00:04:23,610 we operate in the Philippines, 118 118 00:04:23,610 --> 00:04:25,500 and then we also operate in Italy. 119 119 00:04:25,500 --> 00:04:27,330 So right now we have four different time zones 120 120 00:04:27,330 --> 00:04:29,790 that we are synchronizing across our entire staff 121 121 00:04:29,790 --> 00:04:31,860 to make sure we're all working together. 122 122 00:04:31,860 --> 00:04:33,120 Now, that's not such a big deal 123 123 00:04:33,120 --> 00:04:34,050 when we're working together. 124 124 00:04:34,050 --> 00:04:36,810 We can send files asynchronously, not a big deal. 125 125 00:04:36,810 --> 00:04:39,510 I sent an email, somebody will work on it later today. 126 126 00:04:39,510 --> 00:04:41,160 But it is a big issue 127 127 00:04:41,160 --> 00:04:43,170 if we're trying to synchronize our logs 128 128 00:04:43,170 --> 00:04:44,670 because all of these different systems 129 129 00:04:44,670 --> 00:04:47,100 are having data coming in from different time zones 130 130 00:04:47,100 --> 00:04:49,350 and that can become a big challenge for us. 131 131 00:04:49,350 --> 00:04:51,450 So especially if we're trying to correlate an event 132 132 00:04:51,450 --> 00:04:52,890 because we had an intruder come in 133 133 00:04:52,890 --> 00:04:54,240 or some kind of a data breach 134 134 00:04:54,240 --> 00:04:55,860 and we need to reconstruct a timeline, 135 135 00:04:55,860 --> 00:04:57,840 this can become really difficult 136 136 00:04:57,840 --> 00:05:00,660 without synchronizing all of our dates and times. 137 137 00:05:00,660 --> 00:05:02,940 So one of the things I like to do with my SIEMS 138 138 00:05:02,940 --> 00:05:06,240 is I like to use a standard format for all of our time, 139 139 00:05:06,240 --> 00:05:08,220 and every system is going to work together 140 140 00:05:08,220 --> 00:05:09,780 inside that time format. 141 141 00:05:09,780 --> 00:05:11,790 And that way we all have the same time 142 142 00:05:11,790 --> 00:05:13,890 regardless of where we are in the world. 143 143 00:05:13,890 --> 00:05:15,300 Now, that doesn't mean we all decide 144 144 00:05:15,300 --> 00:05:17,040 to use Eastern Standard Time. 145 145 00:05:17,040 --> 00:05:20,100 No, instead, we like to use Coordinated Universal Time, 146 146 00:05:20,100 --> 00:05:21,750 which is UTC. 147 147 00:05:21,750 --> 00:05:23,640 Now, let's take a look at a couple of times real quick 148 148 00:05:23,640 --> 00:05:26,070 and see if we can understand the problem a little better. 149 149 00:05:26,070 --> 00:05:28,050 First, what time is it? 150 150 00:05:28,050 --> 00:05:30,660 Well, this is something in UTC. 151 151 00:05:30,660 --> 00:05:32,700 It very clearly tells me what time it is. 152 152 00:05:32,700 --> 00:05:35,430 It is January 1st, 2021 153 153 00:05:35,430 --> 00:05:39,420 at 00:00 and zero one seconds. 154 154 00:05:39,420 --> 00:05:43,410 It is one second after midnight in UTC. 155 155 00:05:43,410 --> 00:05:45,072 Now what time is this one? 156 156 00:05:45,072 --> 00:05:45,905 2020-12-31 157 157 00:05:47,664 --> 00:05:49,073 158 19:00.01-05:00. 158 159 00:05:52,710 --> 00:05:54,720 Well, this is based on the Eastern time zone 159 160 00:05:54,720 --> 00:05:56,430 of the United States, which includes places 160 161 00:05:56,430 --> 00:05:59,760 like Miami, Florida, Washington, D.C., and New York. 161 162 00:05:59,760 --> 00:06:02,010 Essentially, this is actually the exact same time 162 163 00:06:02,010 --> 00:06:03,390 as the one I just showed you. 163 164 00:06:03,390 --> 00:06:05,220 The difference is it's minus five 164 165 00:06:05,220 --> 00:06:08,250 because it's five time zones to the left of UTC. 165 166 00:06:08,250 --> 00:06:10,110 So in this case, we're actually talking about 166 167 00:06:10,110 --> 00:06:13,920 one second past 7:00 p.m. on New Year's Eve. 167 168 00:06:13,920 --> 00:06:16,560 This is the exact same time as we talked about 168 169 00:06:16,560 --> 00:06:19,080 with UTC being one second after midnight 169 170 00:06:19,080 --> 00:06:21,750 because there's a five-hour difference between the two. 170 171 00:06:21,750 --> 00:06:22,800 And so when we do this, 171 172 00:06:22,800 --> 00:06:25,770 what we like to do is coordinate all these by using UTC. 172 173 00:06:25,770 --> 00:06:27,870 And by using Coordinated Universal Time, 173 174 00:06:27,870 --> 00:06:31,500 this uses a time standard as opposed to a time zone. 174 175 00:06:31,500 --> 00:06:34,680 Everything can be referenced back to UTC very easily 175 176 00:06:34,680 --> 00:06:36,570 and so most of the time you'll see your servers 176 177 00:06:36,570 --> 00:06:39,210 are going to be listed in UTC for their times. 177 178 00:06:39,210 --> 00:06:41,010 So now that we've solved the problem 178 179 00:06:41,010 --> 00:06:43,530 of big data coming to us in all sorts of format 179 180 00:06:43,530 --> 00:06:45,930 by parsing it, we've normalized that data, 180 181 00:06:45,930 --> 00:06:48,780 it's all stored in the right time using UTC, 181 182 00:06:48,780 --> 00:06:50,910 we have one more problem we have to solve. 182 183 00:06:50,910 --> 00:06:52,680 And that's secure logging. 183 184 00:06:52,680 --> 00:06:54,120 We have all this log data 184 185 00:06:54,120 --> 00:06:56,850 and logs contain a lot of good information, 185 186 00:06:56,850 --> 00:07:00,450 but it is information that needs to be secured as well. 186 187 00:07:00,450 --> 00:07:01,620 Now, in a large organization, 187 188 00:07:01,620 --> 00:07:03,570 you can generate a lot of data, 188 189 00:07:03,570 --> 00:07:05,610 in fact, gigabytes or terabytes 189 190 00:07:05,610 --> 00:07:08,370 of log data every single hour. 190 191 00:07:08,370 --> 00:07:10,800 Now, all that data has to be stored someplace 191 192 00:07:10,800 --> 00:07:13,260 and that storage needs to be secure. 192 193 00:07:13,260 --> 00:07:14,880 And so when you're storing your log data, 193 194 00:07:14,880 --> 00:07:16,290 you have to make sure you're securing it 194 195 00:07:16,290 --> 00:07:18,960 using the principles of confidentiality, integrity, 195 196 00:07:18,960 --> 00:07:20,130 and availability. 196 197 00:07:20,130 --> 00:07:21,930 For confidentiality, you want to make sure 197 198 00:07:21,930 --> 00:07:23,010 you're encrypting that data 198 199 00:07:23,010 --> 00:07:25,170 when you're storing it on the SIEM server. 199 200 00:07:25,170 --> 00:07:26,340 For integrity, you want to make sure 200 201 00:07:26,340 --> 00:07:27,930 you're doing hashes to make sure the data 201 202 00:07:27,930 --> 00:07:29,970 isn't being modified or changed. 202 203 00:07:29,970 --> 00:07:32,010 Availability, you want to make sure you're doing backups 203 204 00:07:32,010 --> 00:07:33,360 and you have redundancy. 204 205 00:07:33,360 --> 00:07:35,760 All the things you think about when you deal with CIA, 205 206 00:07:35,760 --> 00:07:38,360 you need to consider that for your log data as well.