1 1 00:00:00,811 --> 00:00:02,430 URL analysis. 2 2 00:00:02,430 --> 00:00:03,840 Now, at this point in your career, 3 3 00:00:03,840 --> 00:00:05,803 you should know what a URL is. 4 4 00:00:05,803 --> 00:00:08,460 A URL is a uniform resource locator. 5 5 00:00:08,460 --> 00:00:09,990 It's that little address that you type 6 6 00:00:09,990 --> 00:00:11,250 at the top of your web browser 7 7 00:00:11,250 --> 00:00:13,299 to get to diontraining.com 8 8 00:00:13,299 --> 00:00:14,790 or comptia.org 9 9 00:00:14,790 --> 00:00:16,770 or whatever other website you want to go to. 10 10 00:00:16,770 --> 00:00:19,380 Now, a major portion of a cybersecurity analyst job 11 11 00:00:19,380 --> 00:00:20,910 is looking at different URLs 12 12 00:00:20,910 --> 00:00:23,220 in a proxy log or other security logs 13 13 00:00:23,220 --> 00:00:25,440 to determine what websites have been visited 14 14 00:00:25,440 --> 00:00:28,500 and what type of data has been passed to those websites. 15 15 00:00:28,500 --> 00:00:29,970 Now URLs have the ability 16 16 00:00:29,970 --> 00:00:32,340 to encode some action or some data, 17 17 00:00:32,340 --> 00:00:33,480 and that can then be submitted 18 18 00:00:33,480 --> 00:00:35,610 to the server during the request. 19 19 00:00:35,610 --> 00:00:37,560 When you conduct a URL analysis, 20 20 00:00:37,560 --> 00:00:39,060 this is basically just an activity 21 21 00:00:39,060 --> 00:00:40,320 that's performed to identify 22 22 00:00:40,320 --> 00:00:42,060 whether a link is already flagged 23 23 00:00:42,060 --> 00:00:43,950 on an existing reputation list, 24 24 00:00:43,950 --> 00:00:44,820 and if not, 25 25 00:00:44,820 --> 00:00:46,560 to identify what malicious script 26 26 00:00:46,560 --> 00:00:48,720 or activity might be coded within it. 27 27 00:00:48,720 --> 00:00:49,560 Now, this is important 28 28 00:00:49,560 --> 00:00:50,393 because there's some encoding 29 29 00:00:50,393 --> 00:00:52,470 that can happen inside the URL 30 30 00:00:52,470 --> 00:00:53,940 and so you need to use the right tools 31 31 00:00:53,940 --> 00:00:55,620 to be able to resolve that percent encoding 32 32 00:00:55,620 --> 00:00:56,820 that might be there, 33 33 00:00:56,820 --> 00:00:58,380 assess what sort of redirection 34 34 00:00:58,380 --> 00:01:00,180 the URL might be performing, 35 35 00:01:00,180 --> 00:01:01,950 and showing the source code of any scripts 36 36 00:01:01,950 --> 00:01:04,440 that are called by the URL without executing them. 37 37 00:01:04,440 --> 00:01:06,870 And you want to do all of this inside a sandbox environment 38 38 00:01:06,870 --> 00:01:08,670 so you don't infect your own machine. 39 39 00:01:08,670 --> 00:01:09,810 Now let's take a look here 40 40 00:01:09,810 --> 00:01:12,420 at a simple URL and I say simple 41 41 00:01:12,420 --> 00:01:13,530 but it's not really that simple. 42 42 00:01:13,530 --> 00:01:15,480 You can see how long it is here on the screen. 43 43 00:01:15,480 --> 00:01:20,367 It's diontraining.com/upload.php?post= 44 44 00:01:20,367 --> 00:01:21,900 and then this long thing. 45 45 00:01:21,900 --> 00:01:23,280 Now, as we go through this lesson, 46 46 00:01:23,280 --> 00:01:25,920 we're going to come back to this URL quite a few times, 47 47 00:01:25,920 --> 00:01:28,050 but first, before we start analyzing it, 48 48 00:01:28,050 --> 00:01:31,320 we really have to understand some basics of URL analysis. 49 49 00:01:31,320 --> 00:01:33,510 That way we can better understand this URL 50 50 00:01:33,510 --> 00:01:35,460 and see if it's really good or bad 51 51 00:01:35,460 --> 00:01:37,680 or if there's some kind of malicious thing going on. 52 52 00:01:37,680 --> 00:01:38,610 Now there are three things 53 53 00:01:38,610 --> 00:01:39,900 we're going to talk about in this lesson. 54 54 00:01:39,900 --> 00:01:42,060 We're going to talk about HTTP methods, 55 55 00:01:42,060 --> 00:01:44,370 we're going to talk about HTTP response codes 56 56 00:01:44,370 --> 00:01:46,380 and we're going to talk about percent encoding. 57 57 00:01:46,380 --> 00:01:48,810 First, what is an HTTP method? 58 58 00:01:48,810 --> 00:01:51,180 Well, this is a set of request methods 59 59 00:01:51,180 --> 00:01:52,710 that indicates the desired action 60 60 00:01:52,710 --> 00:01:55,350 to be performed for a given resource. 61 61 00:01:55,350 --> 00:01:57,000 Now, when I talk about a resource, 62 62 00:01:57,000 --> 00:01:58,050 essentially think about that 63 63 00:01:58,050 --> 00:01:59,520 as something on the server. 64 64 00:01:59,520 --> 00:02:01,710 Could be a webpage, an image, a file 65 65 00:02:01,710 --> 00:02:03,060 or something like that. 66 66 00:02:03,060 --> 00:02:05,370 Now a request is going to contain a method, 67 67 00:02:05,370 --> 00:02:07,470 a resource, a version number, 68 68 00:02:07,470 --> 00:02:09,990 the header and the body of the request. 69 69 00:02:09,990 --> 00:02:11,790 We put all that information together 70 70 00:02:11,790 --> 00:02:14,100 when we send something out through a URL. 71 71 00:02:14,100 --> 00:02:16,170 So when you go to diontraining.com, 72 72 00:02:16,170 --> 00:02:18,780 you are actually sending an HTTP method 73 73 00:02:18,780 --> 00:02:21,930 saying please, go to this website, get the page, 74 74 00:02:21,930 --> 00:02:22,890 and send it back to me 75 75 00:02:22,890 --> 00:02:24,390 with the body and the header 76 76 00:02:24,390 --> 00:02:26,250 and that's what our server does. 77 77 00:02:26,250 --> 00:02:27,630 Now there are many different methods 78 78 00:02:27,630 --> 00:02:28,740 that we're going to talk about here. 79 79 00:02:28,740 --> 00:02:30,510 The first one is get. 80 80 00:02:30,510 --> 00:02:31,920 A get is the principle method 81 81 00:02:31,920 --> 00:02:34,470 that's used when you do an HTTP request 82 82 00:02:34,470 --> 00:02:36,270 and it's used to retrieve a resource. 83 83 00:02:36,270 --> 00:02:38,340 So again, when you go to diontraining.com, 84 84 00:02:38,340 --> 00:02:40,680 you are using a get saying, get me that page, 85 85 00:02:40,680 --> 00:02:43,440 get me the index page of that website. 86 86 00:02:43,440 --> 00:02:44,730 The second one we have is a post 87 87 00:02:44,730 --> 00:02:45,600 and this is what you saw 88 88 00:02:45,600 --> 00:02:47,580 in the URL I showed you earlier. 89 89 00:02:47,580 --> 00:02:49,590 A post is used to send data to the server 90 90 00:02:49,590 --> 00:02:52,410 for processing by the requested resource. 91 91 00:02:52,410 --> 00:02:53,580 So in the example I had, 92 92 00:02:53,580 --> 00:02:55,950 it was the upload.php file 93 93 00:02:55,950 --> 00:02:57,270 and the action was post. 94 94 00:02:57,270 --> 00:03:00,210 So we were sending data to the upload.php file 95 95 00:03:00,210 --> 00:03:01,230 which is a script 96 96 00:03:01,230 --> 00:03:03,300 that would then be run on that server. 97 97 00:03:03,300 --> 00:03:05,040 Next we have the put method 98 98 00:03:05,040 --> 00:03:06,270 and put is going to create 99 99 00:03:06,270 --> 00:03:08,250 or replace the requested resource. 100 100 00:03:08,250 --> 00:03:09,960 So if I put a file on the server, 101 101 00:03:09,960 --> 00:03:11,070 I'm going to upload it there 102 102 00:03:11,070 --> 00:03:13,200 and overwrite what's existing. 103 103 00:03:13,200 --> 00:03:14,730 The next one we have is delete. 104 104 00:03:14,730 --> 00:03:16,830 This is used to remove the required resource. 105 105 00:03:16,830 --> 00:03:18,240 So if I wanted to delete a file, 106 106 00:03:18,240 --> 00:03:20,400 I could use the delete command. 107 107 00:03:20,400 --> 00:03:21,750 Then we have head. 108 108 00:03:21,750 --> 00:03:24,060 Head is going to retrieve the headers for the resource only 109 109 00:03:24,060 --> 00:03:25,740 and it's going to ignore the body. 110 110 00:03:25,740 --> 00:03:27,810 So if I send out the head command to the server, 111 111 00:03:27,810 --> 00:03:32,310 I'm only going to get the head portion of the HTTP request. 112 112 00:03:32,310 --> 00:03:33,270 This is often used 113 113 00:03:33,270 --> 00:03:34,410 as a penetration tester 114 114 00:03:34,410 --> 00:03:35,880 when you're doing a banner grab 115 115 00:03:35,880 --> 00:03:37,440 because all you want to know is information 116 116 00:03:37,440 --> 00:03:39,210 about the server and about the page, 117 117 00:03:39,210 --> 00:03:41,100 like the title of the page. 118 118 00:03:41,100 --> 00:03:44,010 Now data can be submitted via a URL 119 119 00:03:44,010 --> 00:03:45,000 and it's going to be deliminated 120 120 00:03:45,000 --> 00:03:46,620 by the question mark character. 121 121 00:03:46,620 --> 00:03:49,830 You saw earlier when I had my upload.php, 122 122 00:03:49,830 --> 00:03:51,180 there was a question mark there 123 123 00:03:51,180 --> 00:03:53,206 that then said post= 124 124 00:03:53,206 --> 00:03:55,470 and that was saying, for this resource, 125 125 00:03:55,470 --> 00:03:58,230 I want to do this next thing, I want to post. 126 126 00:03:58,230 --> 00:04:01,020 And that's the idea here of submitting data via a URL 127 127 00:04:01,020 --> 00:04:03,210 using that question mark character. 128 128 00:04:03,210 --> 00:04:04,740 Now you also have query parameters 129 129 00:04:04,740 --> 00:04:06,720 and query parameters are usually going to be formatted 130 130 00:04:06,720 --> 00:04:09,870 as one or more name equals value pairs 131 131 00:04:09,870 --> 00:04:13,080 and they're going to use an ampersand delineating each pair. 132 132 00:04:13,080 --> 00:04:15,360 So if you think back to our Google hacking lesson, 133 133 00:04:15,360 --> 00:04:16,530 we used the ampersand 134 134 00:04:16,530 --> 00:04:18,510 to connect different value pairs together 135 135 00:04:18,510 --> 00:04:21,060 as we sent them to Google to do a specific search. 136 136 00:04:21,060 --> 00:04:23,400 Now the next thing we want to talk about is a hashtag 137 137 00:04:23,400 --> 00:04:25,740 and a hashtag is used to indicate a fragment 138 138 00:04:25,740 --> 00:04:27,060 or an anchor ID 139 139 00:04:27,060 --> 00:04:29,520 and it's not actually processed by your web server. 140 140 00:04:29,520 --> 00:04:31,140 If you've ever coded html, 141 141 00:04:31,140 --> 00:04:32,880 you'll know that you can use a hashtag 142 142 00:04:32,880 --> 00:04:34,980 to say this is a particular part of the page. 143 143 00:04:34,980 --> 00:04:36,720 For instance, if a page is longer 144 144 00:04:36,720 --> 00:04:38,190 than what can fit on the screen, 145 145 00:04:38,190 --> 00:04:40,710 maybe you have a very long book that's in one webpage, 146 146 00:04:40,710 --> 00:04:42,270 I can actually have a hashtag 147 147 00:04:42,270 --> 00:04:45,000 to show where those anchor points are for each chapter. 148 148 00:04:45,000 --> 00:04:46,680 And then when you click a link at the top of the page, 149 149 00:04:46,680 --> 00:04:47,580 it would take you directly 150 150 00:04:47,580 --> 00:04:50,070 to that portion of the page and jump down. 151 151 00:04:50,070 --> 00:04:51,600 Again, it's not processed by the server, 152 152 00:04:51,600 --> 00:04:53,310 but you should be on the lookout for these 153 153 00:04:53,310 --> 00:04:55,200 as part of your URL analysis 154 154 00:04:55,200 --> 00:04:57,390 because sometimes people will misuse that hashtag 155 155 00:04:57,390 --> 00:04:59,490 and inject JavaScript into your page 156 156 00:04:59,490 --> 00:05:01,440 by using that as part of the URL. 157 157 00:05:01,440 --> 00:05:03,930 So let's look at the URL one more time. 158 158 00:05:03,930 --> 00:05:06,330 Here you can see the post method is being used 159 159 00:05:06,330 --> 00:05:07,560 like we talked about. 160 160 00:05:07,560 --> 00:05:09,240 This is going to indicate that we're sending data 161 161 00:05:09,240 --> 00:05:11,730 to the server for processing by the resource. 162 162 00:05:11,730 --> 00:05:14,310 In this case, the upload.php script 163 163 00:05:14,310 --> 00:05:16,950 which is sitting on diontraining.com. 164 164 00:05:16,950 --> 00:05:19,560 Also, you'll see there's that question mark there. 165 165 00:05:19,560 --> 00:05:20,790 That is being used to indicate 166 166 00:05:20,790 --> 00:05:22,740 that data being submitted to that page 167 167 00:05:22,740 --> 00:05:24,210 is going to use the post command 168 168 00:05:24,210 --> 00:05:26,460 and its value is after the equal sign. 169 169 00:05:26,460 --> 00:05:28,470 Now, all that stuff after the equal sign, 170 170 00:05:28,470 --> 00:05:30,060 we're going to talk about that in a little bit 171 171 00:05:30,060 --> 00:05:32,040 but we have some more stuff to cover first. 172 172 00:05:32,040 --> 00:05:35,370 Let's go ahead and talk about HTTP response codes. 173 173 00:05:35,370 --> 00:05:37,890 Now when you deal with an HTTP response code, 174 174 00:05:37,890 --> 00:05:39,510 the header value is going to be returned 175 175 00:05:39,510 --> 00:05:40,620 by the server 176 176 00:05:40,620 --> 00:05:42,780 when the client requests a URL. 177 177 00:05:42,780 --> 00:05:45,000 And these different codes are numerical codes, 178 178 00:05:45,000 --> 00:05:46,320 usually three digits, 179 179 00:05:46,320 --> 00:05:48,150 that are going to tell you some information 180 180 00:05:48,150 --> 00:05:49,830 that the server wants you to know. 181 181 00:05:49,830 --> 00:05:52,470 For example, if you get a code 200, 182 182 00:05:52,470 --> 00:05:55,410 this indicates a successful get or post request, 183 183 00:05:55,410 --> 00:05:57,150 meaning you went and got the page 184 184 00:05:57,150 --> 00:05:58,920 and it came back to you successfully. 185 185 00:05:58,920 --> 00:06:00,990 If you get a response code of 201, 186 186 00:06:00,990 --> 00:06:03,810 this indicates that your put request has been successful 187 187 00:06:03,810 --> 00:06:05,250 in creating a resource. 188 188 00:06:05,250 --> 00:06:07,350 If you have a response code of 300, 189 189 00:06:07,350 --> 00:06:08,850 anything in the 300 range, 190 190 00:06:08,850 --> 00:06:09,960 this is going to indicate 191 191 00:06:09,960 --> 00:06:12,060 a redirect has occurred by the server. 192 192 00:06:12,060 --> 00:06:14,550 For example, there's a 301 redirect that we use 193 193 00:06:14,550 --> 00:06:15,570 when we move a webpage 194 194 00:06:15,570 --> 00:06:17,340 from our site to a different site. 195 195 00:06:17,340 --> 00:06:18,810 So that way we tell Google that we want 196 196 00:06:18,810 --> 00:06:21,060 all that information they know about that old page 197 197 00:06:21,060 --> 00:06:22,530 to be transferred to the new page. 198 198 00:06:22,530 --> 00:06:24,000 That's a 301 redirect. 199 199 00:06:24,000 --> 00:06:26,730 You don't need to know all the codes for the 300 series 200 200 00:06:26,730 --> 00:06:29,580 but you should know that 300 and anything is going to mean 201 201 00:06:29,580 --> 00:06:32,070 that a redirect has occurred by your server. 202 202 00:06:32,070 --> 00:06:34,230 The next series we have is the 400 series 203 203 00:06:34,230 --> 00:06:35,460 and this indicates 204 204 00:06:35,460 --> 00:06:36,450 that you're going to have an error 205 205 00:06:36,450 --> 00:06:37,710 in the client request 206 206 00:06:37,710 --> 00:06:40,080 and there's some specific ones here that you should know. 207 207 00:06:40,080 --> 00:06:42,120 The 400 itself is going to indicate 208 208 00:06:42,120 --> 00:06:44,700 that a request could not be parsed by the server. 209 209 00:06:44,700 --> 00:06:45,810 If you have a 401, 210 210 00:06:45,810 --> 00:06:47,070 this indicates the request 211 211 00:06:47,070 --> 00:06:49,560 did not supply the authentication credentials. 212 212 00:06:49,560 --> 00:06:51,270 If you have 403, this indicates 213 213 00:06:51,270 --> 00:06:53,100 that the request did not have sufficient permissions 214 214 00:06:53,100 --> 00:06:54,480 to access that thing. 215 215 00:06:54,480 --> 00:06:56,730 If you have a 404, this is very common. 216 216 00:06:56,730 --> 00:06:57,618 It indicates 217 217 00:06:57,618 --> 00:07:00,720 that the client has requested a non-existent resource. 218 218 00:07:00,720 --> 00:07:03,150 The next series we want to talk about is the 500 series 219 219 00:07:03,150 --> 00:07:05,970 and a 500 series has any code in that area 220 220 00:07:05,970 --> 00:07:08,670 means that there is some kind of a server side issue. 221 221 00:07:08,670 --> 00:07:09,510 Everything up to this point 222 222 00:07:09,510 --> 00:07:11,520 was really a client not getting what they wanted. 223 223 00:07:11,520 --> 00:07:13,350 Now we have some server side issues. 224 224 00:07:13,350 --> 00:07:15,630 A 500 is going to indicate a general error 225 225 00:07:15,630 --> 00:07:17,670 on the server side of the application. 226 226 00:07:17,670 --> 00:07:20,070 A 502 indicates a bad gateway, 227 227 00:07:20,070 --> 00:07:20,910 and you'll often see this 228 228 00:07:20,910 --> 00:07:22,830 when the server is acting as a proxy. 229 229 00:07:22,830 --> 00:07:24,270 A 503 is going to indicate 230 230 00:07:24,270 --> 00:07:25,710 there's an overloading in the server 231 231 00:07:25,710 --> 00:07:28,590 and that's going to be causing service unavailability. 232 232 00:07:28,590 --> 00:07:29,523 And a 504 indicates 233 233 00:07:29,523 --> 00:07:31,710 that a gateway timeout has occurred 234 234 00:07:31,710 --> 00:07:33,060 and this means there's some kind of an issue 235 235 00:07:33,060 --> 00:07:34,680 with the upstream server. 236 236 00:07:34,680 --> 00:07:35,910 Now for the exam, 237 237 00:07:35,910 --> 00:07:38,670 you do not have to memorize all of these error codes 238 238 00:07:38,670 --> 00:07:40,710 but you should be aware of them in your role 239 239 00:07:40,710 --> 00:07:41,970 as a cybersecurity analyst 240 240 00:07:41,970 --> 00:07:43,320 inside the real world 241 241 00:07:43,320 --> 00:07:44,610 because you're going to be dealing with these, 242 242 00:07:44,610 --> 00:07:46,470 especially if you're dealing with web applications 243 243 00:07:46,470 --> 00:07:48,210 and web application testing. 244 244 00:07:48,210 --> 00:07:49,043 Now, let's go ahead 245 245 00:07:49,043 --> 00:07:51,090 and look at our URL one more time here. 246 246 00:07:51,090 --> 00:07:52,650 Now, does knowing those codes help 247 247 00:07:52,650 --> 00:07:55,170 with identifying anything in this URL? 248 248 00:07:55,170 --> 00:07:56,520 Well, not really, 249 249 00:07:56,520 --> 00:07:58,950 but if we tried opening this URL on your web browser, 250 250 00:07:58,950 --> 00:08:00,570 you would get one of these codes back. 251 251 00:08:00,570 --> 00:08:02,340 You would get a 404 error. 252 252 00:08:02,340 --> 00:08:03,480 Now what does that mean? 253 253 00:08:03,480 --> 00:08:05,280 It means the page doesn't exist. 254 254 00:08:05,280 --> 00:08:06,360 Why is that? 255 255 00:08:06,360 --> 00:08:08,430 Well, it's because the requested resource 256 256 00:08:08,430 --> 00:08:11,400 which was the upload.php, can't be found. 257 257 00:08:11,400 --> 00:08:12,870 Now why can't it be found? 258 258 00:08:12,870 --> 00:08:14,820 Well, because I made up this URL. 259 259 00:08:14,820 --> 00:08:15,870 We don't actually have a page 260 260 00:08:15,870 --> 00:08:18,750 on our website called upload.php on our server 261 261 00:08:18,750 --> 00:08:20,520 but there's still a lot in this URL 262 262 00:08:20,520 --> 00:08:21,353 that we don't know. 263 263 00:08:21,353 --> 00:08:22,980 Everything after the equal sign 264 264 00:08:22,980 --> 00:08:24,630 we haven't even looked at yet. 265 265 00:08:24,630 --> 00:08:25,463 And that's going to bring us 266 266 00:08:25,463 --> 00:08:27,660 to the concept of percent encoding. 267 267 00:08:27,660 --> 00:08:29,700 Now, percent encoding is a mechanism 268 268 00:08:29,700 --> 00:08:31,290 to encode eight bit characters 269 269 00:08:31,290 --> 00:08:32,610 that have a specific meaning 270 270 00:08:32,610 --> 00:08:34,560 in the context of a URL. 271 271 00:08:34,560 --> 00:08:37,200 This is also known as URL encoding. 272 272 00:08:37,200 --> 00:08:39,480 Now, a URL can contain unreserved 273 273 00:08:39,480 --> 00:08:42,420 and reserved characters from the ASCII set. 274 274 00:08:42,420 --> 00:08:45,090 Unreserved characters are things like letters and numbers, 275 275 00:08:45,090 --> 00:08:47,310 A through Z, zero through nine. 276 276 00:08:47,310 --> 00:08:49,290 It's the dash, the period, 277 277 00:08:49,290 --> 00:08:51,450 the underscore, and the tilde. 278 278 00:08:51,450 --> 00:08:53,520 Now we have some reserved characters as well. 279 279 00:08:53,520 --> 00:08:55,590 These are special characters like the colon, 280 280 00:08:55,590 --> 00:08:57,690 the back slash, the question mark, 281 281 00:08:57,690 --> 00:09:00,810 the hashtag, the brackets, the at sign, 282 282 00:09:00,810 --> 00:09:03,570 exclamation, dollar sign, ampersand, 283 283 00:09:03,570 --> 00:09:05,730 single quote, parenthesis, 284 284 00:09:05,730 --> 00:09:10,440 star, plus, comma, semicolon and equals. 285 285 00:09:10,440 --> 00:09:12,510 All of these are reserved characters. 286 286 00:09:12,510 --> 00:09:14,340 You can use these inside of a URL. 287 287 00:09:14,340 --> 00:09:15,900 There's nothing wrong with using them 288 288 00:09:15,900 --> 00:09:17,880 but they do have a special meaning. 289 289 00:09:17,880 --> 00:09:20,310 For instance, we talked about the question mark already. 290 290 00:09:20,310 --> 00:09:21,690 We've talked about the ampersand, 291 291 00:09:21,690 --> 00:09:23,340 we've talked about the equal sign. 292 292 00:09:23,340 --> 00:09:24,720 All of these are special characters 293 293 00:09:24,720 --> 00:09:25,740 or reserved characters 294 294 00:09:25,740 --> 00:09:28,350 that can be used inside your URL. 295 295 00:09:28,350 --> 00:09:31,920 Now, a URL cannot contain any unsafe characters. 296 296 00:09:31,920 --> 00:09:33,660 Now what's an unsafe character? 297 297 00:09:33,660 --> 00:09:36,360 Well, these are things like null string termination, 298 298 00:09:36,360 --> 00:09:38,727 carriage returns, line feed, 299 299 00:09:38,727 --> 00:09:41,670 end of line, tab, space, 300 300 00:09:41,670 --> 00:09:43,320 and things like the forward slash, 301 301 00:09:43,320 --> 00:09:45,960 the less than sign, the greater than sign 302 302 00:09:45,960 --> 00:09:47,520 and the curly brackets. 303 303 00:09:47,520 --> 00:09:49,230 All of these are characters 304 304 00:09:49,230 --> 00:09:50,430 that are considered unsafe 305 305 00:09:50,430 --> 00:09:52,950 and cannot be used in a URL. 306 306 00:09:52,950 --> 00:09:55,560 Now, percent encoding is going to allow a user agent 307 307 00:09:55,560 --> 00:09:58,470 to submit any safe or unsafe character 308 308 00:09:58,470 --> 00:10:00,300 or any binary data for that matter 309 309 00:10:00,300 --> 00:10:02,220 to the server within a URL. 310 310 00:10:02,220 --> 00:10:03,750 So if I want to send a space, 311 311 00:10:03,750 --> 00:10:05,220 I can do that if I encode it 312 312 00:10:05,220 --> 00:10:07,110 with one of these percent encodings. 313 313 00:10:07,110 --> 00:10:08,880 Now here's where I need to bring up the warning. 314 314 00:10:08,880 --> 00:10:10,620 Warning, warning, warning, 315 315 00:10:10,620 --> 00:10:12,870 percent encoding can be misused 316 316 00:10:12,870 --> 00:10:15,450 to obfuscate the nature of a URL. 317 317 00:10:15,450 --> 00:10:17,760 This allows them to encode unreserved characters 318 318 00:10:17,760 --> 00:10:20,250 and submit malicious input as a script 319 319 00:10:20,250 --> 00:10:22,710 or binary or some other method like that 320 320 00:10:22,710 --> 00:10:24,300 to perform directory traversals 321 321 00:10:24,300 --> 00:10:25,860 and other bad things. 322 322 00:10:25,860 --> 00:10:27,870 So when you see percent encoding, 323 323 00:10:27,870 --> 00:10:29,070 you probably should think to yourself, 324 324 00:10:29,070 --> 00:10:30,900 there's probably something being hidden here. 325 325 00:10:30,900 --> 00:10:32,640 I need to look further into this. 326 326 00:10:32,640 --> 00:10:34,320 As an analyst, whenever I see that 327 327 00:10:34,320 --> 00:10:36,420 or on the exam, anytime you see that, 328 328 00:10:36,420 --> 00:10:37,530 you need to have the hair 329 329 00:10:37,530 --> 00:10:39,090 on the back of your neck standing up 330 330 00:10:39,090 --> 00:10:40,980 and going I need to look further into this. 331 331 00:10:40,980 --> 00:10:42,900 There's probably something here. 332 332 00:10:42,900 --> 00:10:44,520 Now let's take a look at percent encoding. 333 333 00:10:44,520 --> 00:10:46,680 And here on the screen, you're going to see a chart. 334 334 00:10:46,680 --> 00:10:48,120 On the left, you'll see the characters 335 335 00:10:48,120 --> 00:10:50,010 and on the right, you'll see the code. 336 336 00:10:50,010 --> 00:10:51,390 Now, I'm not going to read them all to you 337 337 00:10:51,390 --> 00:10:52,890 but you do have them here on the screen 338 338 00:10:52,890 --> 00:10:54,330 and you do have them in your notes 339 339 00:10:54,330 --> 00:10:56,250 in the PDF study guide. 340 340 00:10:56,250 --> 00:10:57,810 Now, the thing I want to point out here is 341 341 00:10:57,810 --> 00:11:00,480 that this is not a list of all of the percent encoding. 342 342 00:11:00,480 --> 00:11:02,280 These are just some of the most common ones 343 343 00:11:02,280 --> 00:11:04,080 that you should know and be familiar with. 344 344 00:11:04,080 --> 00:11:05,850 There are many, many others. 345 345 00:11:05,850 --> 00:11:07,590 When you're an analyst in the real world, 346 346 00:11:07,590 --> 00:11:08,460 I do recommend 347 347 00:11:08,460 --> 00:11:09,720 that you print out a PDF 348 348 00:11:09,720 --> 00:11:11,580 that has the entire list of these 349 349 00:11:11,580 --> 00:11:14,490 because you will use them in your URL analysis. 350 350 00:11:14,490 --> 00:11:17,010 For the exam, you do not need to memorize all of these 351 351 00:11:17,010 --> 00:11:18,360 but you do need to know a couple of them 352 352 00:11:18,360 --> 00:11:20,370 and we'll talk more about that later. 353 353 00:11:20,370 --> 00:11:22,710 Now, some really tricky attackers have started to do 354 354 00:11:22,710 --> 00:11:24,300 what's called double encoding 355 355 00:11:24,300 --> 00:11:25,920 and they can double encode the URL 356 356 00:11:25,920 --> 00:11:27,870 by encoding that percent sign too. 357 357 00:11:27,870 --> 00:11:29,190 So when you're looking at the URL, 358 358 00:11:29,190 --> 00:11:31,410 and so you won't see something like %20 359 359 00:11:31,410 --> 00:11:33,990 even though that thing will be percent encoded. 360 360 00:11:33,990 --> 00:11:36,660 Now for the exam, when you see a URL, 361 361 00:11:36,660 --> 00:11:38,760 if they want to talk about URL analysis 362 362 00:11:38,760 --> 00:11:40,230 and they want to talk about percent encoding, 363 363 00:11:40,230 --> 00:11:41,760 they will use the percent sign. 364 364 00:11:41,760 --> 00:11:43,170 They won't do the double encode, 365 365 00:11:43,170 --> 00:11:44,250 but I want you to be aware of that 366 366 00:11:44,250 --> 00:11:45,240 because in the real world, 367 367 00:11:45,240 --> 00:11:47,100 this is something you have to be aware of. 368 368 00:11:47,100 --> 00:11:48,390 All right, one more time, 369 369 00:11:48,390 --> 00:11:50,700 let's go back to our URL. 370 370 00:11:50,700 --> 00:11:52,890 Here again, we have our URL, 371 371 00:11:52,890 --> 00:11:57,870 diontraining.com/upload.php?post=. 372 372 00:11:57,870 --> 00:11:58,800 We've talked about that. 373 373 00:11:58,800 --> 00:12:00,570 This is the website we're going to, 374 374 00:12:00,570 --> 00:12:01,890 the file on that website 375 375 00:12:01,890 --> 00:12:03,180 and the action we're going to do, 376 376 00:12:03,180 --> 00:12:04,650 we're going to post something. 377 377 00:12:04,650 --> 00:12:07,260 Now, let's get into what we are going to post. 378 378 00:12:07,260 --> 00:12:08,760 So if we look at that URL again, 379 379 00:12:08,760 --> 00:12:10,563 we have %3Cscript%3E. 380 380 00:12:12,885 --> 00:12:16,260 Now what is 3C and 3E in percent encoding? 381 381 00:12:16,260 --> 00:12:17,700 Well, if we go back to our chart, 382 382 00:12:17,700 --> 00:12:18,533 you would know 383 383 00:12:18,533 --> 00:12:21,210 that %3C is the less than sign 384 384 00:12:21,210 --> 00:12:23,850 and %3E is the greater than sign. 385 385 00:12:23,850 --> 00:12:27,450 So essentially this is saying post equals bracket, 386 386 00:12:27,450 --> 00:12:29,790 script bracket in HTML. 387 387 00:12:29,790 --> 00:12:31,350 Now if you know anything about HTML, 388 388 00:12:31,350 --> 00:12:33,000 you know that bracket script bracket 389 389 00:12:33,000 --> 00:12:34,860 means we're calling a JavaScript. 390 390 00:12:34,860 --> 00:12:35,700 And so that is something 391 391 00:12:35,700 --> 00:12:36,630 that we need to think about 392 392 00:12:36,630 --> 00:12:38,130 of what script is being called 393 393 00:12:38,130 --> 00:12:39,690 and that may be something bad. 394 394 00:12:39,690 --> 00:12:41,580 So let's go a little bit further into this. 395 395 00:12:41,580 --> 00:12:46,580 The next area we have, %27http%3A%2F%2Fabc123.com, 396 396 00:12:51,433 --> 00:12:52,266 %2Frat%2Ejs. 397 397 00:12:55,410 --> 00:12:56,880 What does that all translate to? 398 398 00:12:56,880 --> 00:12:59,590 Well %27 is a single quote mark, 399 399 00:12:59,590 --> 00:13:01,420 %3A is a colon, 400 400 00:13:01,420 --> 00:13:04,920 %2F is a backslash, and we have that twice. 401 401 00:13:04,920 --> 00:13:06,990 And then percent 2E is a dot. 402 402 00:13:06,990 --> 00:13:07,823 So if I was going to read that 403 403 00:13:07,823 --> 00:13:08,880 without the percent encoding, 404 404 00:13:08,880 --> 00:13:11,220 I would read that as single quote, 405 405 00:13:11,220 --> 00:13:16,220 http://abc123.com/rat.js. 406 406 00:13:18,570 --> 00:13:21,360 And so now you can see what this URL really is saying. 407 407 00:13:21,360 --> 00:13:24,150 It's saying go to diontraining.com, 408 408 00:13:24,150 --> 00:13:27,780 access the file upload.php and post 409 409 00:13:27,780 --> 00:13:31,350 meaning upload send this file of script, 410 410 00:13:31,350 --> 00:13:34,830 this website address with a link to rat.js. 411 411 00:13:34,830 --> 00:13:36,450 So we're trying to send this file, 412 412 00:13:36,450 --> 00:13:38,490 this JavaScript file to this site, 413 413 00:13:38,490 --> 00:13:39,660 and if it was vulnerable, 414 414 00:13:39,660 --> 00:13:41,370 we can then upload malicious code. 415 415 00:13:41,370 --> 00:13:43,470 That's how we do this URL analysis. 416 416 00:13:43,470 --> 00:13:45,270 And for the exam, you are going to be asked 417 417 00:13:45,270 --> 00:13:47,250 to do some URL analysis. 418 418 00:13:47,250 --> 00:13:48,600 Now let me give you an exam tip 419 419 00:13:48,600 --> 00:13:49,800 for this area of the exam 420 420 00:13:49,800 --> 00:13:51,840 when we talk about URL analysis. 421 421 00:13:51,840 --> 00:13:54,120 You don't need to necessarily remember the values 422 422 00:13:54,120 --> 00:13:55,860 for all those percent encodings 423 423 00:13:55,860 --> 00:13:57,570 but you should know what it is 424 424 00:13:57,570 --> 00:13:58,920 if you see percent encoding. 425 425 00:13:58,920 --> 00:14:01,290 It is some kind of a URL analysis question 426 426 00:14:01,290 --> 00:14:02,430 and you're probably going to be asked 427 427 00:14:02,430 --> 00:14:04,680 to try to figure out what they're hiding in there. 428 428 00:14:04,680 --> 00:14:06,390 Now, they're not going to be very specific 429 429 00:14:06,390 --> 00:14:09,060 in the fact of asking you exactly what's going on, 430 430 00:14:09,060 --> 00:14:10,380 but using this, 431 431 00:14:10,380 --> 00:14:12,570 you should be able to see certain things. 432 432 00:14:12,570 --> 00:14:15,450 For example, are they trying to do an SQL inject? 433 433 00:14:15,450 --> 00:14:17,700 If so, they're probably using that single quote 434 434 00:14:17,700 --> 00:14:19,350 and they're using an equal sign. 435 435 00:14:19,350 --> 00:14:21,000 If they're trying to do something like XML, 436 436 00:14:21,000 --> 00:14:22,680 you'll see some XML being passed. 437 437 00:14:22,680 --> 00:14:24,300 If they're trying to load a JavaScript, 438 438 00:14:24,300 --> 00:14:25,133 they're going to do something like 439 439 00:14:25,133 --> 00:14:26,940 what I just showed you here in the URL. 440 440 00:14:26,940 --> 00:14:28,830 These are the kind of things you'll have to do. 441 441 00:14:28,830 --> 00:14:29,850 Now, what do you do 442 442 00:14:29,850 --> 00:14:31,350 if you don't remember all those codes 443 443 00:14:31,350 --> 00:14:33,060 because that's a lot of percent encoding, 444 444 00:14:33,060 --> 00:14:36,060 all those different codes of remembering %3F 445 445 00:14:36,060 --> 00:14:38,520 and %2E, and things like that. 446 446 00:14:38,520 --> 00:14:39,990 Well, on the exam, 447 447 00:14:39,990 --> 00:14:41,580 it's all multiple choice, right? 448 448 00:14:41,580 --> 00:14:43,020 And so when you get a question, 449 449 00:14:43,020 --> 00:14:46,770 you may not remember what %3C and %3E were 450 450 00:14:46,770 --> 00:14:48,030 because that was the brackets 451 451 00:14:48,030 --> 00:14:49,530 for the script on either side, 452 452 00:14:49,530 --> 00:14:51,600 but you did see the word script. 453 453 00:14:51,600 --> 00:14:52,890 And so if you see the word script, 454 454 00:14:52,890 --> 00:14:53,723 you might think, hey, 455 455 00:14:53,723 --> 00:14:55,440 they might be trying pass an HTML script 456 456 00:14:55,440 --> 00:14:56,550 or a JavaScript there. 457 457 00:14:56,550 --> 00:14:57,450 And as you saw the rest of it, 458 458 00:14:57,450 --> 00:15:00,420 you could read things like abc123.com, 459 459 00:15:00,420 --> 00:15:01,380 which is a web link. 460 460 00:15:01,380 --> 00:15:03,720 And so you're seeing that you're going to Dion Training 461 461 00:15:03,720 --> 00:15:05,130 but now you're sending another web link 462 462 00:15:05,130 --> 00:15:06,570 as part of this URL 463 463 00:15:06,570 --> 00:15:07,710 and that should flag to you 464 464 00:15:07,710 --> 00:15:09,120 that there's something wrong here. 465 465 00:15:09,120 --> 00:15:11,790 So even if you don't remember all of the colons 466 466 00:15:11,790 --> 00:15:13,530 and the back slashes and the dots, 467 467 00:15:13,530 --> 00:15:15,210 you can probably put together most of this 468 468 00:15:15,210 --> 00:15:17,400 based on what you're reading in that URL 469 469 00:15:17,400 --> 00:15:19,560 as you're doing your analysis for the exam. 470 470 00:15:19,560 --> 00:15:20,610 In the real world though, 471 471 00:15:20,610 --> 00:15:22,740 I do recommend having one of these charts handy 472 472 00:15:22,740 --> 00:15:24,930 because then you can read exactly what it is 473 473 00:15:24,930 --> 00:15:26,530 and really put it back together.