1 1 00:00:00,270 --> 00:00:02,040 Software assessments. 2 2 00:00:02,040 --> 00:00:04,680 In this lesson, we're going to talk about the different types 3 3 00:00:04,680 --> 00:00:06,150 of software assessments, 4 4 00:00:06,150 --> 00:00:07,410 because it's really important for you 5 5 00:00:07,410 --> 00:00:09,115 to have a comprehensive testing program 6 6 00:00:09,115 --> 00:00:10,541 that validates the effectiveness 7 7 00:00:10,541 --> 00:00:12,870 of your protection of confidentiality, 8 8 00:00:12,870 --> 00:00:16,290 integrity, and availability within your software. 9 9 00:00:16,290 --> 00:00:18,570 If you're doing any kind of web application development 10 10 00:00:18,570 --> 00:00:21,090 or software development, it is really crucial 11 11 00:00:21,090 --> 00:00:24,030 that you have a comprehensive testing program in place. 12 12 00:00:24,030 --> 00:00:25,830 Now, the things we're going to talk about in this lesson 13 13 00:00:25,830 --> 00:00:28,290 include things like static code analysis, 14 14 00:00:28,290 --> 00:00:32,088 formal verification, user acceptance testing or UAT 15 15 00:00:32,088 --> 00:00:34,650 and security regression testing. 16 16 00:00:34,650 --> 00:00:36,990 Now, when we talk about static code analysis, 17 17 00:00:36,990 --> 00:00:40,110 this is the process of reviewing uncompiled source code 18 18 00:00:40,110 --> 00:00:42,870 either manually or using automated tools. 19 19 00:00:42,870 --> 00:00:44,280 So let's pretend that one 20 20 00:00:44,280 --> 00:00:46,140 of my web developers created a program 21 21 00:00:46,140 --> 00:00:48,690 so you could take quizzes on our website. 22 22 00:00:48,690 --> 00:00:51,930 Now that web application is written in some kind of code, 23 23 00:00:51,930 --> 00:00:54,840 and if somebody wanted to look at that code, line by line, 24 24 00:00:54,840 --> 00:00:58,560 and see how it works, that would be a static code analysis. 25 25 00:00:58,560 --> 00:01:00,390 Now, in addition to doing it manually, 26 26 00:01:00,390 --> 00:01:02,430 we could use automated tools though. 27 27 00:01:02,430 --> 00:01:04,920 Now these automated tools can reveal issues 28 28 00:01:04,920 --> 00:01:07,650 ranging from faulty logic to insecure libraries, 29 29 00:01:07,650 --> 00:01:10,320 before you even start running that application. 30 30 00:01:10,320 --> 00:01:12,210 Now, these automated tools are essentially 31 31 00:01:12,210 --> 00:01:13,740 software that will scan the source code 32 32 00:01:13,740 --> 00:01:15,810 for signatures of known issues. 33 33 00:01:15,810 --> 00:01:17,820 Now, a lot of these use different databases 34 34 00:01:17,820 --> 00:01:18,930 for these known issues. 35 35 00:01:18,930 --> 00:01:21,350 For example, we've mentioned OWASP before. 36 36 00:01:21,350 --> 00:01:23,430 OWASP is a really popular place 37 37 00:01:23,430 --> 00:01:25,410 to find these coding vulnerabilities, 38 38 00:01:25,410 --> 00:01:27,892 and there's a top 10 for application security risks 39 39 00:01:27,892 --> 00:01:29,670 or injection vulnerabilities 40 40 00:01:29,670 --> 00:01:30,540 and you can look at those 41 41 00:01:30,540 --> 00:01:33,330 and load those into your automated tools as well. 42 42 00:01:33,330 --> 00:01:35,280 Now, just because you have an automated tool, 43 43 00:01:35,280 --> 00:01:36,570 doesn't mean you can just scan it 44 44 00:01:36,570 --> 00:01:38,310 with a tool and forget about it. 45 45 00:01:38,310 --> 00:01:40,980 No, these tools are not an adequate substitute 46 46 00:01:40,980 --> 00:01:42,420 for human judgment. 47 47 00:01:42,420 --> 00:01:44,910 Instead, we should use these automated tools 48 48 00:01:44,910 --> 00:01:47,670 to get the first pass through that software code, 49 49 00:01:47,670 --> 00:01:49,590 and then we should put a human on it 50 50 00:01:49,590 --> 00:01:50,940 to look through it as well. 51 51 00:01:50,940 --> 00:01:52,830 By doing this one two punch, 52 52 00:01:52,830 --> 00:01:54,390 you're going to get a lot better results 53 53 00:01:54,390 --> 00:01:56,790 because a lot of times the human will catch things 54 54 00:01:56,790 --> 00:01:57,810 the automated tool won't, 55 55 00:01:57,810 --> 00:02:00,720 and the automated tool will catch things the human won't. 56 56 00:02:00,720 --> 00:02:02,130 Now, when you're doing these things, 57 57 00:02:02,130 --> 00:02:03,832 these are known as code reviews. 58 58 00:02:03,832 --> 00:02:06,480 Now, code reviews are this manual process, 59 59 00:02:06,480 --> 00:02:08,100 and it's a process of peer review 60 60 00:02:08,100 --> 00:02:11,070 of uncompiled source code by other developers. 61 61 00:02:11,070 --> 00:02:14,250 So let's say one of my web developers wrote a program. 62 62 00:02:14,250 --> 00:02:16,620 He emails it to me and I read through it. 63 63 00:02:16,620 --> 00:02:19,080 That would be a peer review code review. 64 64 00:02:19,080 --> 00:02:20,880 Now, there's lots of other types you can do. 65 65 00:02:20,880 --> 00:02:21,713 We can do something known 66 66 00:02:21,713 --> 00:02:23,640 as an over-the-shoulder code review. 67 67 00:02:23,640 --> 00:02:25,890 This is where the person who wrote the code will sit 68 68 00:02:25,890 --> 00:02:28,860 at the keyboard and explain exactly what they did, 69 69 00:02:28,860 --> 00:02:30,180 and then I might stand behind them 70 70 00:02:30,180 --> 00:02:31,595 and look over their shoulder 71 71 00:02:31,595 --> 00:02:34,380 to see the code as they're explaining it. 72 72 00:02:34,380 --> 00:02:37,350 Another way you can do it is by doing paraprogramming. 73 73 00:02:37,350 --> 00:02:39,120 This is more of an interactive way 74 74 00:02:39,120 --> 00:02:41,100 where they write a section and I write a section, 75 75 00:02:41,100 --> 00:02:42,540 and while I'm writing my section, 76 76 00:02:42,540 --> 00:02:43,860 they're looking over my shoulder 77 77 00:02:43,860 --> 00:02:45,090 and when they're writing a section, 78 78 00:02:45,090 --> 00:02:46,470 I'm looking over their shoulder. 79 79 00:02:46,470 --> 00:02:49,260 And so that way we can try to catch mistakes. 80 80 00:02:49,260 --> 00:02:51,960 That oversight allows us to catch different assumptions. 81 81 00:02:51,960 --> 00:02:52,793 For instance, I might go, 82 82 00:02:52,793 --> 00:02:55,290 "Hey why are you using that variable declaration? 83 83 00:02:55,290 --> 00:02:56,610 That doesn't make sense. 84 84 00:02:56,610 --> 00:02:58,080 You're making a faulty assumption," 85 85 00:02:58,080 --> 00:02:59,340 and then I can call them out on that 86 86 00:02:59,340 --> 00:03:01,140 and we can get those things fixed. 87 87 00:03:01,140 --> 00:03:03,343 By doing this, you can bring up level of knowledge as well 88 88 00:03:03,343 --> 00:03:05,310 because a more experienced person can help 89 89 00:03:05,310 --> 00:03:08,400 a junior coder or a junior coder might be able 90 90 00:03:08,400 --> 00:03:10,860 to show a new way of doing things to that senior coder. 91 91 00:03:10,860 --> 00:03:13,186 It does work both ways, and so doing these code reviews 92 92 00:03:13,186 --> 00:03:15,270 and peer programming can really help 93 93 00:03:15,270 --> 00:03:16,830 the security of your programs. 94 94 00:03:16,830 --> 00:03:18,480 Now, another thing we can do is what's called 95 95 00:03:18,480 --> 00:03:20,430 a formal verification method. 96 96 00:03:20,430 --> 00:03:23,190 Now, this is a process of validating software design 97 97 00:03:23,190 --> 00:03:24,630 through mathematical modeling 98 98 00:03:24,630 --> 00:03:26,910 of expected inputs and outputs. 99 99 00:03:26,910 --> 00:03:29,670 Now, why might we use a formal verification method 100 100 00:03:29,670 --> 00:03:31,799 instead of just doing a static code review? 101 101 00:03:31,799 --> 00:03:33,737 Well, a lot of times there are these things 102 102 00:03:33,737 --> 00:03:35,530 known as corner cases. 103 103 00:03:35,530 --> 00:03:37,583 And when you're doing a formal verification method 104 104 00:03:37,583 --> 00:03:39,870 these are used in critical software 105 105 00:03:39,870 --> 00:03:42,270 where corner cases have to be eliminated. 106 106 00:03:42,270 --> 00:03:44,100 Now, what do I mean by a corner case? 107 107 00:03:44,100 --> 00:03:45,592 Well, let's say I had a program 108 108 00:03:45,592 --> 00:03:48,540 that does something like a self-driving car. 109 109 00:03:48,540 --> 00:03:49,590 Well, a self-driving car 110 110 00:03:49,590 --> 00:03:51,000 has a lot of different code in it, 111 111 00:03:51,000 --> 00:03:52,888 and most of the time the self-driving car 112 112 00:03:52,888 --> 00:03:54,990 is going to be driving on a road 113 113 00:03:54,990 --> 00:03:56,280 and it needs to be able to detect objects 114 114 00:03:56,280 --> 00:03:58,920 like a hole in the road or a person on the side of the road 115 115 00:03:58,920 --> 00:04:00,390 or a car in front of them, 116 116 00:04:00,390 --> 00:04:02,220 and they need to be able to see these things using 117 117 00:04:02,220 --> 00:04:05,640 its sensors and its logic to be able to drive the car. 118 118 00:04:05,640 --> 00:04:08,275 Now, a corner case might be what does the car do 119 119 00:04:08,275 --> 00:04:11,550 if that person on the side road jumps out in front of me? 120 120 00:04:11,550 --> 00:04:13,550 Now the car only has two options. 121 121 00:04:13,550 --> 00:04:16,200 It can try to stop, but it won't be able to in time, 122 122 00:04:16,200 --> 00:04:18,870 and it will hit the person, or it can swerve to the left 123 123 00:04:18,870 --> 00:04:20,520 and crash into the other car, 124 124 00:04:20,520 --> 00:04:21,660 or it can swerve to the right 125 125 00:04:21,660 --> 00:04:23,760 and hit that other person on the side of the road. 126 126 00:04:23,760 --> 00:04:25,590 All of these are bad options. 127 127 00:04:25,590 --> 00:04:28,260 In this case, this would be known as corner cases, 128 128 00:04:28,260 --> 00:04:30,540 because these are all bad options that we'd want to avoid, 129 129 00:04:30,540 --> 00:04:32,040 but if you have to choose one of them, 130 130 00:04:32,040 --> 00:04:33,600 which one are you going to choose? 131 131 00:04:33,600 --> 00:04:34,800 And so we have to be able to go through 132 132 00:04:34,800 --> 00:04:37,770 into a formal verification method of all the software, 133 133 00:04:37,770 --> 00:04:39,540 especially in something that's a critical environment, 134 134 00:04:39,540 --> 00:04:42,810 like a car that weighs 2000 or 3000 pounds barreling 135 135 00:04:42,810 --> 00:04:44,580 down the highway at 60 miles an hour. 136 136 00:04:44,580 --> 00:04:47,741 That's a great place for using formal verification methods. 137 137 00:04:47,741 --> 00:04:48,842 Another thing we might use 138 138 00:04:48,842 --> 00:04:50,958 is what's called User Acceptance Testing. 139 139 00:04:50,958 --> 00:04:53,784 Now, user acceptance testing or UAT 140 140 00:04:53,784 --> 00:04:56,160 is beta testing by your end users 141 141 00:04:56,160 --> 00:04:57,810 that proves a program is usable 142 142 00:04:57,810 --> 00:05:00,390 and fit for purpose in real world conditions. 143 143 00:05:00,390 --> 00:05:02,633 So back in the late nineties and early 2000s, 144 144 00:05:02,633 --> 00:05:04,440 I owned a company doing web development. 145 145 00:05:04,440 --> 00:05:06,120 That's where I started out. 146 146 00:05:06,120 --> 00:05:07,890 And when I was doing web development back then, 147 147 00:05:07,890 --> 00:05:10,080 we did a lot of user acceptance testing. 148 148 00:05:10,080 --> 00:05:11,880 So I would sit down with somebody, 149 149 00:05:11,880 --> 00:05:14,066 you would tell me exactly what you want this website to do, 150 150 00:05:14,066 --> 00:05:15,750 you'd tell me all the features you want, 151 151 00:05:15,750 --> 00:05:18,030 what you want the design to look like, all of that stuff. 152 152 00:05:18,030 --> 00:05:19,530 We gather all the requirements, 153 153 00:05:19,530 --> 00:05:22,650 we plan out the design, we code it, and we do our testing. 154 154 00:05:22,650 --> 00:05:24,660 We're going to do code reviews, static code reviews. 155 155 00:05:24,660 --> 00:05:25,997 We're going to do automated code reviews. 156 156 00:05:25,997 --> 00:05:28,170 We might even do formal verification methods 157 157 00:05:28,170 --> 00:05:29,003 if we're doing something for 158 158 00:05:29,003 --> 00:05:30,750 say a bank or something like that. 159 159 00:05:30,750 --> 00:05:31,901 Now, once we do all of that, 160 160 00:05:31,901 --> 00:05:35,160 we now need to get into the user acceptance testing. 161 161 00:05:35,160 --> 00:05:38,070 This is where we're going to put hands-on end users 162 162 00:05:38,070 --> 00:05:39,979 on the system and see how they react. 163 163 00:05:39,979 --> 00:05:42,540 This allows us to gather feedback from those end users, 164 164 00:05:42,540 --> 00:05:44,821 identify any workflow issues we may have overseen 165 165 00:05:44,821 --> 00:05:48,210 and ensure that it meets their particular use case. 166 166 00:05:48,210 --> 00:05:51,600 So if I was building a website for a bank, for instance, 167 167 00:05:51,600 --> 00:05:53,790 I might think that I've coded it exactly right 168 168 00:05:53,790 --> 00:05:55,020 and the bank might have thought 169 169 00:05:55,020 --> 00:05:56,820 that I coded it exactly right. 170 170 00:05:56,820 --> 00:05:58,200 But then we put a user on the system, 171 171 00:05:58,200 --> 00:06:00,090 they go, well this doesn't make sense. 172 172 00:06:00,090 --> 00:06:02,400 When I try to transfer money from one account to another 173 173 00:06:02,400 --> 00:06:04,215 I have to go through 17 clicks to do it. 174 174 00:06:04,215 --> 00:06:05,700 That isn't very good. 175 175 00:06:05,700 --> 00:06:07,740 And so that end user, who's representing the customer 176 176 00:06:07,740 --> 00:06:09,510 might go, "We're not going to accept this. 177 177 00:06:09,510 --> 00:06:10,770 Go back and recode it. 178 178 00:06:10,770 --> 00:06:13,050 Or in version 2.0, you need to fix this." 179 179 00:06:13,050 --> 00:06:15,120 This is the idea of user acceptance testing. 180 180 00:06:15,120 --> 00:06:18,000 This is more about the end functionality and fit. 181 181 00:06:18,000 --> 00:06:19,410 Does it do what it's supposed to do? 182 182 00:06:19,410 --> 00:06:21,375 And is it doing it in a well thought out way 183 183 00:06:21,375 --> 00:06:23,700 that makes it easier for the user to use? 184 184 00:06:23,700 --> 00:06:25,200 The final thing we're going to talk about here 185 185 00:06:25,200 --> 00:06:27,032 is security regression testing. 186 186 00:06:27,032 --> 00:06:28,936 Now, this is the process of checking 187 187 00:06:28,936 --> 00:06:31,260 that updates the code do not compromise 188 188 00:06:31,260 --> 00:06:32,850 the existing security functionality 189 189 00:06:32,850 --> 00:06:35,400 or capability of the code itself. 190 190 00:06:35,400 --> 00:06:37,110 Now, why is this important? 191 191 00:06:37,110 --> 00:06:39,330 Well, because every time you install a patch 192 192 00:06:39,330 --> 00:06:41,307 or an update or some kind of new piece of software, 193 193 00:06:41,307 --> 00:06:43,624 you run the risk of breaking things. 194 194 00:06:43,624 --> 00:06:45,750 And so if I take a new security update, 195 195 00:06:45,750 --> 00:06:48,450 and I install it on my network and I didn't test it first, 196 196 00:06:48,450 --> 00:06:49,361 well guess what, 197 197 00:06:49,361 --> 00:06:50,903 I might have introduced new bugs 198 198 00:06:50,903 --> 00:06:53,010 or I might have broken features. 199 199 00:06:53,010 --> 00:06:55,830 So instead we want to do a security regression testing. 200 200 00:06:55,830 --> 00:06:58,320 We're going to set it up in a lab, we're going to test that thing 201 201 00:06:58,320 --> 00:07:00,030 and make sure we didn't break something else 202 202 00:07:00,030 --> 00:07:01,440 when we fixed it. 203 203 00:07:01,440 --> 00:07:03,180 Security regression testing is going to enable 204 204 00:07:03,180 --> 00:07:04,943 the identification of security mechanisms 205 205 00:07:04,943 --> 00:07:06,540 that have worked before 206 206 00:07:06,540 --> 00:07:09,180 but are now broken after the latest changes. 207 207 00:07:09,180 --> 00:07:10,620 This is a common problem 208 208 00:07:10,620 --> 00:07:13,110 because one of the things we say as a programmer is that 209 209 00:07:13,110 --> 00:07:15,660 I can add one piece of code to fix something, 210 210 00:07:15,660 --> 00:07:18,270 but I might end up breaking three or four other things. 211 211 00:07:18,270 --> 00:07:19,560 Anytime you add a change, 212 212 00:07:19,560 --> 00:07:21,690 there's a possibility something's going to get broken. 213 213 00:07:21,690 --> 00:07:23,280 And so one of the things you always want to go back 214 214 00:07:23,280 --> 00:07:24,600 and check is that you didn't break 215 215 00:07:24,600 --> 00:07:25,806 any of your security mechanisms, 216 216 00:07:25,806 --> 00:07:27,540 because that would be a bad thing for us, 217 217 00:07:27,540 --> 00:07:29,609 especially since we're trying to be cybersecurity experts 218 218 00:07:29,609 --> 00:07:31,593 by being cybersecurity analysts.