1 1 00:00:00,360 --> 00:00:02,190 Audits and assessments. 2 2 00:00:02,190 --> 00:00:04,590 In this lesson, we are going to talk all about audits 3 3 00:00:04,590 --> 00:00:06,720 and assessments and all of the different components 4 4 00:00:06,720 --> 00:00:10,410 that are comprised as part of this auditing and assessments. 5 5 00:00:10,410 --> 00:00:12,870 Now this is going to include things like quality control 6 6 00:00:12,870 --> 00:00:16,590 and quality assurance, verification and validation, 7 7 00:00:16,590 --> 00:00:20,160 assessments and evaluations, audits themselves 8 8 00:00:20,160 --> 00:00:22,710 and schedule reviews and continual improvement. 9 9 00:00:22,710 --> 00:00:23,820 As we go through this lesson, 10 10 00:00:23,820 --> 00:00:25,620 we're going to define each of these terms 11 11 00:00:25,620 --> 00:00:27,330 and talk about how they differ. 12 12 00:00:27,330 --> 00:00:29,190 First, quality control. 13 13 00:00:29,190 --> 00:00:31,500 Now quality control is the process of determining 14 14 00:00:31,500 --> 00:00:34,950 whether a system is free from defects or deficiencies. 15 15 00:00:34,950 --> 00:00:37,410 For example, if you're putting together a car, 16 16 00:00:37,410 --> 00:00:39,330 you want to go back and look at that car 17 17 00:00:39,330 --> 00:00:41,970 and make sure it meets the quality control standards. 18 18 00:00:41,970 --> 00:00:45,030 This is how manufacturing plants work on daily basis. 19 19 00:00:45,030 --> 00:00:46,200 Now working with that 20 20 00:00:46,200 --> 00:00:48,240 is something known as quality assurance. 21 21 00:00:48,240 --> 00:00:50,790 Now quality assurance is the process that analyzes 22 22 00:00:50,790 --> 00:00:52,320 what constitutes quality 23 23 00:00:52,320 --> 00:00:54,240 and how it can be measured and checked. 24 24 00:00:54,240 --> 00:00:56,760 So quality control is the actions that happen 25 25 00:00:56,760 --> 00:00:58,980 at the time of putting something together. 26 26 00:00:58,980 --> 00:01:01,350 So if you're building a new piece of software, for instance, 27 27 00:01:01,350 --> 00:01:04,350 you might go through a quality control of that software. 28 28 00:01:04,350 --> 00:01:06,480 Quality assurance is the program around it 29 29 00:01:06,480 --> 00:01:08,460 that includes things like quality control 30 30 00:01:08,460 --> 00:01:11,310 to make sure we reach a certain level of quality. 31 31 00:01:11,310 --> 00:01:13,680 Now when you're dealing with QC and QA, 32 32 00:01:13,680 --> 00:01:15,300 a lot of times this will take the form 33 33 00:01:15,300 --> 00:01:19,380 of something known as verification and validation or V&V. 34 34 00:01:19,380 --> 00:01:20,400 This is especially true 35 35 00:01:20,400 --> 00:01:22,500 within the software development world. 36 36 00:01:22,500 --> 00:01:23,850 So when we talk about this, 37 37 00:01:23,850 --> 00:01:26,250 we have to first start with verification. 38 38 00:01:26,250 --> 00:01:29,040 Verification is a compliance testing process 39 39 00:01:29,040 --> 00:01:31,710 to ensure that the security system meets the requirements 40 40 00:01:31,710 --> 00:01:33,990 of a framework or regulatory environment 41 41 00:01:33,990 --> 00:01:37,410 or that the product or system meets its design goals. 42 42 00:01:37,410 --> 00:01:40,770 Essentially, verification is a form of quality control 43 43 00:01:40,770 --> 00:01:44,010 but we call it verification inside the digital world. 44 44 00:01:44,010 --> 00:01:45,720 Now when we start dealing with validation, 45 45 00:01:45,720 --> 00:01:47,190 this is the process of determining 46 46 00:01:47,190 --> 00:01:50,310 whether the security system is fit for purpose. 47 47 00:01:50,310 --> 00:01:52,860 Does it actually meet the set designs we have 48 48 00:01:52,860 --> 00:01:54,780 and has it been installed correctly? 49 49 00:01:54,780 --> 00:01:56,730 That's really what we talk about with validation. 50 50 00:01:56,730 --> 00:01:59,250 Verification, we might look at a piece of code and say, 51 51 00:01:59,250 --> 00:02:01,290 yes, this is doing what it's supposed to. 52 52 00:02:01,290 --> 00:02:03,420 Validation is once we install that code, 53 53 00:02:03,420 --> 00:02:04,740 has it been installed properly? 54 54 00:02:04,740 --> 00:02:05,880 Has it been configured properly 55 55 00:02:05,880 --> 00:02:07,740 and does it do what it's meant to do? 56 56 00:02:07,740 --> 00:02:09,390 Is it fit for purpose? 57 57 00:02:09,390 --> 00:02:11,430 Now fit for purpose is kind of a weird term, 58 58 00:02:11,430 --> 00:02:12,960 it comes from the ITIL world. 59 59 00:02:12,960 --> 00:02:14,070 In the ITIL framework, 60 60 00:02:14,070 --> 00:02:17,280 the term fit for purpose is known as utility. 61 61 00:02:17,280 --> 00:02:18,840 And utility really is just defined 62 62 00:02:18,840 --> 00:02:21,930 as meeting the design needs of a software or a service 63 63 00:02:21,930 --> 00:02:23,730 for a particular use case. 64 64 00:02:23,730 --> 00:02:26,190 So if I have something like a word processor, 65 65 00:02:26,190 --> 00:02:28,020 it needs to be able to do those functions 66 66 00:02:28,020 --> 00:02:30,000 that I want a word processor to do. 67 67 00:02:30,000 --> 00:02:32,520 If it can do those things, it has good utility, 68 68 00:02:32,520 --> 00:02:33,660 it is fit for purpose, 69 69 00:02:33,660 --> 00:02:36,540 I can use it for the thing I want to use it for. 70 70 00:02:36,540 --> 00:02:38,130 Now the next two terms we're going to talk about 71 71 00:02:38,130 --> 00:02:40,230 are assessment and evaluations. 72 72 00:02:40,230 --> 00:02:42,030 And these also sound very similar 73 73 00:02:42,030 --> 00:02:44,220 but there is a distinction between them. 74 74 00:02:44,220 --> 00:02:45,510 When we talk about assessment, 75 75 00:02:45,510 --> 00:02:47,670 this is the process of testing the subject 76 76 00:02:47,670 --> 00:02:49,200 against a checklist of requirements 77 77 00:02:49,200 --> 00:02:51,510 in a highly structured way for measurement 78 78 00:02:51,510 --> 00:02:53,460 against an absolute standard. 79 79 00:02:53,460 --> 00:02:55,440 Now that's the key here when we talk about an assessment, 80 80 00:02:55,440 --> 00:02:57,810 it is an absolute standard that we're going against. 81 81 00:02:57,810 --> 00:03:01,140 So often you'll see these used with a checklist approach. 82 82 00:03:01,140 --> 00:03:02,760 Now an evaluation on the other hand 83 83 00:03:02,760 --> 00:03:05,910 is a less methodical process that's used for testing, 84 84 00:03:05,910 --> 00:03:07,890 that is aimed at examining the outcomes 85 85 00:03:07,890 --> 00:03:11,040 or providing usefulness of a subject being tested. 86 86 00:03:11,040 --> 00:03:13,230 So when I start talking about evaluation, 87 87 00:03:13,230 --> 00:03:15,090 evaluation is much more likely to use 88 88 00:03:15,090 --> 00:03:17,640 comparative measurements as more likely to depend 89 89 00:03:17,640 --> 00:03:19,350 on the judgment of the evaluator 90 90 00:03:19,350 --> 00:03:21,510 than on a checklist or a framework. 91 91 00:03:21,510 --> 00:03:23,670 So a lot of times if you're evaluating something, 92 92 00:03:23,670 --> 00:03:25,950 you're saying, "I think this looks good." 93 93 00:03:25,950 --> 00:03:26,880 When you're assessing it, 94 94 00:03:26,880 --> 00:03:28,830 you're going to have it against a checklist. 95 95 00:03:28,830 --> 00:03:30,420 So if you ever bought a used car, 96 96 00:03:30,420 --> 00:03:31,590 one of the things they sell you now 97 97 00:03:31,590 --> 00:03:33,180 with the certified pre-owned used cars 98 98 00:03:33,180 --> 00:03:35,430 is we have this 83-point checklist. 99 99 00:03:35,430 --> 00:03:36,360 That's an assessment. 100 100 00:03:36,360 --> 00:03:38,910 They're going to go down and check all 83 things. 101 101 00:03:38,910 --> 00:03:41,220 Whereas if you took that same car to your local mechanic, 102 102 00:03:41,220 --> 00:03:42,930 he might do an evaluation for you 103 103 00:03:42,930 --> 00:03:45,517 and just look over the car, test a couple of things and say, 104 104 00:03:45,517 --> 00:03:46,597 "Yep, looks good to me. 105 105 00:03:46,597 --> 00:03:48,210 "I think this is a good car." 106 106 00:03:48,210 --> 00:03:49,043 That's the difference 107 107 00:03:49,043 --> 00:03:50,280 between an assessment and an evaluation. 108 108 00:03:50,280 --> 00:03:52,920 The evaluation is much more fluid. 109 109 00:03:52,920 --> 00:03:55,140 It is much more of an opinion-based thing 110 110 00:03:55,140 --> 00:03:57,360 than a checklist based thing. 111 111 00:03:57,360 --> 00:03:59,790 Now the next term we want to talk about is audit. 112 112 00:03:59,790 --> 00:04:02,250 And an audit is a much more rigid process 113 113 00:04:02,250 --> 00:04:04,560 than an assessment or an evaluation. 114 114 00:04:04,560 --> 00:04:06,960 In this case, the auditor is going to compare the organization 115 115 00:04:06,960 --> 00:04:10,110 against a predefined baseline to identify any areas 116 116 00:04:10,110 --> 00:04:12,060 that require remediation. 117 117 00:04:12,060 --> 00:04:13,980 Now audits are generally going to be required 118 118 00:04:13,980 --> 00:04:15,540 in regulated industries. 119 119 00:04:15,540 --> 00:04:17,310 So if you're dealing with the payment card industry 120 120 00:04:17,310 --> 00:04:19,140 or healthcare data processing, 121 121 00:04:19,140 --> 00:04:20,940 all of these things are highly regulated 122 122 00:04:20,940 --> 00:04:22,950 and so you're going to go through either quarterly 123 123 00:04:22,950 --> 00:04:24,600 or annual audits. 124 124 00:04:24,600 --> 00:04:25,890 Now when you go through those, 125 125 00:04:25,890 --> 00:04:28,350 they are going to do it not based on an evaluation, 126 126 00:04:28,350 --> 00:04:29,820 not based on their opinion, 127 127 00:04:29,820 --> 00:04:31,890 they may not even do it based on an assessment 128 128 00:04:31,890 --> 00:04:34,530 or a checklist, they're going to have something much more rigid 129 129 00:04:34,530 --> 00:04:37,410 and they're going to compare you against a known standard. 130 130 00:04:37,410 --> 00:04:39,330 Everybody needs to meet this standard 131 131 00:04:39,330 --> 00:04:42,150 whereas an assessment everyone needs to meet this standard 132 132 00:04:42,150 --> 00:04:43,320 for your organization. 133 133 00:04:43,320 --> 00:04:45,000 It's more tailored to you. 134 134 00:04:45,000 --> 00:04:45,833 So that's one of the difference 135 135 00:04:45,833 --> 00:04:47,880 there between audits and assessments. 136 136 00:04:47,880 --> 00:04:50,220 Now we start talking about scheduled reviews. 137 137 00:04:50,220 --> 00:04:52,350 These are similar to a lessons learned review 138 138 00:04:52,350 --> 00:04:54,690 except that it occurs at a regular interview 139 139 00:04:54,690 --> 00:04:57,030 such as quarterly or annually. 140 140 00:04:57,030 --> 00:04:59,700 Now having a scheduled review is really important 141 141 00:04:59,700 --> 00:05:01,110 and when you go through a scheduled review, 142 142 00:05:01,110 --> 00:05:03,240 there's going to be some things you're going to be thinking about. 143 143 00:05:03,240 --> 00:05:05,520 You're going to be thinking about the previous major incidents 144 144 00:05:05,520 --> 00:05:07,380 you had in the last period. 145 145 00:05:07,380 --> 00:05:09,300 You might think about the trend and analysis 146 146 00:05:09,300 --> 00:05:11,610 of threat intelligence that might affect your company 147 147 00:05:11,610 --> 00:05:13,620 based on the cybersecurity posture. 148 148 00:05:13,620 --> 00:05:15,450 You might think about changes and additions 149 149 00:05:15,450 --> 00:05:17,610 to your security controls or your systems 150 150 00:05:17,610 --> 00:05:19,860 that may increase or decrease risk. 151 151 00:05:19,860 --> 00:05:21,660 And you want to think about the progress you made 152 152 00:05:21,660 --> 00:05:23,550 during the last period as you've adopted 153 153 00:05:23,550 --> 00:05:26,460 or updated different compliance within your frameworks. 154 154 00:05:26,460 --> 00:05:28,110 All of this is things you have to consider 155 155 00:05:28,110 --> 00:05:31,110 as part of your scheduled reviews to see if you're on track 156 156 00:05:31,110 --> 00:05:33,270 to make progress towards your goals. 157 157 00:05:33,270 --> 00:05:35,640 Now speaking of making progress towards your goals, 158 158 00:05:35,640 --> 00:05:36,900 one of the best ways to do this 159 159 00:05:36,900 --> 00:05:38,730 is through continual improvement. 160 160 00:05:38,730 --> 00:05:40,860 Now continual improvement is the process 161 161 00:05:40,860 --> 00:05:44,460 of making small incremental gains to products and services 162 162 00:05:44,460 --> 00:05:46,650 by identifying defects and inefficiencies 163 163 00:05:46,650 --> 00:05:48,270 for further refinement. 164 164 00:05:48,270 --> 00:05:51,030 Now what this means is we are going to do small changes 165 165 00:05:51,030 --> 00:05:52,980 over and over and all those small changes 166 166 00:05:52,980 --> 00:05:54,690 add up to big changes. 167 167 00:05:54,690 --> 00:05:57,120 Now this is important because we can't take on 168 168 00:05:57,120 --> 00:05:59,490 some big project and think we're going to get it all done 169 169 00:05:59,490 --> 00:06:01,440 in a week, it's just not going to happen. 170 170 00:06:01,440 --> 00:06:03,300 But if I could take that big project and say, 171 171 00:06:03,300 --> 00:06:06,150 I have this goal of being here in a year, 172 172 00:06:06,150 --> 00:06:08,580 I can then break that down into what do I need each week 173 173 00:06:08,580 --> 00:06:11,190 to happen to get to that final outcome? 174 174 00:06:11,190 --> 00:06:13,380 And this is the idea of continual improvement. 175 175 00:06:13,380 --> 00:06:15,570 By making small changes over time, 176 176 00:06:15,570 --> 00:06:18,930 we can make big changes over a longer period of time. 177 177 00:06:18,930 --> 00:06:21,060 Now there are many different continual improvement methods 178 178 00:06:21,060 --> 00:06:24,540 out there, including the Six Sigma Method, the Deming Cycle, 179 179 00:06:24,540 --> 00:06:27,450 the Lean Method and even the Seven Step Improvement Method 180 180 00:06:27,450 --> 00:06:28,650 within ITIL. 181 181 00:06:28,650 --> 00:06:30,480 For the CYSA exam though, 182 182 00:06:30,480 --> 00:06:33,000 you don't need to know any of these specific methods, 183 183 00:06:33,000 --> 00:06:35,490 just know that it's important to be continually improving 184 184 00:06:35,490 --> 00:06:37,110 in your organization. 185 185 00:06:37,110 --> 00:06:39,480 All of these methods do follow a similar concept 186 186 00:06:39,480 --> 00:06:41,970 of defining your goals, establishing a baseline 187 187 00:06:41,970 --> 00:06:44,280 through measurement, and then analyzing it, 188 188 00:06:44,280 --> 00:06:45,330 putting it into practice 189 189 00:06:45,330 --> 00:06:47,070 with some kind of improvement action, 190 190 00:06:47,070 --> 00:06:49,980 monitoring that action results and then measuring again, 191 191 00:06:49,980 --> 00:06:52,080 making more changes and continuing through this 192 192 00:06:52,080 --> 00:06:54,390 incremental loop over and over and over again 193 193 00:06:54,390 --> 00:06:56,280 until you get to the final place you want to be 194 194 00:06:56,280 --> 00:06:59,040 which ultimately is a great system that works perfectly 195 195 00:06:59,040 --> 00:07:00,390 every single time. 196 196 00:07:00,390 --> 00:07:01,680 Now that's the hope and the dream 197 197 00:07:01,680 --> 00:07:03,540 but a lot of times we never get there 198 198 00:07:03,540 --> 00:07:06,090 but we do get close by doing all these single improvements, 199 199 00:07:06,090 --> 00:07:08,283 over time they add up to big change.