Instructor: Reverse engineering. 2 2
What is reverse engineering? 3 3
Well, reverse engineering is the process 4 4
of analyzing the structure of hardware or software 5 5
to reveal more about its functions. 6 6
And in this lesson we're going to focus 7 7
on that software part, 8 8
specifically when we're talking about malware. 9 9
Now, when I was talking about malware 10 10
in the last lesson, we were talking about 11 11
dynamic analysis where we would turn on a sandbox, 12 12
we would install the malware, and let something happen, 13 13
and then we would observe those effects. 14 14
Well, another way to do this is by doing a static analysis 15 15
by reverse engineering something. 16 16
Now a malware reverse engineer can determine who 17 17
actually wrote the code by starting to learn their patterns. 18 18
Most people who code have their own style, 19 19
their own way of writing. 20 20
If you've ever read somebody's handwriting, 21 21
you can look at it and over time 22 22
start learning whose handwriting belongs with which person. 23 23
Well, coders are much the same 24 24
and as you start reading enough of somebody's code, 25 25
you can start seeing exactly their processes 26 26
and you start identifying this piece 27 27
of malware with this particular person. 28 28
And that's one of the benefits 29 29
finding a strong malware reverse engineer 30 30
working on your team if you're doing 31 31
a lot of instant response or you're working 32 32
for a threat intelligence gathering agency. 33 33
Now as you reverse engineering malware, 34 34
some malware is easier to reverse engineer than others. 35 35
For example, if you're using something like Java, 36 36
this can be easily decompiled back 37 37
into the source code that looks like this. 38 38
Once you have the source code, you can then understand 39 39
what's being done inside the malware and then you can 40 40
figure out better ways to block this malware 41 41
and prevent it from doing any damage to your systems. 42 42
Now, unfortunately not all malware 43 43
is easy to read and dissect 44 44
like this Java code. 45 45
A lot of malware writers will actually 46 46
obfuscate their code before it's assembled or compiled 47 47
to prevent analysis by a reverse engineer. 48 48
This makes it convoluted and very non-straightforward 49 49
in the way that somebody could read it 50 50
and this makes your job much harder 51 51
if you're a reverse engineer. 52 52
Now there's a couple of tools 53 53
that will help us with that though. 54 54
For example, the first one is what we talked 55 55
about in the last lesson, which was sandboxing. 56 56
Whatever difficulty you may have when 57 57
you're reverse engineering malware 58 58
if you're doing it in a sandbox environment, 59 59
it's going to have no impact on your production systems. 60 60
This is especially necessary for malware 61 61
when you're trying to detect what is being done 62 62
as it's being deconstructed or run 63 63
doing something in a more dynamic method. 64 64
Now, when you start doing things statically, 65 65
you have to actually get to the code 66 66
and we use a couple of tools to do this. 67 67
The first one is known as a disassembler. 68 68
Now, a disassembler is a computer program that translates 69 69
that machine language into assembly language. 70 70
Now, machine code when we talk about machine language 71 71
is simply the binary code that's executed by your processor. 72 72
Typically it's represented by two hex digits for each byte 73 73
and each byte is eight ones and zeros. 74 74
Now what does this look like? 75 75
Well, here on the screen you can see the memory view 76 76
and I have that six F highlighted in blue. 77 77
That is the first byte that we're looking at 78 78
at that particular memory location. 79 79
That is a two digit hex code that translates to a series 80 80
of ones and zeros, and that tells the computer 81 81
what it should be doing based on that instruction code. 82 82
Now as a human though, that doesn't mean much to me. 83 83
Those codes are very difficult to read 84 84
and if I have a whole screen filled 85 85
with those type of hex digits, it'd be very hard 86 86
for me to figure out exactly what that program's doing. 87 87
Now there are a couple of hex codes 88 88
that are important for you to know though 89 89
and these are known as file signatures or magic numbers. 90 90
Now a magic number is the first two bytes 91 91
of a binary header and it indicates the file type. 92 92
This is basically going to tell you what type of file it is. 93 93
So when I look at a file, like an image file 94 94
and I see dot png at the end, 95 95
that tells me it's a graphics file, 96 96
but an attacker doesn't have to put PNG at the end. 97 97
They can change that to EXE or something else 98 98
and you really won't know what type of file it is. 99 99
If you look at the first two bytes of that file though, 100 100
it will always tell you what that file is 101 101
and that's what we're talking about, this magic number. 102 102
If you want to look these up, 103 103
you can go to file signatures.net 104 104
and there's a whole database for all of them. 105 105
So look at the first two to four hex digits inside the file. 106 106
Look it up at filesignatures.net 107 107
and you'll know exactly what it is 108 108
because even if I change the extension to dot PNG 109 109
but it's an executable, 110 110
it's going to have the correct file signature on it. 111 111
For instance, when you're reading the first two bytes 112 112
of a Windows portable executable file, 113 113
whether it's an EXE, a DLL, 114 114
an SYS, a DRV, or a com file, 115 115
they're always going to start with 116 116
the hex digits of four D five A. 117 117
That says this is a Windows portable executable file. 118 118
So even if I rename that file 119 119
and put JPG at the end or dot DOC, 120 120
it still would execute because of 121 121
that four D five A, and it knows 122 122
that that is a Windows portable executable file. 123 123
Attackers use this against you all the time, 124 124
so this is one of those ones 125 125
that you should know for the real world. 126 126
Now if you translate that into ASCII, that's MZ 127 127
and if you translate this into Base64 encoding, it's TV. 128 128
So anytime you see any of those three codes there 129 129
that tells you this is an executable file. 130 130
Now the next step up from machine code 131 131
is going to be assembly code. 132 132
Where machine code is written 133 133
in ones and zeros or hex digits, 134 134
assembly code is written 135 135
in something that's more human readable. 136 136
Now this is the native processor instruction 137 137
set that's used to implement a program. 138 138
In the old days, we used 139 139
to actually write programs inside assembly, 140 140
but nowadays that is also a little bit hard to do. 141 141
So if we look here at the next screen, 142 142
you can see up at the top 143 143
I've highlighted the first two lines 144 144
of that hex machine code. 145 145
Now, if you see a highlight at the bottom 146 146
in the disassembly section, 147 147
this is that same hex codes from the top. 148 148
Again, not very easy for us to read, 149 149
but if I look over to the right side, 150 150
you will see the assembly code. 151 151
Now, this is more human readable. 152 152
You can read this, I see move, 153 153
which tells me I'm probably moving stuff 154 154
from one register of the memory to another. 155 155
I have push, which says I'm putting something into memory. 156 156
I'm calling, I'm pulling things from memory. 157 157
These aren't necessarily the easiest things to read, 158 158
but it is still something that a human can read. 159 159
And back in the seventies and eighties, 160 160
a lot of our computer programs 161 161
were written straight in assembly. 162 162
Now to get to assembly, we use a decompiler 163 163
and a decompiler is software that translates a binary 164 164
or low level machine language into a higher level language. 165 165
So if I take the binary and I move it to assembly, 166 166
a decompiler can do that or a disassembler can do that. 167 167
Now, once I have that, I can put it through a decompiler 168 168
and get it to a higher level code. 169 169
Now, when I talk about a higher level code, 170 170
what is that really? 171 171
Well, high level code is real or pseudo code 172 172
in human readable form that makes it easier 173 173
to identify functions, variables, programming logic 174 174
and other things like that that are used in the code. 175 175
Now, when I talk about a higher level code, 176 176
I'm talking about things like C 177 177
or basic or Java or something like that. 178 178
Those are all languages that 179 179
if I gave you some script from that, 180 180
you could probably figure out what's going on 181 181
just by reading it 'cause it reads similar to English 182 182
not like assembly, and definitely not like machine language. 183 183
Now, let's take a look here on the screen. 184 184
Here on the screen I have a program called IDA Pro, 185 185
which is a famous decompiler. 186 186
It takes the ones and zeros, 187 187
that binary and hex values, 188 188
translates it into assembly language 189 189
by doing a disassembly function, 190 190
and then it can take that disassembly function and run it 191 191
through a decompiler and come up with this pseudo code. 192 192
Now, this code is not what the original programmer wrote, 193 193
but this is an equivalent program 194 194
that would do the functions 195 195
of what that person wrote. 196 196
And so this will allow us to go through 197 197
as we're doing our reverse engineering 198 198
and understand a little bit more 199 199
than using something like assembly. 200 200
Now, some languages like Java have their own decompilers 201 201
and they can take it from the ones 202 202
and zeros all the way back up 203 203
to something that looks like real Java. 204 204
As you could see here on the screen, 205 205
this is an example of a Java decompiler 206 206
and we've taken this binary class file 207 207
and it's gone through and brought it back 208 208
into the text where it's very easy for you and I to read it. 209 209
Now, why do we care about getting 210 210
all this stuff back to a human readable form? 211 211
Well, there's a couple of reasons. 212 212
One is we can figure out what logic is being used 213 213
inside this malware and what functions it's doing 214 214
and another is to identify strings because 215 215
as a reverse engineer, we want to attempt 216 216
to identify malware by finding strings 217 217
to use for signature-based rule detection. 218 218
Now, by doing that, we can identify 219 219
these different strings from the code, 220 220
we can put that into our rules, 221 221
and then we can flag that and alert 222 222
on that when malware tries to attack us. 223 223
Now, when I talk about a string, 224 224
a string is simply any sequence 225 225
of encoded characters that appears 226 226
within the executable file. 227 227
These strings can be lots of different things. 228 228
You can find things like ASCII sequences 229 229
that contain usernames, or passwords, 230 230
or file names, or function calls, 231 231
or even things like a URL. 232 232
For instance, if a malware contains a string 233 233
with a function called InternetOpenUrl 234 234
and another string that says URL, 235 235
you probably can guess that's trying 236 236
to attempt to download something 237 237
from that web address as part of the malware. 238 238
Maybe this is a dropper, 239 239
it's going to download some second stage malware 240 240
by going to this URL. 241 241
Now, there's a tool that you can use 242 242
to dump these strings very easily 243 243
without having to go through all the reverse engineering. 244 244
It's called strings and the strings tool 245 245
will dump all the strings that are 246 246
over three characters in ASCII or unicode encoding 247 247
to a text file or to your screen 248 248
so you can then look at it and analyze it yourself. 249 249
Now, the one problem when you're using strings is 250 250
that it will find anything that is over three characters, 251 251
which means it can find a lot of garbage too. 252 252
When I run this, when I'm doing malware analysis 253 253
and I'm going to show you a demonstration 254 254
of this later in the course, you will see 255 255
that a lot of the stuff that it outputs 256 256
to us is stuff that just looks like gobbledygook. 257 257
It's something you can't even see, 258 258
but you'll also find some valuable things in there too. 259 259
And so that's why we still 260 260
run strings against malware samples. 261 261
Now, the next tool we want to talk about 262 262
in our reverse engineering of malware is program packers. 263 263
Now, malware can use a program packer, 264 264
which is a method of compression in which 265 265
the executable is mostly compressed 266 266
and the part that isn't compressed basically 267 267
just contains code to decompress the executable. 268 268
Essentially, a pack program is a type 269 269
of self extracting archive. 270 270
Now, there are a couple of reasons 271 271
that malware writers use these pack programs. 272 272
One is that it makes the file size a lot smaller 273 273
so it's easier to transmit this malware. 274 274
And the second is it makes it harder 275 275
for us as reverse engineers to see the files content 276 276
because we have to unpack it first before we can see it. 277 277
This is another way that they do this to obfuscate things. 278 278
Now, some people think, 279 279
well, anything that's packed must be bad, right? 280 280
So I'm just going to block it all. 281 281
Well, not necessarily. 282 282
A lot of organizations who share proprietary software 283 283
will use program packing to deter theft 284 284
of intellectual property and 285 285
copyright violations and piracy. 286 286
So you can't just block every program that's packed 287 287
because it doesn't necessarily mean it's malicious, 288 288
but a lot of them are. 289 289
So you have to think about this as you're going through 290 290
and developing your signatures, 291 291
and you can't just automatically assume 292 292
a pack program equals malware. 293 293
This would lead to a lot of false positives 294 294
and it could have a lot of bad effects 295 295
inside your operations if you 296 296
block everything that's packed. 297 297
So what do you do if you have a piece of packed malware? 298 298
Well, first you're going to unpack it 299 299
before you can analyze it because until it is unpacked, 300 300
packed malware can mask literal strings 301 301
and effectively modify its signatures 302 302
to avoid triggering signature based scanners 303 303
and this is why you're going to have 304 304
to use this as one of your steps. 305 305
So you'll unpack the malware 306 306
and then perform your analysis on it. 307 307
In fact, when I do malware analysis, 308 308
we actually will do the analysis on the pack program first 309 309
and then we will do it again once we unpack the program 310 310
because sometimes we can find different strings in there 311 311
and use those to build our signature rules.