In this lesson,
we're going to talk about logging services,
including what Linux logs are,
where you can find these logs
and how you can interpret them.
Now, security is not just a process
of designing a hardened system,
but you also may be able to identify malicious behavior
or misconfigurations when those things happen,
and the best way to do that
is by generating and maintaining logs of those events
and going through those logs to understand that.
Operating system logs provide a wealth
of diagnostic information about your computer
and Linux is no exception.
Everything from kernel events to user actions
is being logged by Linux.
And this allows you to see almost any action
that's been performed on your servers or workstations.
Now, system logs are a record
of system activities and events
that are tracked and maintained by the syslogd daemon.
Now, system logs are going to use
the syslog standard by default
and this facilitates a centralized logging server
that can then receive and process syslog data
from many different system systems across your network.
This is called remote logging.
Now, the syslog standard also supports local logging
where these logs are going to be stored on the same system
that generates them, but for best security,
you should set up a syslog server
and offload your logs over to that syslog server
for centralization.
Now, in most Linux distributions,
system logs are going to be located by default,
in your /var/log directory.
Inside this directory are the logs themselves
and each file corresponds to a different service,
application or feature of the operating system.
Now, /var/log/syslog for example,
is going to contain all the system events
except for the authentication messages.
Those are going to be kept in /var/log/auth.log,
which is going to contain all your authentication messages,
such as your login successes and failures.
Now, both of these are going to be used primarily
by Debian-based distros.
So if you're using something like Kali Linux
or another Debian-based distro,
that's where you'll find them.
Now, on the other hand,
if you're using Red Hat Enterprise Linux
or CentOS like we are in this class,
you're going to find these messages kept
in the /var/log/messages
for all of your non-critical system events.
And all of your authentication messages
are going to be kept in /var/log/secure,
where all of your login successes
and failures are going to be kept.
Now, in both of these systems,
if you want to find out information about the kernel,
you can look at the kernel log in /var/log/kern.log
and that file is going to have all of your kernel messages,
such as all the output you're going to get
from the DMESG command.
Now, another thing you want to look at,
is for any of your application files.
Most of these are going to be kept in /var/log/
and the name of the application.
For example, if you're looking for logs
about cron or firewalld or mailog,
you can find those in /var/log/firewalld for example.
Now, log rotation is another thing we need to think about.
Log rotation is the practice creating new versions
of a log file to maintain a minimum log file size.
Now, there's a utility to help us with this
and perform automatic rotation of our logs.
It's known as log rotate.
Now, log rotate, when you execute it,
is going to add a 0.1 to the end of the file name
of the current version of the log file.
Previously rotated files are suffixed with 0.2,
0.3 and so on.
Using this automatic rotation,
all versions of a log file will be maintained on that system
until you specify otherwise.
Now, log rotation can actually be configured as well
by changing the behavior of it.
To be able to configure this log rotation behavior,
you're going to set up configuration files
inside the /etc/logrotate.d directory.
When you do this, you'll be able to define the logs
for that file and how large they should be,
what permissions those log files should have
and how many log files should you keep
before deleting them to make space.
The next thing we're going to talk about
is the configuration for the rsyslogd service.
You can find the configuration file
at /etc/rsyslog.conf
and be able to edit this configuration file.
Now, this file is going to determine
how you're going to handle syslog messages
through a variety of rules that you can modify
to suit your needs.
The file takes up a two column format.
The first column lists the message facilities or severities.
The severities are defined in word format
rather than the normal numbers of zero through seven.
The second column is going to define
what action should be taken for those messages
that correspond to that facility or severity.
The next thing we're going to talk about
is the new replacement for syslogd,
which is known as syslog-ng, or next generation.
Now, syslog-ng is a replacement for syslogd
and it offers similar functionality
to the old syslog or the rsyslogd daemon.
Now, syslog-ng does have a different syntax
than the former syslog or rsyslog service,
so keep that in mind when you're configuring it.
Now, another thing to think about
is your centralized administration capabilities
and when you do the syslog ability
by grabbing all of that data,
it really helps centralize your ability
to manage your systems.
This gives you greater control
over your logging and your network
and it really is a great way to do it.
Unfortunately though, syslog is not universally supported
by all platforms.
Windows, for example, by default uses a proprietary format
known as the Windows event log format
to record all of its system messages.
So if you want to facilitate interaction
between syslog and non-syslog platforms,
you're going to have to use third party agents.
Now, an agent is a software program
that acts on behalf of some other program or service.
Essentially, you're going to install a syslog agent
on that platform and that way,
even if it doesn't support that standard natively,
like Windows, you can then still collect that information
and send it back in a syslog format.
Now, which agent you're going to install
is going to depend on the platform you want to monitor,
as well as the feature sets that you're trying to target.
For example, rsyslog and syslog-ng
both require their own agent software
and it's not the same one.
Once that agent is installed though,
you'll be able to configure it to capture messages
in a syslog format and send those messages
over to your centralized syslog server.
Now, another command you need to be aware of
is known as journalctl.
Now, journalctl is going to enable you
to view and query log files
that are created by the journal component
of the systemd suite.
Log information here is going to be collected
and stored via the systemd journald service.
And you can use the journalctl
to print the entire journal log
or you can issue various options to that command
to filter that log in a number of different ways,
such as matching a service name or only printing messages
that match a specified severity level.
The journald service is often used
in conjunction with a traditional syslog daemon,
such as syslogd or rsyslogd.
The settings for journald are configured
in the /etc/systemd/journald.conf file.
To run the journalctl command,
simply type in journalctl,
the options and the matches.
Now, as I said, the journalctl utility provides
a lot of ways for you to query the journald log data.
Some of the most frequently used options
are things like -n and the number of lines.
This option is used to specify the number of lines
of the journal log you want to display.
The -o and the output format option
is going to be used to specify the format of the output.
The -f option is going to be used
to display the most recent journal entries,
so I like to think about it as fresh for F
and then you can continually update
the display with new entries
as they're added to the journal in this interactive mode.
Then we have -p and -p is going to be used
to filter out the log output by severity.
If you use -u, this option is going to be used
to filter journal log output by their specified unit,
such as the name of the service
and the -b [boot ID] option
is going to be used to show log messages
that are only from the current boot session
or from the boot ID that you specified,
since that is an optional characteristic.
Now, in its default configuration,
the systemd journal only stores logs in memory
and these logs are going to be cleared out
on each system reboot.
You can also have those journald logs persist
after a reboot by creating a /var/log/journal directory
and that way they can be stored there.
The systemd service is configured
to automatically maintain logs in this directory
if that directory exists.
So to enable journaling and logging of your journal,
simply create a directory at /var/log/journal.
The next command we're going to talk about is the last command.
Now, the last command displays the running history
of the user login and logout events,
along with the actual time and date.
This also has various options that you can enable
to filter the results, such as filtering by the users
who have logged in through a specific terminal.
For example, last 1 will display
all the details of the users who logged in
using the first terminal known as tty 1.
The last command will also retrieve information
from the /var/log/wtemp file.
Now, to run the last command,
simply type in last and the options you want to use.
Finally, let's talk about the lastlog command.
The lastlog command is similar to the last command,
but instead of listing out the most recent login events,
it's going to list all the users
and the last time they logged in.
This command retrieves information
from the /var/log/lastlog file.
So as you can see, there's lots of different ways
to log information on your Linux system.
It's really important that you have good logging in place,
because if you're the victim
of a data breach or cyber attack,
your instant responders are going to use those logs
to figure out what happened,
how the bad person got into your network
and what kind of things they did
while they were in your network.
So always ensure you have good logging
and remember to set up a centralized administration server
using syslog or something similar
to collect all those logs in one place.
Your instant responders will thank you for it.