Linux systems maintain quite a collection of log files, many of which you are probably rarely tempted to view. Some of these log files are quite valuable, though, and options for exploring them might be more interesting and varied than you imagine. Let's look at some system logs and get a handle on some of the ways in which log data might be easier to probe.\nLog file rotation\nFirst, there's the issue of log rotation. Some Linux log files are \u201crotated.\u201d In other words, the system stores more than one "generation" of these files, mostly to keep them from using too much disk space. The older logs are then compressed but left available for a while. Eventually, the oldest in a series of rotated log files will be automatically deleted in the log rotation process, but you\u2019ll still have access to a number of the older logs so that you can examine log entries that were added in the last few days or weeks when and if you need to look a little further back into some issue you're tracking.\n\nTo get a feel for what types of system information are being saved, simply cd over to the \/var\/log directory and list its contents.\n\/var\/log# lsalternatives.log btmp.1 kern.log.2.gz syslog.3.gzalternatives.log.1 cups kern.log.3.gz syslog.4.gzalternatives.log.2.gz dist-upgrade kern.log.4.gz syslog.5.gzalternatives.log.3.gz dpkg.log lastlog syslog.6.gzalternatives.log.4.gz dpkg.log.1 mail.err syslog.7.gzalternatives.log.5.gz dpkg.log.2.gz mail.err.1 sysstatapport.log dpkg.log.3.gz mail.err.2.gz tallylogapport.log.1 dpkg.log.4.gz mail.err.3.gz ufw.logapt dpkg.log.5.gz mail.err.4.gz ufw.log.1atop faillog mail.log ufw.log.2.gzauth.log fontconfig.log mail.log.1 ufw.log.3.gzauth.log.1 gdm3 mail.log.2.gz ufw.log.4.gzauth.log.2.gz gpu-manager.log mail.log.3.gz unattended-upgradesauth.log.3.gz hp mail.log.4.gz wtmpauth.log.4.gz installer speech-dispatcher wtmp.1boot.log journal syslogbootstrap.log kern.log syslog.1btmp kern.log.1 syslog.2.gz\n\nThis is fairly large collection of logs and log directories \u2014 69 files and directories in \/var\/log in this case, but 180 files when you include the files inside those directories.\n$ cd \/var\/log\n$ ls | wc -l\n69\n$ find . -type f -print | wc -l\n180\n\nWhen you examine your log files, you will see pretty clearly which are generations of the same basic log. For example, one of the primary log files \u2014 the syslog file \u2014 is broken into nine separate files. These represent what is basically a week's worth of historical data along with the current file. Most of the older files are zipped to preserve space.\n$ ls -l syslog*\n-rw-r----- 1 syslog adm 588728 Oct 15 20:42 syslog\n-rw-r----- 1 syslog adm 511814 Oct 15 00:09 syslog.1\n-rw-r----- 1 syslog adm 31205 Oct 14 00:06 syslog.2.gz\n-rw-r----- 1 syslog adm 34797 Oct 13 00:06 syslog.3.gz\n-rw-r----- 1 syslog adm 61107 Oct 12 00:08 syslog.4.gz\n-rw-r----- 1 syslog adm 31682 Oct 11 00:06 syslog.5.gz\n-rw-r----- 1 syslog adm 32004 Oct 10 00:07 syslog.6.gz\n-rw-r----- 1 syslog adm 32309 Oct 9 00:05 syslog.7.gz\n\nThe syslog files contain messages from many different system services \u2014 cron, sendmail and the kernel itself are just examples. You'll also see evidence of user sessions and cron (scheduled tasks).\nMost Linux systems no longer use the old messages and dmesg files that served as landing places for the bulk of our system messages for many years. Instead, a large variety of files and some special commands have become available to help present the log information that is likely to be most relevant to what you are looking for.\nDepending on the file in question, you might simply use more or tail commands, or you might use a file-specific command like this use of the who command to pull user login data from the wtmp log.\n$ who wtmp\nshs pts\/1 2018-10-05 08:42 (192.168.0.10)\nshs pts\/1 2018-10-08 09:41 (192.168.0.10)\nshs pts\/1 2018-10-11 14:00 (192.168.0.10)\nshs :0 2018-10-14 19:11 (:0)\nshs pts\/0 2018-10-14 19:16 (192.168.0.25)\nshs pts\/0 2018-10-15 07:39 (192.168.0.25)\nshs :0 2018-10-15 19:58 (:0)\ndory pts\/0 2018-10-15 20:01 (192.168.0.11)\nshs pts\/0 2018-10-15 20:42 (192.168.0.6)\nshs pts\/0 2018-10-16 07:18 (192.168.0.6)\nnemo pts\/1 2018-10-16 07:46 (192.168.0.14)\n\nSimilarly, you might see nothing when you run a tail faillog command, but a command like this shows you that it's simply full of zeroes:\n# od -bc faillog\n0000000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000\n \n*\n0076600\n\nYou may also see very little when you try to tail lastlog only to discover that you need to use the lastlog command to view that log's data.\nSo, here is a listing of log files in \/var\/log and some descriptions of what they contain and how to view their contents.\n\nalternatives.log -- "run with" suggestions from update-alternatives\napport.log -- information on intercepted crashes\nauth.log -- user logins and authentication mechanisms used\nboot.log -- boot time messages\nbtmp -- failed login attempts\ndpkg.log -- information on when packages were installed or removed\nlastlog -- recent logins (use the lastlog command to view\nfaillog -- information on failed login attempts -- all zeroes if none have transpired (use faillog command to view)\nkern.log -- kernel log messages\nmail.err -- information on errors detected by the mail server\nmail.log -- information from mail server\nsyslog -- system services log\nufw -- firewall log\nwtmp -- login records\n\njournalctl\nIn addition to the log files maintained in \/var\/log, there is also the systemd journal. While not a simple "log file" in the usual sense of a single log file, this journal represents an important collection of information on user and kernel activity. The information is retrieved from a variety of sources on the system.\nTo view the information that has been collected, you would use the journalctl command.\nHow much information you will see depends on whether you are a member of the adm group or not. Non-adm users will see relatively little information, but members of the adm group will have access to a massive amount of data \u2014 as shown in this example, which is merely showing us how many lines of information are available for this adm group member to review:\n$ journalctl | wc -l\n666501\n\nThat's more than 666,000 lines of text! To pare this down to a hopefully more digestible display, you're probably going to want to use arguments that tailor what you will see displayed. Some of the options available with journalctl include:\n--utc (change the time format to UTC)\n-b (only show records added since the last boot)\n-b -1 (only show records added since the previous to last boot)\n--since and --until (only show records added within the specify timeframe, e.g., --since "2018-10-15" --until "2018-10-11 06:00"\n\nHere's an example:\n$ journalctl --since "2018-10-16 13:28"\n-- Logs begin at Mon 2018-05-14 15:16:11 EDT, end at Tue 2018-10-16 13:28:57 EDT. --\nOct 16 13:28:25 butterfly kernel: [UFW BLOCK] IN=enp0s25 OUT= MAC=01:00:5e:00:00:01:02:\nOct 16 13:28:25 butterfly kernel: [UFW BLOCK] IN=enp0s25 OUT= MAC=01:00:5e:00:00:fb:00:\nOct 16 13:28:57 butterfly su: pam_unix(su:session): session closed for user root\nOct 16 13:28:57 butterfly sudo: pam_unix(sudo:session): session closed for user root\nlines 1-5\/5 (END)\n\nYou can also examine log entries just for some particular service. This is probably one of the more useful things that the journalctl command can do for you:\n$ journalctl -u networking.service\n-- Logs begin at Mon 2018-05-14 15:16:11 EDT, end at Tue 2018-10-16 08:06:31 EDT\nMay 14 15:16:12 shs-Inspiron-530s systemd: Starting Raise network interfaces.\nMay 14 15:16:12 shs-Inspiron-530s systemd: Started Raise network interfaces.\nMay 14 15:49:18 butterfly systemd: Stopping Raise network interfaces...\nMay 14 15:49:18 butterfly systemd: Stopped Raise network interfaces.\n-- Reboot --\nMay 14 15:49:50 butterfly systemd: Starting Raise network interfaces...\nMay 14 15:49:51 butterfly systemd: Started Raise network interfaces.\n-- Reboot --\n\nNotice how the system reboots are displayed in this output.\nTo get a list of services, try a command such as this:\n$ service --status-all | column\n [ + ] acpid [ + ] network-manager\n [ - ] alsa-utils [ - ] networking\n [ - ] anacron [ - ] plymouth\n [ + ] apparmor [ - ] plymouth-log\n [ + ] apport [ - ] pppd-dns\n [ + ] atd [ + ] procps\n [ + ] atop [ - ] quota\n [ + ] atopacct [ - ] quotarpc\n [ + ] avahi-daemon [ - ] rsync\n [ - ] bluetooth [ + ] rsyslog\n [ - ] console-setup.sh [ - ] saned\n [ + ] cron [ + ] sendmail\n [ + ] cups [ + ] speech-dispatcher\n [ + ] cups-browsed [ - ] spice-vdagent\n [ + ] dbus [ + ] ssh\n [ - ] dns-clean [ + ] sysstat\n [ + ] gdm3 [ - ] thermald\n [ + ] grub-common [ + ] udev\n [ - ] hwclock.sh [ + ] ufw\n [ + ] irqbalance [ + ] unattended-upgrades\n [ + ] kerneloops [ - ] uuidd\n [ - ] keyboard-setup.sh [ + ] whoopsie\n [ + ] kmod [ - ] x11-common\n\nIn the display above:\n+ = active\n- = inactive\n? = no status option available\n\nHere's a useful command for getting a quick report on disk space usage:\n$ journalctl --disk-usage\nArchived and active journals take up 824.1M in the file system.\n\nIf you want to focus on a particular process, you can do that by providing a PID (truncated) as in the example below.\n$ journalctl _PID=787\n-- Logs begin at Mon 2018-05-14 15:16:11 EDT, end at Tue 2018-10-16 08:25:17 EDT\nAug 03 18:02:46 butterfly apport: * Starting automatic crash report genera\nAug 03 18:02:46 butterfly apport: ...done.\n-- Reboot --\nSep 16 13:26:34 butterfly atopacctd: Version: 2.3.0 - 2017\/03\/25 09:59:59\nSep 16 13:26:34 butterfly atopacctd: accounting to \/run\/pacct_source\n-- Reboot --\nOct 03 18:08:41 butterfly apport: * Starting automatic crash report genera\nOct 03 18:08:41 butterfly apport: ...done.\n-- Reboot --\nOct 15 14:07:11 butterfly snapd: AppArmor status: apparmor is enabled but s\nOct 15 14:07:12 butterfly snapd: AppArmor status: apparmor is enabled but s\nOct 15 14:07:12 butterfly snapd: daemon.go:344: started snapd\/2.35.2 (serie\nOct 15 14:07:12 butterfly snapd: autorefresh.go:376: Cannot prepare auto-re\n\nNOTE: The systemd journal's configuration file is \/etc\/systemd\/journald.conf.\nWrap-up\nThe variety of log files on Linux systems is somewhat overwhelming, but discovering a handful of commands that can help pinpoint problems can save you a lot of time and stress.