Blog

A failed SSH login, a service that will not start, and a disk reporting errors leave different kinds of clues in Linux logs. There is no single file that holds them all. On one machine, the useful entry may be in the systemd journal; on another, it may be in a file under /var/log. Finding the right record—and knowing what it can prove—makes troubleshooting and security checks more reliable.

What a system log records

A log is a time-ordered record of events reported by software or the operating system. An entry may include a timestamp, the program that produced it, a severity level, a process ID, and a message. A service might report that it started, could not read a configuration file, or lost its connection to another service.

Logs help answer practical questions: When did the failure begin? Did the service restart? Was an authentication attempt rejected? Did the kernel report a storage problem? They are especially useful when a problem comes and goes before anyone can inspect the machine.

But a log message is a report, not a complete account. Programs choose what to record, and some failures happen before they can write anything. A successful authentication entry confirms that an authentication step succeeded; it does not establish what the user did afterward. Check entries against other evidence, including service status and configuration, before drawing a conclusion.

The main places Linux keeps logs

Many current distributions use systemd-journald to collect messages from services, the kernel, and other sources into the journal. Read it with journalctl. Journal entries can include structured fields, letting you filter by service, boot, or priority rather than searching plain text alone.

Traditional syslog services, such as rsyslog or syslog-ng, can write messages to text files, often under /var/log. Some machines use both: journald collects events, while a syslog service saves selected messages to files. Container images, minimal installations, and distributions make different choices, so check what your machine actually has.

You may find /var/log/syslog on a Debian-derived system or /var/log/messages on another distribution. Authentication-related records may appear in /var/log/auth.log or /var/log/secure. None of these filenames is guaranteed. Application logs may be elsewhere under /var/log, in the journal, or wherever the application is configured to write them.

Logs are not the same as application data or an audit trail designed to capture every action. A web server access log records requests according to that server’s settings; it may not show what the application did with them. If an investigation needs precise records of particular operations, those events must be logged deliberately and retained for an appropriate period.

A safe first pass with journalctl

On a systemd-based machine you administer, start with a focused view instead of dumping the entire journal. These commands read existing records; they do not change service configuration:

• journalctl -b shows entries from the current boot.
• journalctl -b -p warning shows current-boot messages at warning severity and above.
• journalctl -u ssh.service -b filters current-boot entries for that unit, if it exists under that name.
• journalctl --since '2025-03-08 09:00:00' --until '2025-03-08 10:00:00' limits results to a time window; replace the example dates with the period you are investigating.
• journalctl -f follows new entries as they arrive; press Ctrl+C to stop.

Unit names vary: some distributions use ssh.service, while others use sshd.service. If a unit filter returns nothing, confirm the name and check whether the event was logged elsewhere before deciding nothing happened. Regular users may see only part of the journal. Administrative access can reveal more, but use it only on systems you are authorized to inspect.

For an intermittent service failure, note when it stopped and inspect a short window around that time. Read the entries just before the error, too. A database connection failure could follow a network interruption or database restart rather than cause the problem. Widen the window when the first set of entries leaves a question unanswered.

Reading text logs

If the relevant events are in a file, less /var/log/syslog lets you browse without editing it; press q to leave. tail -n 50 /var/log/syslog shows the last 50 lines, and tail -f /var/log/syslog follows new ones. Use the file that exists on your system. Some logs require elevated read permissions because they contain sensitive operational details.

A search such as grep -i 'failed' /var/log/auth.log can locate a term, but it is not an incident detector. “Failed” may appear in unrelated contexts, while relevant events may use different words. Read the surrounding lines and identify which program produced the match.

How to interpret an entry

Start with four details: time, source, severity, and message. The timestamp places the event in sequence; the source identifies the reporting service or component. Severity reflects how that producer classified the event, which may differ from its importance to your investigation. The message says what the producer observed.

Take particular care with time. Hosts may use different time zones or formats, and a system clock can be wrong. A line copied without its date or host may mislead you. Before comparing machines, establish their time zones and any clock offset. On one machine, a boot boundary may explain why a process ID or service startup appears to repeat.

Suppose a service reports “permission denied” while opening a file. That identifies a blocked operation, not necessarily the right fix. Check the exact path, the account running the service, and related messages. Broadly changing permissions just to silence the log could create a security problem. Likewise, a kernel disk warning calls for checking storage health and backups, not simply restarting the application that failed afterward.

Repeated authentication failures can be useful defensive signals on a system you manage, especially alongside source addresses and timing. Repetition alone does not establish intent: an outdated client configuration or a scheduled job using old credentials can produce similar entries. Preserve timestamps and surrounding records before making changes. Do not paste usernames, addresses, or tokens into public troubleshooting posts.

Why logs matter beyond a single error

• Troubleshooting: Entries can connect a user-visible symptom to a service start, crash, configuration error, or resource problem.
• Security monitoring: Authentication and service events can flag unexpected access attempts or configuration changes for authorized review.
• Change verification: After an approved update or restart, logs can show whether a service initialized normally or immediately reported a problem.
• Capacity and reliability: Recurring warnings may reveal pressure or failure patterns before an outage. Pair logs with measurements when assessing performance.

That value depends on what has been retained. A journal may persist across reboots or live only in volatile storage, depending on the configuration and environment. Text logs can be rotated, compressed, and eventually deleted to limit disk usage. A quiet journal after a reboot does not prove that nothing happened earlier. Check retention settings and older files before deciding how far back you can investigate.

Protecting log quality and privacy

Logs can contain account names, network addresses, file paths, request details, and secrets accidentally emitted by applications. Limit access and redact sensitive fields before sharing excerpts. Developers should avoid logging passwords, session tokens, private keys, and unnecessary personal data; a useful diagnostic record is not worth exposing credentials.

If records need to survive a host failure or possible local tampering, an organization may send them to a separately managed collection service. That setup needs access controls, secure transport, retention rules, and sufficient storage. Local logs are still useful for immediate diagnosis, but someone with enough control over a compromised machine may alter or remove them. Remote collection does not automatically make every event complete or trustworthy, either.

To practice on your own Linux machine, note the current time and restart a noncritical service you are authorized to manage. Inspect that unit’s journal for the same time window, then compare its stop and start messages with its actual status. The log helps you find the event; the status check helps you verify what happened.