journalctl tail: follow a service's logs and find out why it failed
Run journalctl -u nginx -f to follow a service's log live, like tail -f. It prints the last 10 lines, then waits for new ones; press Ctrl+C to stop. Use -n 50 for more history, or leave out -u to follow the whole system.
To find out why a unit failed, run systemctl status nginx, then journalctl -u nginx -b -n 50. Logs from before the last reboot (-b -1) exist only if the journal is stored on disk in /var/log/journal.
How do I tail logs with journalctl?
journalctl reads the systemd journal, the log store that collects messages from every service, the kernel and systemd itself. It has its own versions of the tail options:
| You want | Command |
|---|---|
Follow new lines live (tail -f) | journalctl -f |
| Follow one service | journalctl -u nginx -f |
| Last 50 lines, then follow | journalctl -u nginx -n 50 -f |
Last 50 lines and stop (tail -n 50) | journalctl -u nginx -n 50 --no-pager |
| Several services at once | journalctl -u nginx -u php8.3-fpm -f |
| Jump to the end in the pager | journalctl -u nginx -e |
| Newest first | journalctl -u nginx -r |
Following cron on our Ubuntu server for three seconds shows the default: the last lines first, then it waits.
$ timeout 3 journalctl -u cron -f -n 3
Nov 14 16:45:01 web-01 CRON[270734]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Nov 14 16:45:01 web-01 CRON[270735]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Nov 14 16:45:01 web-01 CRON[270734]: pam_unix(cron:session): session closed for user root
A few details matter in practice:
-fshows 10 lines first unless you pass-n. Without-f, journalctl prints the whole journal from the oldest entry, in a pager. That's why plainjournalctl -u nginxseems to start in the wrong place.-ejumps to the end inside the pager and implies-n 1000, so it stays fast on a big journal.--no-pagerprints straight to the terminal. Use it in scripts and when you pipe the output.- Permissions: only root and members of the
systemd-journal,admandwheelgroups can read the whole system journal. Run it withsudoif you see nothing.
How do I see the logs for one service (systemctl logs)?
systemctl has no logs command. Use journalctl -u <unit>. It shows what the service printed and systemd's own messages about it, such as “Started”, “Reloaded” and “Failed”. systemctl status <unit> shows the last 10 of those lines under the status block, which is often enough for a first look.
A service's own log files are a separate matter. nginx on Ubuntu writes requests and errors to /var/log/nginx/. The journal only gets what nginx prints to its standard error, such as a config error at startup or reload. For the request log, tail -f /var/log/nginx/access.log is still the right tool.
How do I show logs from a time range (--since, --until)?
--since and --until take full timestamps ("2026-11-14 15:50"), the words today, yesterday and now, and relative times such as -1h or "1 hour ago":
journalctl -u nginx --since "1 hour ago"
journalctl -u nginx --since today -p warning
journalctl --since "2026-11-14 15:50" --until "2026-11-14 15:55"
A narrow window is the fastest way to see what happened during a change. This is a restart of cron on our Ubuntu server:
$ journalctl -u cron --since "2026-11-14 15:51" --until "2026-11-14 15:52" -o short-iso --no-pager
2026-11-14T15:51:54+00:00 web-01 systemd[1]: Stopping cron.service - Regular background program processing daemon...
2026-11-14T15:51:54+00:00 web-01 systemd[1]: cron.service: Deactivated successfully.
2026-11-14T15:51:54+00:00 web-01 systemd[1]: Stopped cron.service - Regular background program processing daemon.
2026-11-14T15:51:54+00:00 web-01 systemd[1]: Started cron.service - Regular background program processing daemon.
2026-11-14T15:51:54+00:00 web-01 (cron)[215777]: cron.service: Referenced but unset environment variable evaluates to an empty string: EXTRA_OPTS
2026-11-14T15:51:54+00:00 web-01 cron[215777]: (CRON) INFO (pidfile fd = 3)
2026-11-14T15:51:54+00:00 web-01 cron[215777]: (CRON) INFO (Skipping @reboot jobs -- not system startup)
Times without a time zone are read in the server's local time zone. Add UTC to the timestamp, or use -o short-iso as above, when you compare logs from servers in different zones.
How do I see logs from the previous boot (-b -1)?
-b limits the output to one boot: -b alone is the current boot, -b -1 the one before, -b -2 the one before that. List the boots the journal knows about:
$ journalctl --list-boots --no-pager
IDX BOOT ID FIRST ENTRY LAST ENTRY
... (3 older boots trimmed)
-1 9230e4878d3144818364576d04af040c Sat 2026-11-14 12:20:57 UTC Sat 2026-11-14 12:33:43 UTC
0 030bf938e9fd4f53a0bad8d7121d2419 Sat 2026-11-14 12:33:53 UTC Sat 2026-11-14 16:33:25 UTC
The end of the previous boot shows how the server went down. Here nginx stopped cleanly for a planned reboot:
$ journalctl -b -1 -u nginx -n 3 --no-pager
Nov 14 12:33:43 web-01 systemd[1]: Stopping nginx.service - A high performance web server and a reverse proxy server...
Nov 14 12:33:43 web-01 systemd[1]: nginx.service: Deactivated successfully.
Nov 14 12:33:43 web-01 systemd[1]: Stopped nginx.service - A high performance web server and a reverse proxy server.
If the previous boot ends with normal service lines and no shutdown messages, the server probably lost power, hung or was reset rather than rebooted. If --list-boots shows only boot 0 on a server that has rebooted, the journal isn't kept on disk; the persistent-storage section below shows what that looks like and how to fix it. Our reboot guide covers confirming that a server came back after a planned reboot.
How do I show only errors with journalctl (-p err)?
-p filters by priority. A single level shows that level and everything more serious; a range such as -p warning..err shows just that range.
| Level | Name | Shown by -p err | Shown by -p warning |
|---|---|---|---|
| 0–2 | emerg, alert, crit | Yes | Yes |
| 3 | err | Yes | Yes |
| 4 | warning | No | Yes |
| 5–7 | notice, info, debug | No | No |
There's a trap. When a service writes to standard output or standard error, systemd logs each line at info by default, unless the program prefixes the line with a level. So -p err can hide the line that explains the problem. On a Debian 12 test server we broke nginx's config on purpose, with a directive that nginx version doesn't know, and reloaded it:
$ journalctl -u nginx -b -p err --no-pager | head -n 1
Nov 14 16:59:22 web-01 systemd[1]: Reload failed for nginx.service - A high performance web server and a reverse proxy server.
That says that it failed, not why. The priorities show where the reason went (output trimmed to the reload):
$ journalctl -u nginx -b -o json --no-pager | jq -r '[.PRIORITY, .MESSAGE] | @tsv'
...
6 Reloading nginx.service - A high performance web server and a reverse proxy server...
6 2026/11/14 16:59:22 [emerg] 1780#1780: unknown directive "http2" in /etc/nginx/sites-enabled/default:23
5 nginx.service: Control process exited, code=exited, status=1/FAILURE
3 Reload failed for nginx.service - A high performance web server and a reverse proxy server.
nginx called it [emerg], but the journal stored it at priority 6 (info). Searching the message text finds both lines. -g takes a regular expression and ignores case when the pattern is all lowercase:
$ journalctl -u nginx -b -g 'emerg|failed' -o short-iso --no-pager | head -n 2
2026-11-14T16:59:22+0000 web-01 nginx[1780]: 2026/11/14 16:59:22 [emerg] 1780#1780: unknown directive "http2" in /etc/nginx/sites-enabled/default:23
2026-11-14T16:59:22+0000 web-01 systemd[1]: Reload failed for nginx.service - A high performance web server and a reverse proxy server.
Use -p err to scan a whole server for trouble (journalctl -b -p err). When you investigate one unit, read all of its lines around the failure.
Why did my service fail? Read systemctl status, then journalctl -xeu
When a start or restart fails, systemctl tells you where to look. This is from our Debian 12 test VM, after we broke nginx's config on purpose:
$ sudo systemctl restart nginx
Job for nginx.service failed because the control process exited with error code.
See "systemctl status nginx.service" and "journalctl -xeu nginx.service" for details.
systemctl status <unit> --no-pager. Read theActive:line (for examplefailed (Result: exit-code)), the line with the exit status, and the last log lines.journalctl -xeu <unit>. That's-ufor the unit,-eto jump to the end, and-xto add explanations from systemd's message catalog. For a cleaner view, usejournalctl -u <unit> -b -n 50 --no-pager.- Find the first error, not the last. The final lines are systemd's summary. The cause is usually a few lines up, printed by the program: a bad config line, a port in use, a missing file, a permission error.
- Check for a restart loop with
systemctl show -p NRestarts,Result <unit>. A growingNRestartsmeans it keeps crashing. “Start request repeated too quickly” means systemd gave up; fix the cause, runsudo systemctl reset-failed <unit>, and start it again.
The same Debian VM had a unit that failed at boot. Its status shows the pattern:
$ systemctl status rc-local --no-pager
× rc-local.service - /etc/rc.local Compatibility
Loaded: loaded (/etc/systemd/system/rc-local.service; enabled; preset: enabled)
Drop-In: /usr/lib/systemd/system/rc-local.service.d
└─debian.conf
Active: failed (Result: exit-code) since Sat 2026-11-14 16:58:24 UTC; 7s ago
Process: 421 ExecStart=/etc/rc.local start (code=exited, status=203/EXEC)
CPU: 214us
Nov 14 16:58:24 web-01 systemd[1]: Starting rc-local.service - /etc/rc.local Compatibility...
Nov 14 16:58:24 web-01 (rc.local)[421]: rc-local.service: Failed to locate executable /etc/rc.local: Permission denied
Nov 14 16:58:24 web-01 (rc.local)[421]: rc-local.service: Failed at step EXEC spawning /etc/rc.local: Permission denied
Nov 14 16:58:24 web-01 systemd[1]: rc-local.service: Control process exited, code=exited, status=203/EXEC
Nov 14 16:58:24 web-01 systemd[1]: rc-local.service: Failed with result 'exit-code'.
Nov 14 16:58:24 web-01 systemd[1]: Failed to start rc-local.service - /etc/rc.local Compatibility.
The last three lines only say that it failed. The second line says why: /etc/rc.local exists but isn't executable. Exit status 203 confirms it; systemd.exec(5) lists 203 (EXIT_EXEC) as “the actual process execution failed”, most likely a missing or inaccessible program. Here, sudo chmod +x /etc/rc.local would fix it, if you want that script to run at boot.
-x adds catalog text that names the step that failed. From the broken nginx restart (trimmed):
$ journalctl -xeu nginx.service --no-pager
... (trimmed)
Nov 14 16:59:22 web-01 nginx[1803]: 2026/11/14 16:59:22 [emerg] 1803#1803: unknown directive "http2" in /etc/nginx/sites-enabled/default:23
Nov 14 16:59:22 web-01 nginx[1803]: nginx: configuration file /etc/nginx/nginx.conf test failed
Nov 14 16:59:22 web-01 systemd[1]: nginx.service: Control process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░
░░ An ExecStartPre= process belonging to unit nginx.service has exited.
░░
░░ The process' exit code is 'exited' and its exit status is 1.
Nov 14 16:59:22 web-01 systemd[1]: nginx.service: Failed with result 'exit-code'.
... (trimmed)
It was the ExecStartPre= config test, not nginx itself, that failed, so the fix is in the config file. The journalctl manual asks you not to use -x when you paste logs into a bug report, where the extra text is noise. Our systemctl guide shows the same test with a reload, which failed safely and left nginx running.
How do I get logs as ISO timestamps or JSON (-o)?
-o value | Output | Use it for |
|---|---|---|
short | Classic syslog style (the default) | Reading |
short-iso | Same, with ISO 8601 timestamps and time zone | Comparing servers, tickets, incident notes |
short-precise | Syslog timestamps with microseconds | Ordering events within one second |
cat | The message only | Counting or grepping messages |
verbose | Every field of every entry | Finding which field to filter on |
json, json-pretty | One JSON object per entry | Scripts, jq, shipping logs elsewhere |
JSON with jq lets you pick fields. This pulls the error entries for nginx in the current boot:
$ journalctl -u nginx -b -p err -o json --no-pager | jq -c '{time: .__REALTIME_TIMESTAMP, priority: .PRIORITY, message: .MESSAGE}'
{"time":"1794659706954226","priority":"3","message":"Reload failed for nginx.service - A high performance web server and a reverse proxy server."}
The time is in microseconds since 1970. --output-fields=MESSAGE,PRIORITY trims the JSON, but journalctl still adds its own cursor and timestamp fields.
Other useful filters: kernel messages, PIDs and one run of a unit
-kshows kernel messages only, likedmesg, and implies-b. Combine it with-b -1to read the kernel log from before a crash.- Any journal field works as a filter:
journalctl _PID=1531,journalctl _COMM=sshd.-o verboseshows which fields an entry has. - One run of a unit:
journalctl _SYSTEMD_INVOCATION_ID=$(systemctl show -p InvocationID --value nginx)shows only lines from the current run, not from earlier starts.
$ journalctl -k -g 'Linux version' -o short-iso --no-pager
2026-11-14T12:33:53+00:00 web-01 kernel: Linux version 6.8.0-149-generic (buildd@lcy02-amd64-099) ... (line shortened)
That first kernel line records which kernel booted, which helps when you check the kernel version after an update.
Will my logs survive a reboot? Persistent vs volatile storage
journald keeps logs either on disk in /var/log/journal (persistent) or in memory in /run/log/journal (volatile, gone at reboot). The Storage= setting in journald.conf decides:
Storage= | Where logs go |
|---|---|
auto (default) | Disk if /var/log/journal exists, otherwise memory |
persistent | Disk; journald creates the directory if needed |
volatile | Memory only |
none | Nowhere (forwarding to syslog still works) |
Because the default depends on whether a directory exists, the three systems we tested differ:
| System (as shipped) | /var/log/journal | Earlier boots in the journal | Text log files |
|---|---|---|---|
| Ubuntu 24.04 server | Yes | Yes | /var/log/syslog (rsyslog) |
| Debian 12 VM | Yes | Yes | None: rsyslog isn't installed |
| AlmaLinux 9.6 VM | No | No | /var/log/messages (rsyslog) |
On the AlmaLinux VM, which had rebooted 20 minutes earlier, the journal only knew the current boot:
$ ls -ld /var/log/journal
ls: cannot access '/var/log/journal': No such file or directory
$ journalctl --list-boots --no-pager
IDX BOOT ID FIRST ENTRY LAST ENTRY
0 f8ec48fcd7614c2e965f3616d7a852d3 Sat 2026-11-14 16:38:05 UTC Sat 2026-11-14 17:01:24 UTC
$ journalctl -b -1
Specifying boot ID or boot offset has no effect, no persistent journal was found.
So if that server crashes, its journal can't tell you why. To switch to disk, the systemd-journald manual gives these commands; the last one moves the logs already in memory to disk. We ran them on the AlmaLinux VM:
$ sudo mkdir -p /var/log/journal
$ sudo systemd-tmpfiles --create --prefix /var/log/journal
$ sudo journalctl --flush
$ ls -l /var/log/journal/*/
total 8196
-rw-r-----+ 1 root systemd-journal 8388608 Nov 14 17:10 system.journal
The current boot's logs were kept, and from the next reboot on, -b -1 works. Boots from before the change are gone for good. Debian 12's release notes warn that systems upgraded with old settings may not store the journal on disk either, so check every server rather than assuming.
How much disk does the journal use, and how do I limit it?
$ journalctl --disk-usage
Archived and active journals take up 51.2M in the file system.
By default the persistent journal may use up to 10% of its file system and must leave 15% free, each capped at 4 GB. journald logs the limit it worked out each time it starts. On our Debian 12 VM, with a 40 GB disk:
$ journalctl -u systemd-journald -b -o cat | grep '^System Journal'
System Journal (/var/log/journal/…) is 8.0M, max 3.9G, 3.9G free.
To set your own limit, add a drop-in file rather than editing the main config, then restart journald so it reads it:
$ sudo mkdir -p /etc/systemd/journald.conf.d
$ printf '[Journal]\nSystemMaxUse=500M\n' | sudo tee /etc/systemd/journald.conf.d/size.conf > /dev/null
$ sudo systemctl restart systemd-journald
$ journalctl -u systemd-journald --since -1min -o cat | grep '^System Journal'
System Journal (/var/log/journal/…) is 8.0M, max 500.0M, 491.9M free.
(We removed the drop-in from the VM afterwards.) To free space right away, sudo journalctl --vacuum-size=500M or --vacuum-time=30d deletes the oldest archived journal files. That can't be undone, so save anything you need for an incident first.
How we checked this: On 14 Nov 2026 (UTC): an Ubuntu 24.04.5 LTS production server (systemd 255.4) with read-only commands, journalctl -f run under timeout 3; a Debian 12.15 test VM (systemd 252.39), where we broke and fixed an nginx config to produce a real failure; and an AlmaLinux 9.6 test VM (systemd 252), where we checked the volatile journal, ran the three commands that make it persistent, then restored it to volatile without losing that boot's logs. Hostnames and one site file name are changed, and long output is trimmed. On the Debian VM we also set and then removed a journal size limit. The vacuum commands are quoted from the journalctl manual and were not run. Anything we didn't run ourselves comes from the official documentation listed under Sources.
Restart services with an approval step
HostCrust restarts only running or failed services, never critical ones like sshd, and only on servers that allow service restarts. By default an approver signs off first; then the agent on the server reports whether the unit is active again, and the result goes in a receipt. Free for up to 3 servers.
Check which CVEs are really open on your server
HostCrust Patch Proof compares every installed package with your distribution's own security data. It counts fixes your distro backported and checks the kernel that is actually running, not just the one on disk. Read-only, no root, free.
FAQ
How do I stop journalctl -f?
Press Ctrl+C. -f keeps waiting for new lines until you interrupt it. In a script, wrap it in timeout, for example timeout 30 journalctl -u nginx -f, so it can't hang forever.
Why does journalctl show no logs for my service?
Three common reasons. You're not root and not in the systemd-journal, adm or wheel group, so you can't read the system journal (use sudo). The unit name is wrong; check it with systemctl list-units --type=service. Or the program writes its own log files: nginx on Ubuntu writes to /var/log/nginx/error.log and access.log, and the journal only gets systemd's messages and what nginx prints to stderr.
How do I show the last 100 lines of logs?
journalctl -n 100 for the whole system, or journalctl -u nginx -n 100 for one service. Add --no-pager if you want the output printed straight to the terminal or piped into another command.
How long does journald keep logs?
Until it hits a size limit, not a time limit. By default the journal on disk may use up to 10% of its file system (capped at 4 GB) and must leave 15% free. The oldest files are deleted first. MaxRetentionSec= adds a time limit, but it's off by default.
Is journalctl the same as /var/log/syslog?
Not quite. journald stores logs in its own binary files, and journalctl reads them. Where rsyslog is installed, it writes a text copy: /var/log/syslog on our Ubuntu 24.04 server, /var/log/messages on our AlmaLinux 9.6 VM. Debian 12 no longer installs rsyslog by default, and our Debian 12 VM had no syslog file at all, so journalctl was the only way to read its logs.
Sources
- journalctl(1) manual page, systemd 255 (Ubuntu 24.04)
- journald.conf(5): Storage=, SystemMaxUse=, SystemKeepFree= (Ubuntu 24.04)
- systemd-journald.service(8): persistent storage and journalctl --flush (Ubuntu 24.04)
- systemd.time(7): timestamp formats for --since and --until (Ubuntu 24.04)
- systemd.exec(5): SyslogLevel= and SyslogLevelPrefix= (Ubuntu 24.04)
- systemd v255 source: the “journalctl -xeu” hint systemctl prints when a job fails
- Debian 12 release notes: changes to system logging