On an AlmaLinux 9 server, chunks of January and February entries keep appearing inside June logs across several files, including messages, cron, and maillog. Restarting services and running log rotation did not stop it. What would you check first?
Why are old log entries showing up in my current logs?
Answers
Since the same date range appears in multiple logs, I would look at the journal-to-rsyslog path before assuming each log file is independently damaged. Check whether rsyslog is reading from journald through imjournal, then inspect the journal for errors with journalctl --verify. If it reports corruption, preserve anything you need before attempting cleanup or vacuuming, since those actions remove journal data.
There are known journal-related edge cases, but do not add a workaround parameter just because a search result mentions it. Check the installed rsyslog documentation and confirm the option applies to your versions and configuration.
Reply to this answer
Also check whether rsyslog is being restarted or reloaded around the time the entries reappear. With imfile inputs, a missing, stale, or unusable state file can make rsyslog reread content from the start of a file. Verify the configured state-file path, permissions, and whether it is actually being used for each input. Log rotation hooks are worth checking too.
Reply to this answer
Correct-looking timestamps make a simple clock jump less likely, but I would still compare the system clock, journal timestamps, and the file modification times around a recurrence. If the journal checks clean and the input state looks right, then broaden the investigation to storage or filesystem trouble. Check system logs for I/O errors, and plan any filesystem checks for an appropriate maintenance window.
Reply to this answer
Add your answer
Answers are reviewed before publication.
T9.Bobbin:
journalctl --verify reported corrupted files. I cleaned up old journal data and restarted services, so I am watching to see whether the blocks return.