Cron job sends duplicate reports after DST change because schedule uses server local time
Cron job sends duplicate reports after DST change because schedule uses server local time
At 01:30 on the fall-back Sunday, the nightly invoice report ran twice. Clients received two PDFs with the same data, support handled “which one is correct?” tickets, and finance manually reconciled the run log. The cost was not just duplicate email; it was an hour of trust repair and a report that could no longer be treated as a single source of truth.
The root cause was a cron duplicate job after DST: the crontab used server local time, the job was scheduled inside the repeated hour, and the job itself was not idempotent.
Why a cron duplicate job after DST happens
Cron matches minute, hour, day, and month against the server’s local clock unless the schedule explicitly carries a timezone [link needed]. During the DST fall-back transition, the same wall-clock hour occurs twice. A job set for 30 1 * * * can therefore match twice if the cron implementation and daemon behavior allow it [link needed]. In spring, the opposite happens: local times inside the skipped hour may not run at all [link needed].
That turns a normal report into a cron duplicate job after DST when the script assumes “one schedule match equals one logical report.” If the job writes a file, sends email, or calls an API without a unique run key, two matches produce two outputs.
Confirm it in logs and crontab
Check the server timezone and the exact schedule:
date
timedatectl
crontab -l
Then inspect cron logs around the transition. Depending on the system, use journalctl -u cron, journalctl -u crond, or /var/log/syslog [link needed]. Grep for the command and compare timestamps. If you use systemd timers, run systemctl list-timers --all and inspect the unit’s OnCalendar= value.
Look for two runs within the repeated hour, or a missing run in spring. Confirm whether the job has any guard: lock file, database uniqueness constraint, or “already sent for report date” check.
Fix the schedule and make the job safe
A cron duplicate job after DST is usually a timezone bug plus a non-idempotent job. Fix both.
1. Inventory every cron entry that can fire between 00:00 and 03:00 local time, plus anything near the DST transition.
2. Move fixed infrastructure schedules to UTC where the run time does not need to track local wall time. If your cron supports it, set CRON_TZ=UTC at the top of the crontab and convert the entry [link needed]. If not, use a systemd timer with an explicit UTC OnCalendar= value [link needed].
3. Keep local-business logic explicit. For a report that must represent “yesterday” in a business timezone, calculate the business date in the script from a defined timezone, not from the server clock.
4. Add an idempotency key. For example, create /var/lock/report-$(date -u +%F).lock or insert a unique row keyed by report name and logical run date before sending.
5. Test both transitions. Simulate fall-back and spring-forward with a fixed TZ and a frozen clock if available [link needed], then verify one logical report per period.
When NOT to do this
Do not move every job to UTC just because DST broke one report. Jobs tied to local wall time—store opening, payroll cutoff, utility meter reads—may need local-time scheduling; changing them to UTC can shift the run relative to the business day. Also, do not “fix” duplicates by adding a long sleep; it reduces overlap but does not create a durable idempotency guarantee.
Pre-deploy checklist
- [ ] List all schedules in the 00:00–03:00 local window.
- [ ] Set an explicit timezone in cron or the timer.
- [ ] Add a unique run key for each logical report period.
- [ ] Add a lock that survives the repeated hour.
- [ ] Verify one run in fall-back and one run in spring-forward.
- [ ] Alert on duplicate or missing report keys.
- [ ] Keep the previous schedule ready for rollback.