All posts

Engineering

How Sonae schedules a million runs a month

Jonas Duarte · Mar 10, 2026 · 8 min read

12:00:01.204run.queuedpr-reviewer

12:00:01.488step.startfetch_diff

12:00:02.611step.endfetch_diff

12:00:02.613step.startrun_tests

12:00:05.870step.endrun_tests

12:00:05.872run.done3 steps

Our first scheduler was a cron table and a worker pool. It held up until teams started triggering agents from webhooks, at which point bursts of a few thousand runs per minute became normal.

We replaced it with an append-only event log. Every trigger, step start, step end and retry is an event, and the scheduler is a consumer that decides what to run next. Replaying the log rebuilds the state of any run, which is also how the run log in the dashboard is drawn.

The result: p95 queue time went from 4.1s to 280ms, and a failed worker no longer loses a run.

Next postPicking a model for each step