6c8ed0a3b8Grouping the whole activities table by thread every run is unusable on a large instance: on a 312 GB database it plans as a seq scan plus a 50M-row on-disk sort. Instead, walk activities in id order (flake ids are time-ordered) from a persisted cursor up to the id that corresponds to the deadline, and verify only the threads those activities belong to through the (type, context) index. Measured on that instance: 147 ms for a 50k-activity range scan, 349 ms warm / 14 s cold to verify ~700 threads. The cursor is persisted per chunk so a killed run resumes. The watermark walk keeps a cursor of its own so it cannot leave the age-based walk stranded behind the deadline. Adds the retention_cursors table. |
||
|---|---|---|
| .. | ||
| cron | ||
| delete_worker_test.exs | ||
| poll_worker_test.exs | ||
| publisher_worker_test.exs | ||
| purge_expired_activity_test.exs | ||
| purge_expired_filter_test.exs | ||
| purge_expired_token_test.exs | ||
| reachability_worker_test.exs | ||
| receiver_worker_test.exs | ||
| remote_fetcher_worker_test.exs | ||
| scheduled_activity_worker_test.exs | ||
| signature_retry_worker_test.exs | ||