Making a slow queue keep up
July 30, 2026 · 15 min read
An AI-vision worker pipeline was serializing everything into 20–60 minute waits. Here is how queue-class separation, backlog-based autoscaling, a computable concurrency budget, and a whole-job heartbeat fixed it — plus the SQS/Celery gotchas I hit.
QueuesSQSCeleryAWSScalingReliabilityBackend