Avoiding insurmountable queue backlogs
≡ paperIt's so tempting to add a queue to a distributed system as a way to deal with spikes in work that can be deferred. Unfortunately these queues act as a gigantic reservoir that can unleash a tidal wave on a service when the dam breaks. Or even if the rate of draining is managed, they can be too large to drain in practice. Meanwhile, urgent work gets blocked behind the "non-urgent" backlog. This article goes into a ton of detail about approaches to manage this.
→ Read the paper
https://aws.amazon.com/builders-library/avoiding-insurmountable-queue-backlogs/