6 ms·
"Synchronized demand is the moment a large cohort of clients acts almost together. In a service with capacity ... requests per second and background load ...
by mparnisari 1y ago
"Synchronized demand is the moment a large cohort of clients acts almost together. In a service with capacity ...
requests per second and background load ..., the usable headroom is ..."
To the author of the article: I stopped reading after the first two sentences. I have no idea what you are talking about.
- cstrahan 1y ago"Synchronized demand is the moment a large cohort of clients acts almost together." Imagine everyone in a particular timezone browsing Amazon as they sit down for their 9 to 5; or an outage occurring, and a number of automated systems (re)trying requests just as the service comes back up. These clients are all "acting almost together". "In a service with capacity mu requests per second and background load lambda_0, the usable headroom is H = mu - lambda_0 > 0" Subtract the typical, baseline load (lambda_0) from the max capacity (mu), and that gives you how much headroom (H) you have. The signal processing definition of headroom: the "space" between the normal operating level of a signal and the point at which the system can no longer handle it without distortion or clipping. So headroom here can be thought of "wiggle room", if that is a more intuitive term to you.
- fuckaj 1y agoIs the pragmatic solution to return 503 and have clients back off. Or, if possible make latency a feature (embrace the queue!). For service to service internal stuff e.g. something like a request to hard delete something, this can always be a queue. And obviously you can scale up as the queue backs up. I do love the maths tho!