Zobrazit v EDS

Engineering for Millions of Requests Per Second: Building Ultra-Low Latency, High-Availability Services at Scale.

Uloženo v:
Podrobná bibliografie
Název: Engineering for Millions of Requests Per Second: Building Ultra-Low Latency, High-Availability Services at Scale.
Autoři: Jayakumar, Naveen Kumar
Zdroj: Journal of Computer Science & Technology Studies; 2026, Vol. 8 Issue 1, p60-73, 14p
Témata: DISTRIBUTED computing, FAULT tolerance (Engineering), COMPUTER network architectures, COMPUTER performance, MATHEMATICAL optimization
Abstrakt: The growth of digital services has intensified the need for distributed systems that can sustain millions of requests per second while maintaining ultra low latency and continuous availability. Engineering such workloads requires coordinated decisions about programming language runtimes, serialization formats, network topology, caching, and fault tolerance. This paper proposes a design framework for ultra low latency, high availability microservice-based services, grounded in published empirical studies and documented industrial systems operating at high throughput. The framework is based on the evidence on the latency impact of binary serialization and language runtimes such as Rust and Java, network hop minimization and cellular architectures for failure isolation, multi-tier caching and precomputation, and adaptive resilience mechanisms including token bucket based retry budgets, circuit breakers, and additive increase multiplicative decrease control mechanisms. Rather than reporting new experiments, the paper synthesizes findings from existing empirical evidence and organizes these findings into a layered set of design dimensions and best practice guidelines intended to support predictable tail latency, high availability, and cost-aware operation in large-scale cloud environments. [ABSTRACT FROM AUTHOR]
Copyright of Journal of Computer Science & Technology Studies is the property of Al-Kindi Center for Research & Development and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Databáze: Complementary Index
Popis
Abstrakt:The growth of digital services has intensified the need for distributed systems that can sustain millions of requests per second while maintaining ultra low latency and continuous availability. Engineering such workloads requires coordinated decisions about programming language runtimes, serialization formats, network topology, caching, and fault tolerance. This paper proposes a design framework for ultra low latency, high availability microservice-based services, grounded in published empirical studies and documented industrial systems operating at high throughput. The framework is based on the evidence on the latency impact of binary serialization and language runtimes such as Rust and Java, network hop minimization and cellular architectures for failure isolation, multi-tier caching and precomputation, and adaptive resilience mechanisms including token bucket based retry budgets, circuit breakers, and additive increase multiplicative decrease control mechanisms. Rather than reporting new experiments, the paper synthesizes findings from existing empirical evidence and organizes these findings into a layered set of design dimensions and best practice guidelines intended to support predictable tail latency, high availability, and cost-aware operation in large-scale cloud environments. [ABSTRACT FROM AUTHOR]
ISSN:2709104X
DOI:10.32996/jcsts.2025.8.1.5