LLM Inference at Scale
Author: Srinivasa Rao Aravilli
Rights: Worldwide
Edition: 2027
Dimension: 7.5*9.25 Inches
Book Type: Paperback
- Description
- Table Of Contents
- About The authors
Large language model inference powers modern AI applications, but serving LLMs efficiently in production requires managing complex GPU memory constraints and hardware execution bottlenecks. This book is a practical guide to building fast, cost-efficient, and reliable serving infrastructure with vLLM and SGLang.
This book systematically covers the entire production inference architecture. It begins with autoregressive processing, GPU compute mechanics, and memory limits, then details vLLM internals, continuous batching, PagedAttention, and multi-GPU parallelism. You will then explore SGLang, RadixAttention prefix reuse, and structured decoding. The text details key optimizations, including KV cache quantization, speculative decoding, multi-tiered caching, prefill-decode disaggregation, and AI gateway traffic routing. Finally, you will master benchmarking TTFT and TPOT metrics, build a multi-tenant production capstone, and explore discrete diffusion models alongside future serverless architectures.
By the end of this book, you will possess the specialized systems engineering expertise needed to optimize, benchmark, and deploy enterprise-grade LLM inference engines capable of handling high-concurrency production workloads.
What will you learn
Master autoregressive inference foundations and production SLAs.
Analyze GPU memory bandwidth and CUDA utilization.
Implement vLLM PagedAttention and continuous batching.
Scale tensor, pipeline, and MoE parallelism strategies.
Program SGLang RadixAttention prefix reuse execution workflows.
Accelerate generation using speculative draft-verifier pipelines.
Who this book is for
This book is for machine learning engineers, generative AI engineers, ML platform professionals, and students deploying LLMs in production. Readers need intermediate Python skills and foundational transformer knowledge. Prior GPU computing experience helps but is not required.
1. Foundations of LLM Inference Systems
2. GPU Architecture for Inference Workloads
3. vLLM Engine Architecture and Internals
4. Parallelism Strategies in vLLM.
5. Programmable Inference with SGLang Runtime
6. KV Cache Quantization
7. Accelerating Generation with Speculative Decoding
8. Prefix Cache
9. Disaggregated Inference Architecture and KV Movement
10. The AI Gateway Layer for LLM Inference
11. Inference Optimization for Discrete Diffusion Language Models
Srinivasa Rao Aravilli is a technology leader, author, and Senior Director of Machine Learning Engineering with over 30 years of experience spanning enterprise middleware to modern Generative AI and Agentic AI systems. His career includes foundational ML engineering and applied research roles at HP Labs, Cisco, and Capital One, where he led multiple large teams to build secure, large-scale AI and machine learning platforms.
His work centers on distributed computing, generative AI, AI security, governance, and privacy. He has published research on distributed secure systems, adversarial testing for large language models, and disaggregated inference paradigms, and authored the book Privacy-Preserving Machine Learning for engineers building safer AI without compromising user data. He leads the team publishing papers in large
Currently, Srinivas focuses on large language model inference at scale, Gen AI security and privacy, ML governance, agent monitoring and observability. He holds a Master's degree in Computer Applications from Andhra University.