Latest Research Papers
2025-01-16
arXiv
Foundations of Large Language Models
The book focuses on foundational concepts of large language models, covering pre-training, generative models, prompting techniques, and alignment methods. It is designed for college students, professionals, and practitioners in NLP and related fields.
This is a book about large language models. As indicated by the title, it
primarily focuses on foundational concepts rather than comprehensive coverage
of all cutting-edge technologies. The book is structured into four main
chapters, each exploring a key area: pre-training, generative models, prompting
techniques, and alignment methods. It is intended for college students,
professionals, and practitioners in natural language processing and related
fields, and can serve as a reference for anyone interested in large language
models.
2024-12-27
arXiv
A Survey on Large Language Model Acceleration based on KV Cache Management
This survey provides a comprehensive overview of Key-Value (KV) cache management strategies for accelerating Large Language Model (LLM) inference, categorizing them into token-level, model-level, and system-level optimizations. It aims to offer insights and support the development of efficient and scalable KV cache management techniques for practical LLM deployment.
Large Language Models (LLMs) have revolutionized a wide range of domains such
as natural language processing, computer vision, and multi-modal tasks due to
their ability to comprehend context and perform logical reasoning. However, the
computational and memory demands of LLMs, particularly during inference, pose
significant challenges when scaling them to real-world, long-context, and
real-time applications. Key-Value (KV) cache management has emerged as a
critical optimization technique for accelerating LLM inference by reducing
redundant computations and improving memory utilization. This survey provides a
comprehensive overview of KV cache management strategies for LLM acceleration,
categorizing them into token-level, model-level, and system-level
optimizations. Token-level strategies include KV cache selection, budget
allocation, merging, quantization, and low-rank decomposition, while
model-level optimizations focus on architectural innovations and attention
mechanisms to enhance KV reuse. System-level approaches address memory
management, scheduling, and hardware-aware designs to improve efficiency across
diverse computing environments. Additionally, the survey provides an overview
of both text and multimodal datasets and benchmarks used to evaluate these
strategies. By presenting detailed taxonomies and comparative analyses, this
work aims to offer useful insights for researchers and practitioners to support
the development of efficient and scalable KV cache management techniques,
contributing to the practical deployment of LLMs in real-world applications.
The curated paper list for KV cache management is in:
\href{https://github.com/TreeAI-Lab/Awesome-KV-Cache-Management}{https://github.com/TreeAI-Lab/Awesome-KV-Cache-Management}.
2017-06-12
arXiv
Attention Is All You Need
The paper introduces the Transformer, a new neural network architecture that relies entirely on attention mechanisms, eliminating the need for recurrence and convolutions. This model outperforms existing methods in machine translation tasks, achieving state-of-the-art results with faster training times. The Transformer also demonstrates strong performance in other tasks like English constituency parsing.
The dominant sequence transduction models are based on complex recurrent or
convolutional neural networks in an encoder-decoder configuration. The best
performing models also connect the encoder and decoder through an attention
mechanism. We propose a new simple network architecture, the Transformer, based
solely on attention mechanisms, dispensing with recurrence and convolutions
entirely. Experiments on two machine translation tasks show these models to be
superior in quality while being more parallelizable and requiring significantly
less time to train. Our model achieves 28.4 BLEU on the WMT 2014
English-to-German translation task, improving over the existing best results,
including ensembles by over 2 BLEU. On the WMT 2014 English-to-French
translation task, our model establishes a new single-model state-of-the-art
BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction
of the training costs of the best models from the literature. We show that the
Transformer generalizes well to other tasks by applying it successfully to
English constituency parsing both with large and limited training data.