Picture for Hunjong Lee

Hunjong Lee

Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization

Add code
Mar 24, 2025
Figure 1 for Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization
Figure 2 for Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization
Figure 3 for Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization
Figure 4 for Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization
Viaarxiv icon