FlashState is a serving system for hybrid large language models that optimizes prefix cache management for agentic applications. It enables finer-grained state reuse to increase throughput and lower response times.

Weili Xu
147K 992 114 74 3d ago