![](\"https:\/\/mms.businesswire.com\/media\/20260723669125\/en\/2858209\/5\/Tensormesh_Logo.jpg\"){kind=logo}
<\/a><\/a>For users, running more models on the same set of GPUs means lower costs. LMCache users can now reuse the infrastructure they have built to virtually expand their GPUs' capacity. In addition, customers can reuse KV cache chunks stored for short-term memory virtualization later for prefix or non-prefix KV cache matching, maximizing system efficiency.<\/p>
The Results<\/b><\/p>
This new approach is far more efficient than building GPUs with more memory, which has led to the industry\u2019s current memory supply crisis and increased the number of GPUs. Enterprises running AI over large document sets can now have a better experience and a lower bill. Testing showed:<\/p>
- Near-instant responses at scale: <\/b>On a 300 GB document set (Kimi-K2.6), time-to-first-token dropped from 3.4 seconds to under half a second, a nearly 7x improvement, thanks to reusing cached KV data from DRAM and NFS instead of recomputing it from scratch.<\/li>
- No slowdown as usage grows: <\/b>Output throughput held steady at ~48 tokens per second regardless of workload size, while unoptimized inference throughput fell by nearly 40% under the same load.<\/li>
- Twice the model density:<\/b> The same hardware doubled the model density.<\/li><\/ul>
\u201cThis powerful new collaboration builds on AMD\u2019s recent strategic investment in Tensormesh<\/a> and expands the capabilities of AMD GPUs,\u201d explained Junchen Jiang, Tensormesh CEO and LMCache co-creator. \u201cTogether, we\u2019re greatly enhancing the memory that inference engines can access for model weights and KV cache, using all of the memory resources on each node.\u201d<\/p>
\u201cWe recognize Tensormesh and LMCache as KV cache management leaders,\u201d said Anush Elangovan, AMD\u2019s vice president of AI software. \u201cAnd we\u2019re thrilled to announce AMD\u2019s breakthrough in virtual GPU memory management, amplified by LMCache and Tensormesh.\u201d<\/p>
The companies are sharing performance results from their collaboration at this week\u2019s AMD Advancing AI 2026<\/a>, a month before the public release of the LMCache open-source code.<\/p>
Additional Resources:<\/b><\/p>
- Fixing AI\u2019s Most Expensive Problem with Junchen Jiang, Tensormesh CEO<\/a><\/li>
- LMCache: Build The Foundation of AI Memory Tensor with KV Cache Infrastructure<\/a><\/li>
- Persistent KV Cache: Own Your Context Caching Lifecycle<\/a><\/li><\/ul>
About Tensormesh<\/b><\/p>
Tensormesh is the leader in caching-accelerated inference optimization for enterprise AI. Founded by faculty, PhD researchers and alumni from the University of Chicago, UC Berkeley, and Carnegie Mellon, and led by Junchen Jiang, University of Chicago faculty member and co-creator of LMCache, Tensormesh builds on years of academic research in distributed systems and AI infrastructure. The company has raised $24.5 million in total funding and is backed by Valley Capital Partners, NVentures, AMD Ventures, CoreWeave, and Laude Ventures.<\/p>
- LMCache: Build The Foundation of AI Memory Tensor with KV Cache Infrastructure<\/a><\/li>
- No slowdown as usage grows: <\/b>Output throughput held steady at ~48 tokens per second regardless of workload size, while unoptimized inference throughput fell by nearly 40% under the same load.<\/li>