Overview
We're introducing P2P In-Memory Caching to improve cache hit ratio and memory utilization efficiency across pods in the online feature store. Each pod will now have the capability to fetch and serve features directly from the memory of peer pods, eliminating redundant storage and improving cost efficiency.

Why This Matters

1. Higher Cache Efficiency: Reduce overall memory footprint by avoiding duplicate feature storage across pods.
2. Lower Latency on Misses: A local miss no longer defaults to storage/backend—peer memory access is faster.
3. Infra Cost Savings: Fewer calls to downstream data stores
4. Self-Healing + Dynamic: Pods can dynamically discover peers and their cached content, adapting to cluster changes.