Dynamic Scale Factor Calculations for Key/Value Scales With FP8 KV Caching#317

gshtras merged 15 commits intomainROCm/vllm:mainfrom

kv-scales-on-the-flyROCm/vllm:kv-scales-on-the-flyCopy head branch name to clipboard