RedHatAI
/

Qwen2.5-VL-7B-Instruct-quantized.w8a8

text-generation-inference

8-bit precision

compressed-tensors

Model card Files Files and versions

shubhrapandit commited on Feb 26

Commit

ff4cfb0

·

verified ·

1 Parent(s): 7f609fb

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -298,7 +298,7 @@ lm_eval \
 ## Inference Performance
-This model achieves up to 1.04x speedup in single-stream deployment, depending on hardware and use-case scenario.
 The following performance benchmarks were conducted with [vLLM](https://docs.vllm.ai/en/latest/) version 0.7.2, and [GuideLLM](https://github.com/neuralmagic/guidellm).
 <details>

 ## Inference Performance
+This model achieves up to 1.56x speedup in single-stream deployment and 1.5x in multi-stream deployment, depending on hardware and use-case scenario.
 The following performance benchmarks were conducted with [vLLM](https://docs.vllm.ai/en/latest/) version 0.7.2, and [GuideLLM](https://github.com/neuralmagic/guidellm).
 <details>