vLLM benchmark

Описание: This video shows how to start vLLM built-in benchmark.

GPUs:
Nvidia RTX 4090
Nvidia A100
Nvidia H100
Nvidia H200

LLM models:
Qwen/Qwen3-Coder-30B-A3B-Instruct
openai/gpt-oss-120b
google/gemma-3-4b-it
meta-llama/Llama-3.1-8B-Instruct
mistralai/Mistral-7B-Instruct-v0.3
Qwen/Qwen2.5-Coder-7B-Instruct

Compare: Tensor Parallel vs Data Parallel

Commands and results are in the text version of this video:
https://pavlokhmel.com/llm_inferencin...