ycliper

Популярное

Музыка Кино и Анимация Автомобили Животные Спорт Путешествия Игры Юмор

Интересные видео

2025 Сериалы Трейлеры Новости Как сделать Видеоуроки Diy своими руками

Топ запросов

смотреть а4 schoolboy runaway турецкий сериал смотреть мультфильмы эдисон
Скачать

Don't Buy A GPU For Local AI Until You See These Numbers

Автор: Local Explained

Загружено: 2026-11-15

Просмотров: 23372

Описание: A used Tesla P100 GPU costs $180 and reads its memory at 732GB per second, which prices its speed at about $0.25 per GB/s against $1.60 for a used RTX 3090 at $1,500, but the whole box, its power bill and a software clock that is already ticking change the sum. This video goes through the retired data center cards for local AI at November prices, builds the whole machine around the budget pick, and puts it next to the cloud, so you can decide which of your two tabs to close.

Complete Local AI course: join the waitlist → https://localanalysis.ai/le/curso

CHAPTERS
00:00 Two tabs open
01:40 Why everyone ends up with a 3090
03:10 What an old card actually sells you
05:26 The price per ounce of speed
07:23 The P40, the famous one
08:52 The P100, the card nobody looked at
11:24 The MI50, and AMD's early exit
13:51 The V100, the one-card answer
15:33 The 2080 Ti that never left
16:30 The 353 tokens per second, checked
18:37 How long the software keeps them alive
21:05 What the whole box costs
23:53 What it costs to leave it on
25:39 The box against the cloud
27:03 When the answer flips
29:08 Before you click buy
30:34 The one rule

What is a used server GPU for local AI?
It is a retired data center graphics card, like the Tesla P40, P100 and V100 from NVIDIA or the MI50 from AMD, that people buy cheaply to run language models at home. It has no fan and no display output, and it takes a server-style power plug, but its memory is what the model reads for every token it writes, and that memory is fast.

The catch is time: NVIDIA and AMD have already dropped support for these cards, so every fast number you see comes from community forks of llama.cpp that a few people keep alive.

What this video covers
• A Tesla P100 at $180 and 732GB per second comes to about $0.25 per GB/s of reading speed, roughly a sixth of the $1.60 a used RTX 3090 charges at $1,500 and 936GB per second.
• The famous 24GB Tesla P40 costs $339 and was $239 on September 4, and one owner measured about 152 tokens per second reading a 32,000-token prompt, which is 211 seconds of waiting before the first word.
• Two P100s run a 27-billion-parameter model at 24.5 tokens per second on stock llama.cpp and 33.7 with community patches, and the whole box comes to about $830, or $24.60 per token per second against about $36 for the 3090.
• The 16GB MI50 went from $150 in July to $299 on November 11, the 32GB version now has a $734 median listing, and two patched cards reached 48 tokens per second at 85 watts each.
• A 32GB V100 runs the same model on one card at 33.4 tokens per second but asks $1,039, and one adapter fan measured 82 decibels, about a hair dryer on your desk.
• The post claiming four V100s write 353 tokens per second checks out at 322 on the project's own measurement, and the same setup falls to 72 when the NVLink bridge is switched off.
• CUDA 13 removed Maxwell, Pascal and Volta, which covers the P40, P100 and V100, and ROCm stopped supporting the MI50 in April 2025, so the dates on the clock are already written down.
• A small V100 box draws about 293 watts while writing, which is $12.77 a month at 8 hours a day and 18.16 cents per kWh, and at the cloud median of $2.20 per million tokens the $830 box takes about 15 months to pay off.
• The answer flips to no for light users, whose payback takes more than ten years, and to yes if you already own a workstation that can take the cards, where the cards alone come out more than 3 times cheaper per token per second.

Figures from forum and community posts, like the 24.5 and 33.7 tokens per second of the P100 pair, are individual users' experiences, illustrative, not averages.

#localai #llm #gpu #teslap40 #mi50

Не удается загрузить Youtube-плеер. Проверьте блокировку Youtube в вашей сети.
Повторяем попытку...
Don't Buy A GPU For Local AI Until You See These Numbers

Поделиться в:

Доступные форматы для скачивания:

Скачать видео

  • Информация по загрузке:

Скачать аудио

Похожие видео

© 2025 ycliper. Все права защищены.



  • Контакты
  • О нас
  • Политика конфиденциальности



Контакты для правообладателей: [email protected]