Running Local LLMs on NVIDIA DGX Spark – A Field Report

Описание: This is not a tutorial on running local LLMs.

It’s a field report from building Gepetto – a personal thinking environment where human and machine intelligence collaborate as partners.

Local LLMs are not the center of that system, but they are an important layer.

In this video, I test the NVIDIA DGX Spark as a physical backbone for local execution and document what actually happens when abstraction disappears and reality kicks in.

What worked, what broke, and why predictable behavior matters more than benchmarks.

This is not a product review, not investment advice, and not a ranking of models.

It’s an honest snapshot of building local thinking infrastructure at this moment in time.

0:00 – Intelligence, the Final Frontier
0:27 – First Mission: Building Local Execution Infrastructure
1:14 – Hardware as Reality Check (NVIDIA DGX Spark)
2:50 – What the Local Layer Needs to Do
4:30 – Runtimes
5:18 – Testing the Runtimes
14:00 – When the System Had to Survive Without Me
16:00 – The UI as a Thinking Prosthesis
19:30 – Where This Leaves the Spark

ABOUT THIS VIDEO

This video documents one specific use case:
single-user, exploratory work where cognitive flow matters more than peak throughput.

The NVIDIA DGX Spark hardware is excellent.
The surrounding software ecosystem is not fully there yet.

If you want something finished and frictionless, waiting is the smarter move.
If you want to explore where personal thinking infrastructure might be heading, this shows where the landmines are.

ABOUT GEPETTO

Gepetto is an evolving thinking environment designed around the idea that intelligence can emerge between humans and machines – not as tools, but as collaborators.

This video covers one layer of that system: local execution.

HARDWARE & SOFTWARE TESTED

Hardware:
NVIDIA DGX Spark (ARM64, unified memory)

Models:
DeepSeek Coder 16B
Qwen Coder 30B
Qwen 2.5 32B
Qwen 2.5 72B

Runtimes:
Ollama
SG-Lang
TensorRT-LLM

#localllm #aiinfrastructure #dgxspark #nvidia #Gepetto #ThinkingEnvironment #aiengineering

December 28th, 2025