Let's code on cloud GPUs with VSCode and Jupyter notebooks

Описание: In this video I show how to connect VSCode to cloud GPUs for remote development.

This is an extremely simple, and free way to set up a remote development environment that is persistent and scalable. I also show how to run forks of the environment (as jobs) on their own machines to trivially parallelize workloads.

The Studio offers at least 3 ways of coding, 1) connect your local VSCode, 2) Use the native web-based VSCode on the Studio or 3) Run Jupyter notebooks on the browser.

Studio is a much more powerful alternative to Colab that is production grade and highly scalable.

Chapters:

00:00 Introduction
00:25 Start a Studio
00:38 VSCode on a cloud CPU machine
00:55 Jupyter notebooks on a cloud CPU machine
01:40 Connect your local VSCode to the cloud machine
02:58 SSH and terminal access
03:05 Install python and system packages
03:40 Explain the persistent cloud environment
04:10 Example 1: Running a Python script for training a model
04:48 The optimal development workflow for GPUs
05:02 Run on a GPU (without code changes)
05:30 Start another Studio for a clean environment (studio.lightning.ai)
06:50 Python script automatically uses the GPU
07:50 Run a hyperparameter sweep from the Studio
11:20 Making code changes from local VSCode to remote server
14:00 Launch async Jobs from the Studio
15:20 Add 4 GPUs to the Studio
16:13 Monitor and interface with the Jobs
17:08 How to speed up model training by scaling to more GPUs
18:23 Profile GPU utilization
19:12 Start Tensorboard to compare models training
20:15 Switch back to CPU to debug
21:15 Summary