Using GPT4 Vision & LangChain to Query Memories from a Video

Описание: In this video lets experiment with using GPT-4 Vision to craft video descriptions and then use a simple retrieval augmented generation approach leveraging LangChain to query those descriptions! Discover how we navigate through the process of creating a descriptive log from video clips – turning captured memories across the city into searchable text data.

Important disclaimer: I am using the word "memories" here loosely, it just means objects and entities seen in a video.

📚 Chapters
00:01 - Introduction and project idea explanation
01:09 - Creating a new notebook and outlining the approach
01:35 - Discussing the need for a descriptor from video clips using GP4 Vision
02:23 - Testing live with a frame from a sample video
04:38 - Integrating descriptions with a querying system
05:46 - Fetching code snippets from OpenAI's cookbook for the project setup
07:10 - Setting up a conda environment and installing packages
12:18 - Loading and processing a sample video for description
14:15 - Writing a function to generate descriptions from videos
15:20 - Writing output descriptions to a text file and logging information
17:08 - Setting up text file generation with video file date extraction
19:28 - Beginning retrieval setup using Link Chain for querying descriptions
24:30 - Implementing the text loading and embedding process for retrieval
27:40 - Setting up the retrieval Q&A for querying document descriptions
32:01 - Running a test query on the system to find a specific video detail


If you enjoyed this video don't forget to like, subscribe, and stay tuned for more coding sessions like this one! Cheers :)!

🔗 Links:
Subscribe!:    / @automatalearninglab  
Tiktok: https://www.tiktok.com/@enkrateialucc...
Twitter:   / lucasenkrateia  
LinkedIn:   / lucas-soares-969044167  

Support the Channel!
Buy me a cup of coffee: https://tr.ee/7tYsD-tUu2
Paypal: https://paypal.me/lucasenkrateia?coun...