Discussion on Model Backends GPTQ 4-Bit Quantisation: Compressing The Models After Pretraining

Описание: Loading a huge language models into GPU is one of the challenging tasks that many dev-ops will have in near future. GPTQ is the State of the Art Quantisation tech that has been released, and already has got 4 different implementations.

As things are moving fast, the video tries to provide an over-arching view of the GPTQ implementation, their repositories and show how Text-Gen-UI by Oobabooga brings it all together.

We also touch a bit on oobabooga TG-UI, and discuss how the UI server implemented with Gradio, and how Cloud servers can be leveraged for trying out the UI.

The presentation used in this discussion is located at
https://github.com/insightbuilder/pyt...

Chapter Navigation:
0:00 What is GPTQ: Overview
1:10 Background On Algorithms
2:17 Text Generation UI : Model Loaders
4:20 Purpose of the Video
7:12 Which Libraries Help in Quantisation
11:40 GPTQ Evolution Map
14:35 What GPTQ does
15:05 Benchmarks Discussion
19:46 Supporting Git Repo
21:40 Recap & Outro

I believe you will like this video, and subscribe to the channel. Further uploads related to Big Data, Large Language models and Artificial Intelligence will be shared to your Youtube Dashboard Directly.

The supporting playlists are
The bard Project
   • Google Bard LLM : New AI Model in the Block  
Practical Projects Playlist
   • The Future is Here: Large Language Models ...  
Huggingface Playlist
   • Mastering NLP with Hugging Face: A Compreh...  
Python Data Engineering Playlist
   • Learn to Data Engineer and Problem Solve: ...  
Python Ecosystem of Libraries
   • Mastering the Python Ecosystem: Must-Know ...  
ChatGPT and AI Playlist
   • Learn about AI Language Models and Reinfor...  
AWS and Python AWS Wrangler
   • Building a Powerful Data Pipeline with AWS...  

PS: Got a question or have a feedback on my content. Get in touch
By leaving a Comment in the video
topmate.io/insightbuilder?utm_source=topmate&utm_medium=popup&utm_campaign=SocialProfile
@mail [email protected]
@twitter Handle is @KQrios
@medium   / about  
@github https://github.com/Kamalabot