Testing Groq's Speculative Decoding version of Meta Llama 3.3 70 B

Описание: In this special video, my 10-year-old daughter and I team up to investigate whether speculative decoding delivers on its promise of speed without sacrificing quality. Watch as she helps conduct real AI model testing using our Prompt Judy tool!
What We Cover:

Website: https://promptjudy.com/
Public evaluations: https://app.promptjudy.com/public-runs
Free Signup: https://app.promptjudy.com/login?sign...

Intro:    • Introducing Prompt Judy! AI-Powered LLM & ...  
Quick Start:    • Prompt Judy Intro  


Evaluations in this video:
https://app.promptjudy.com/public-run...
https://app.promptjudy.com/public-run...

The theory behind speculative decoding (how smaller models can accelerate larger ones)
Testing GROQ's Llama 3.3 70B specdec against their standard Llama 3.3 70B versatile
My daughter running our comprehensive test suite:
• Harmful Question detection test
• Named Entity Recognition test
• SQL code generation test
• Retrieval Augmented Generation test

See how accessible AI testing can be as my daughter helps reveal our findings on speed improvements (2x-5x depending on the task).