I Tested Google's New Decisions API on 2,000 Questions (Gemini Decision 1)
Автор: Singularity Feed | AI Models , Agents & Robotics
Загружено: 2026-11-11
Просмотров: 812
Описание:
Google launched the Gemini Decisions API on 11 Nov 2026, powered by a new model, Gemini Decision 1. It does not write text: you give it options and it scores them in one pass, with a probability and a confidence for every answer. I ran 2,000 questions with known answers through it, and the same 2,000 through Gemini 3.5 Flash-Lite.
My results (11 Nov 2026):
Correct: Decision 1 1,726 of 2,000 (86.3%) | Flash-Lite 1,670 (83.5%)
Median time per answer: 0.67 s | 1.30 s (from my PC, includes network)
Cost of the run: $0.10 | $0.24
SST-2 sentiment, yes/no (500): Decision 1 97.0% | Flash-Lite 95.2%
SMS spam, yes/no (500): Decision 1 97.4% | Flash-Lite 89.6%
Banking77 intent, 77 options (500): Decision 1 78.2% | Flash-Lite 78.2%
Yelp stars, rubric score, exact (300): Decision 1 65.3% | Flash-Lite 61.7%
CIFAR-10 photos, 10 options (200): Decision 1 83.5% | Flash-Lite 85.0%
Yelp within one star: 96.3% | 95.0%
Confidence 0.9 or more: 1,176 answers, 98.0% right. Below 0.9: 824 answers, 69.5% right
1,000 questions in one request: 4.0 s, $0.0067, 92.5% correct
4 of 2,000 inputs were blocked by Google's safety filter and are scored as wrong
The same 20 requests sent 5 times gave identical answers
Test script, data sampling and every raw result: https://github.com/gnana70/gemini-dec...
Chapters:
0:00 A model that only decides
0:21 The short answer: accuracy, speed, cost
0:36 How it works: scoring options in one pass
0:54 Yes/no, multiple choice, rubric score, images
1:23 Can you trust its confidence?
1:39 1,000 questions in one request
1:53 The fine print: Google's claims vs my numbers
2:17 Google's benchmark table
2:31 How to try it
Sources:
Google launch post: https://blog.google/innovation-and-ai...
Model page on OpenRouter: https://openrouter.ai/google/gemini-d...
Datasets: PolyAI Banking77, Stanford SST-2, UCI SMS Spam Collection, Yelp Review Full, CIFAR-10
Flash-Lite was called through the Interactions API with minimal thinking, priced at $0.30 in / $2.50 out per 1M tokens
Not sponsored. Numbers are from my own runs and will differ with other data, longer inputs or another location.
#Gemini #GoogleAI #AIagents
Повторяем попытку...
Доступные форматы для скачивания:
Скачать видео
-
Информация по загрузке: