Benchmarks That Lie, Honest AI & $0.34 Medical Triage: AI Research Digest — Mar 11, 2026

Описание: AI benchmark scores might be measuring memorization, not real reasoning. And making AI more honest could be as simple as letting it think first.

Today's episode of AI Research Chat breaks down 10 new artificial intelligence papers on AI safety, benchmarking, large language models, and real-world deployment. We look at why standard machine learning benchmarks may be overstating reasoning ability, how AI agents can be intercepted before causing harm, and what genuine AI honesty looks like mechanistically. Plus: a clinical triage AI agent that matches doctor-level accuracy for $0.34 a case - a result that could reshape healthcare AI in 2026.

In this episode:
MASEval - The orchestration framework around an AI model can matter as much as the model itself. Teams picking between agent frameworks now have a way to compare options fairly.
EsoLang-Bench - Models scoring 85-95% on standard coding tasks collapse to 0-11% on identical problems in obscure languages. Benchmark scores may reflect training data coverage, not genuine reasoning.
OOD-MMSafe - Frontier AI models fail up to 67.5% of the time on safety scenarios where nobody had harmful intent - a "causal blindness" problem. A new method cuts failure rates to under 8%.
TrustBench - A real-time safety filter for AI agents that intercepts harmful actions before they happen, cutting them by 87% in under 200 milliseconds.
The Reasoning Trap - Stronger logical reasoning could accidentally create dangerous self-awareness in AI. The paper calls for safety advances to keep pace with reasoning improvements.
PRECEPT - A planning agent using structured exact-match retrieval instead of fuzzy search gains a +41 percentage-point first-try advantage on complex logistics tasks.
EU AI Act Benchmark - An automated compliance checker hits 0.87 F1 on prohibited-use detection, making large-scale legal compliance feasible for the first time.
PrivPRISM - More than half of nearly 10,000 Android apps understate their data collection in Play Store safety labels compared to their own privacy policies.
Think Before You Lie - LLMs that reason before responding are consistently more honest across model scales and families. Chain-of-thought may be a cheap, underused alignment tool.
Sentinel - An autonomous AI agent matches or exceeds individual clinician performance on clinical triage at $0.34 per case, cutting turnaround from days to minutes.

Research Papers:
MASEval: Extending Multi-Agent Evaluation from Models to Systems
https://arxiv.org/abs/2603.08835
EsoLang-Bench: Evaluating Genuine Reasoning in Large Language Models via Esoteric Programming Languages
https://arxiv.org/abs/2603.09678
OOD-MMSafe: Advancing MLLM Safety from Harmful Intent to Hidden Consequences
https://arxiv.org/abs/2603.09706
Real-Time Trust Verification for Safe Agentic Actions using TrustBench
https://arxiv.org/abs/2603.09157
The Reasoning Trap - Logical Reasoning as a Mechanistic Pathway to Situational Awareness
https://arxiv.org/abs/2603.09200
PRECEPT: Planning Resilience via Experience, Context Engineering & Probing Trajectories
https://arxiv.org/abs/2603.09641
AI Act Evaluation Benchmark: An Open, Transparent, and Reproducible Evaluation Dataset for NLP and RAG Systems
https://arxiv.org/abs/2603.09435
PrivPRISM: Automatically Detecting Discrepancies Between Google Play Data Safety Declarations and Developer Privacy Policies
https://arxiv.org/abs/2603.09214
Think Before You Lie: How Reasoning Improves Honesty
https://arxiv.org/abs/2603.09957
From Days to Minutes: An Autonomous AI Agent Achieves Reliable Clinical Triage in Remote Patient Monitoring
https://arxiv.org/abs/2603.09052

Keywords: AI safety, artificial intelligence, machine learning, AI news 2026, AI research, AI podcast, large language models, LLM benchmarks, AI alignment, AI agents, ChatGPT, EU AI Act, privacy Android apps, clinical AI, chain-of-thought reasoning, AI honesty, AI triage, benchmark evaluation, agentic AI

---
New episode every weekday. Subscribe for daily AI research summaries.
Full digest: https://eddyariki.github.io/news-feed...
🤖 Audio generated with Google Gemini TTS.