Pragmatic Orchestrator: A Multi-Agent Multimodal AI System for Computational Linguistics Research

Описание: This video demonstrates the Pragmatic Orchestrator—a computational linguistics research platform that operationalizes theoretical frameworks from speech act theory, pragmatics, and sociolinguistics through a multi-agent AI architecture.
What You'll See:
The system implements a proactive intelligence loop that routes queries across specialized linguistic agents using pragmatic inference. Each agent is grounded in specific theoretical foundations:
Pragmatic Router (Qwen 3 1.7B via Ollama): Implements Searle's Speech Act Theory (1969) and Grice's Conversational Maxims (1975) for intent classification and routing decisions.
Vision Analyzer (Gemini 2.5 Flash via Google AI API + Google Cloud Vision API): Performs multimodal analysis combining OCR, label detection, and detailed visual semantics. The Cloud Vision API provides pixel-accurate character recognition, which is then enhanced by Gemini's deep visual understanding.
Code Switcher (Kimi K2 Thinking via Moonshot AI): Applies Myers-Scotton's Matrix Language Frame Model (1993) and Sperber & Wilson's Relevance Theory (1986) to preserve cultural opacity in bilingual Chinese-English discourse.
Discourse Analyzer (Kimi K2 Thinking): Implements Rhetorical Structure Theory (Mann & Thompson, 1988) for coherence relation analysis in meeting transcripts.
Semantic Parser (Claude Sonnet 4.5 via AWS Bedrock): Converts natural language to formal logic using Montague Semantics (1973).
Memory Curator (Claude Haiku 4.5 via AWS Bedrock): Extracts episodic memory with semantic role labeling based on Fillmore's Frame Semantics.
RLAIF Critic (Qwen 3 1.7B): Performs preference learning through linguistic feature analysis.
Explainability Bridge (Claude Haiku 4.5 via AWS Bedrock): Generates abductive reasoning explanations for routing decisions.
Technical Architecture:
The system integrates multiple cloud and local AI services:
AWS Bedrock: Claude Sonnet 4.5 and Claude Haiku 4.5 for complex reasoning tasks
Google AI API: Gemini 2.5 Flash for multimodal vision analysis
Google Cloud Vision API: OCR and label detection with structured data extraction
Moonshot AI: Kimi K2 Thinking for code-switching and discourse analysis
Ollama: Local inference for Qwen 3 1.7B (pragmatic routing) and Gemma 3 1B (parsing)
MLX Whisper: Audio transcription for multimodal input
Key Features Demonstrated:
Multimodal Processing: Image analysis combining Cloud Vision OCR with Gemini's detailed visual semantics
Proactive Intelligence Loop: Iterative refinement (2 iterations) that chains previous responses into subsequent analysis
Manual RLHF Rating System: 5-category evaluation (Overall, Speech Act Alignment, Syntactic Clarity, Cultural Appropriateness, Pragmatic Coherence) stored in SQLite
Episodic Memory: Frame Semantics-based semantic role labeling with persistent storage
Theoretical Grounding: Each agent displays its theoretical citation during processing
Research Applications:
This platform demonstrates how computational linguistics theory can be operationalized in AI systems, showing:
How speech act classification enables intelligent routing
How cultural preservation requires sociopragmatic analysis beyond translation
How multimodal understanding benefits from combining structured OCR with deep visual semantics
How human feedback (RLHF) can guide agent behavior through preference learning
The project encompasses:
Technical Depth: Integration of multiple cloud APIs (AWS Bedrock, Google AI, Google Cloud Vision) with local inference
Theoretical Rigor: Grounding in established computational linguistics frameworks
Research Potential: Demonstrates how theory can be operationalized in practical AI systems
Academic Alignment: Direct connection to CS & Linguistics research areas
Code Repository:
https://github.com/AICoolK8e8vC83i/Pr...
Technologies Used:
Python 3.11+
Streamlit (interactive dashboard)
SQLite (episodic memory and RLHF annotations)
AWS Bedrock (Claude models)
Google AI API (Gemini 2.5 Flash)
Google Cloud Platform (GCP) Vision API (OCR and label detection)
Moonshot AI (Kimi K2 Thinking)
Ollama (local LLM inference)
MLX Whisper (audio transcription)
OpenCV (video frame extraction)
Technical Terms:
computational linguistics, speech act theory, pragmatic inference, multi-agent AI, AWS Bedrock, Google Cloud Vision API, Gemini 2.5 Flash, Claude Sonnet, code-switching, sociolinguistics, AI research, multimodal AI, RLHF, episodic memory
Notes:
This project represents independent research that bridges computational linguistics theory and NLP with practical AI system implementation. The system is designed for research applications in computational pragmatics and sociolinguistics.