Can a Small Local AI Model Triage Real Email? Python + Ollama Agent Test

Описание: Can a small local AI model do real executive assistant work on a regular laptop?

In this video, I continue my local AI agent experiment by building a communication triage pipeline. The goal is to see whether a small model can sort email-shaped data into useful categories like needs reply, needs action, waiting on, follow up, and FYI — then turn that into a readable briefing and action handoff files.

This is not a polished production framework or a full Python coding tutorial. It is an experiment build. I’m testing what small local models can actually do, where they fail, what settings and guardrails help, and how the agent workflow improves over time.

In this episode:
Build the email triage pipeline with test data
Move from Markdown output to strict JSON output
Tune num_predict and num_ctx
Add batching for larger message sets
Test whether threading helps
Create output paths for a briefing and action handoff files
Swap in real Gmail data
Clean and filter noisy real email before sending it to the model
Compare Qwen, Llama, Gemma, and Gemini on the same workflow

The series hypothesis:
A small local AI model can handle useful work tasks on a regular laptop when given clear inputs, tight limits, and controlled access to data.

Local models tested:
Qwen3:4B
Qwen3:1.7B
Llama3.2:3B
Gemma3:4B

Cloud benchmark:
Gemini 2.5 Flash

The big takeaway: small local models can do real work, but the surrounding system matters. The connector, cleanup, prompt, batching, schema, and output files are what make the workflow usable.

This series is for people interested in local AI, practical AI agents, Python workflows, Ollama, privacy-focused AI, and realistic experiments with small models.

0:00 Intro: Can a Small Model Triage Email?
2:08 Part 1: Building the Triage Pipeline
4:09 Prompt Structure + Output Categories
6:11 Tuning the Local Model
8:33 Adding More Data + Batching
10:02 Does Threading Help?
11:05 Briefing + Action Handoffs
13:23 Adjusting Prompt Rules
14:06 Part 2: Testing Real Gmail Data
15:40 Part 3: Local vs Cloud Model Comparison