Day 27/90 – Outlier Detection & Handling | AI Course in Tamil

Описание: Day 26 of your Complete AI Course in Tamil focuses on outlier detection and handling – one of the most critical data preprocessing steps before training AI/ML models. Outliers can distort model performance, create biased predictions, and ruin statistical analysis, so this session teaches you how to identify, analyze, and treat outliers using Pandas and statistical methods with clear Tamil explanations.

You'll first learn visual outlier detection using boxplot() and hist() to spot extreme values in distributions. Then you'll master IQR method (Interquartile Range): calculate Q1 (25th percentile), Q3 (75th percentile), IQR = Q3-Q1, and flag values outside Q1-1.5*IQR to Q3+1.5*IQR as outliers. Tamil examples show how this works on sales data, customer ages, and transaction amounts.

Next, you'll implement Z-score method using scipy.stats.zscore() – values with |z-score| 3 are outliers (more than 3 standard deviations from mean). You'll compare IQR vs Z-score on the same dataset to see when each method works best (IQR better for skewed data, Z-score for normal distributions).

Outlier handling strategies covered:

Remove outliers with boolean indexing: df = df[~outlier_mask]

Cap outliers (winsorizing): replace with Q1-1.5IQR or Q3+1.5IQR

Impute outliers with median (robust to outliers)

Flag outliers as a new binary column for ML models

Real AI examples show domain-specific outlier decisions: ₹50,000 salary might be normal in Bangalore but outlier in small towns; 1000+ daily transactions might indicate fraud or power users. You'll learn business context matters more than blind rules.

Multiple column outlier detection using loops or df.select_dtypes(include=[np.number]).apply() processes all numeric columns simultaneously. You'll create outlier summary reports showing percentage of outliers per column to prioritize cleaning efforts.

By the end of Day 26 (Tamil), you'll confidently detect outliers using IQR/Z-score/visual methods, choose appropriate handling strategies, and prepare clean datasets for robust AI/ML model training.