GIGO Video 1: Moving from Manual Scripts to Auto-Data Systems

Описание: GIGO Video 1: Moving from Manual Scripts to Auto-Data Systems

"Data is 80% of data science, yet we still treat data cleaning like a manual chore."

Welcome to the first module of the GIGO (Garbage In, Garbage Out) series. In this video, Nik Bear Brown introduces a revolutionary approach to data preparation: Auto-Data.

Just as AutoML runs thousands of models to find the best fit, Auto-Data runs dozens of automated "tests" on a dataset to identify what is "recyclable" (useful features) and what belongs in the "landfill" (noise and bias).

Key Concepts Covered:

The Recycling Analogy: Why we shouldn't "rewire the factory" for every new bag of garbage.

The Scriptlet Strategy: Moving away from buggy, one-off "Boo Scripts" to a library of standardized, tested scripts.

The Auto-Data Report: How to generate a comparison between raw data and transformed data to validate your corrections.

Detective Work: Identifying missing values, skewness, imbalance, and bias through a systematic "interrogation" of the data.

Over the next four weeks, we will build these "scriptlets" together, explaining the math behind the tests and creating a system that suggests—and applies—corrections automatically.

Subscribe to follow the full GIGO series and stop letting garbage data ruin your models.

#DataScience #GIGO #AutoML #AutoData #DataCleaning #MachineLearning #NortheasternUniversity #DataDetective #MycroftAndDiti #BigData #TechEducation #FeatureEngineering