Intro to MetadataOps with Alejandro Aboy

Описание: Alejandro Aboy (https://substack.com/profile/22949723...) and I sat down to talk about the mess of modern AI. He told me he feels like an octopus lately. He spent years mastering dbt and Airflow. Now his day involves managing agent protocols and context windows.


The problem is that AI agents are the most demanding stakeholders you have ever had. They have zero intuition. They do not understand your tribal knowledge or your hidden business logic. If you give them a table without a deep layer of metadata, they will guess. When they guess, they hallucinate. When they hallucinate on your watch, you lose your seat at the table.


This conversation with Ale highlighted that the job has changed. You are no longer just shipping rows, but shipping the instructions that keep the machines from lying.


The Death Of The Static Catalog


Traditional data catalogs are where documentation goes to die. You spend a month tagging columns and then nobody looks at them. Ale calls the new approach MetadataOps.


In this model, your agents provide data back to the system. They help you understand how they use the information. If an agent struggles to identify a primary key, that is a production bug.


Documentation is now the fuel for your production AI. If you treat metadata as an afterthought, your AI solutions will stay in the playground. Real operators build systems that document themselves through use.





Data Gibberish is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.





Managing The Unstructured Mess


Unstructured, human generated data like images, audio or markdown documentation is a real mess. You must build guardrails that expect the mess. This means using regular expressions and specific skills to pull structure out of chaos. You are acting as a bridge between human habits and machine requirements.


This requires deciding when a data source is “good enough” to ship. You have to weigh the risk of a hallucination against the speed of the business. These are leadership calls, not technical ones.


The AI Stakeholder Shift


Your stakeholders think everything is possible now because they saw a demo on Twitter. This creates a massive gap in expectations. People expect you to ship features ten times faster. They do not see the infrastructure required to make those features reliable.


You must translate the technical risk of AI into business terms. If you fail to manage the context, you fail to manage the trust.


You are a translator now. You explain why a simple prompt is not a production strategy. You show them that the quality of the output depends on the quality of the metadata you have been screaming about for years.


Links


We mentioned a few resources. Here are the links:


Alejandro’s publication (https://thepipeandtheline.substack.com/) : Make sure you subscribe.


Substack Recapped (https://substackrecapped.com/) : A must try if you write on Substack.


Final Thoughts


The gap between a senior individual contributor and a data leader is the ability to manage ambiguity. Ale’s shift from dbt to agent protocols is a map for your own career.





The code is becoming a commodity. The ability to structure the world for a machine is the new high-value skill.





You cannot hide in the IDE and hope the data stays clean. You have to lean into the mess of unstructured files and vague stakeholder requests. Metadata is the only tool that bridges that gap.


You need to start thinking about it as managing the context of your entire company. If you do not own the metadata, you do not own the outcome. Seniority is about taking ownership of the results, not just the scripts. Use your metadata to build a system that thinks.


Until next time,


Yordan from Data Gibberish


PS: We live stream these conversations. Join and chat with us next time.





Data Gibberish is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.





This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.datagibberish.com/subscribe (https://www.datagibberish.com/subscri...)