XDrift: Explainable Concept Drift Detection via Multi-Method Consensus

Описание: XDrift: Explainable Concept Drift Detection in Data Streams through Multi-Method Consensus and Advanced Tree Comparison

by Abraham Itzhak Weinberg

Concept drift in data streams is a major challenge for machine learning systems. Detecting changes alone is not enough; understanding why they happen is critical.
In this video, we introduce XDrift, a novel explainable AI framework that combines multiple drift detection methods with advanced decision tree comparison metrics to deliver both accurate detection and clear, interpretable explanations.
XDrift integrates four complementary drift detectors: KS-Test, performance-based monitoring, DDM, and Page-Hinkley, together with XAI techniques such as SHAP, LIME, counterfactual explanations, and new tree-structure comparison metrics including edit distance, path similarity, and feature usage divergence.
Through extensive experiments across six drift scenarios, XDrift achieves a 47.2 percent improvement in detection accuracy over baseline methods, with an average detection error of 546 samples. Gradual drift is detected most precisely, with only 75 samples error, but causes the largest performance drop at 18.3 percent accuracy loss, highlighting important trade-offs in drift behavior.
XDrift is designed for real-time use, processing data streams 50 to 67 times faster than typical arrival rates while still providing actionable explanations. Our analysis also reveals a strong inverse correlation between path similarity and detection performance, with a correlation coefficient of minus 0.62 and statistically significant results, offering new insights into concept drift dynamics.