Evidently is ​​an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test your data and models from research to production.

Algorithms for outlier, adversarial and drift detection

nannyml: post-deployment data science in python

Curated list of open source tooling for data-centric AI on unstructured data.

Frouros: an open-source Python library for drift detection in machine learning systems.

⚓ Eurybia monitors model drift over time and securizes model deployment with data validation

Free Open-source ML observability course for data scientists and ML engineers. Learn how to monitor and debug your ML models in production.

A curated list of awesome open source tools and commercial products for monitoring data quality, monitoring model performance, and profiling data 🚀

A toolkit for evaluating and monitoring AI models in clinical settings

A comprehensive solution for monitoring your AI models in production

PKBoost: Adaptive GBDT for Concept Drift, Built from scratch in Rust, PKBoost manages changing data distributions in fraud detection with a fraud rate of 0.2%. It shows less than 2% degradation under drift. In comparison, XGBoost experiences a 31.8% drop and LightGBM a 42.5% drop

Online and batch-based concept and data drift detection algorithms to monitor and maintain ML performance.

Passively collect images for computer vision datasets on the edge.

A tiny framework to perform adversarial validation of your training and test data.

Sales Conversion Optimization MLOps: Boost revenue with AI-powered insights. Features H2O AutoML, ZenML pipelines, Neptune.ai tracking, data validation, drift analysis, CI/CD, Streamlit app, Docker, and GitHub Actions. Includes e-mail alerts, Discord/Slack integration, and SHAP interpretability. Streamline ML workflow and enhance sales performance.

Drift-Lens: an Unsupervised Drift Detection Framework for Deep Learning Classifiers on Unstructured Data

A long-form article introducing the Twin Test: a practical standard for high-stakes machine learning where models must show nearest “twin” examples, neighborhood tightness, mixed-vs-homogeneous evidence, and “no reliable twins” abstention. Argues similarity and evidence packets beat probability scores for trust and safety.

In this repository, we will present techniques to detect covariate drift, and demonstrate how to incorporate your own custom drift detection algorithms and visualizations with SageMaker model monitor.

A ⚡️ Lightning.ai ⚡️ component for train and test data drift detection