Popular GitHub Repositories for Data Quality

Learn how to design, develop, deploy and iterate on production-grade ML applications.

Evidently is ??an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

The Open Source Feature Store for AI/ML

Learn how to design, develop, deploy and iterate on production-grade ML applications.

FeatHub - A stream-batch unified feature store for real-time machine learning

Profile and monitor your ML data pipeline end-to-end

Free Open-source ML observability course for data scientists and ML engineers. Learn how to monitor and debug your ML models in production.

DataOps Data Quality TestGen is part of DataKitchen's Open Source Data Observability. DataOps TestGen delivers simple, fast data quality test generation and execution by data profiling,  new dataset hygiene review, AI generation of data quality validation tests, ongoing testing of data refreshes, & continuous anomaly monitoring

The Lightning Catalog is an open-source data catalog designed for preparing data at any scale in ad-hoc analytics, data virtualization, data warehousing, lake houses, and ML projects.

Data quality control tool built on spark and deequ

Data-IQ: Characterizing subgroups with heterogeneous outcomes in tabular data (NeurIPS 2022)

The most extensive open massively multilingual corpus of datasets for training sentiment models. The corpus consists of 79 manually selected from over 350 datasets reported in the scientific literature based on strict quality criteria and covers 27 languages.

Data-SUITE: Data-centric identification of in-distribution incongruous examples (ICML 2022)

Automatic Label Error Correction www.techrxiv.org/users/679328/articles/731085

Examples for trying out the harpin AI identity resolution and data quality toolkit

Synthetic data quality evaluation & visualization