azure-enterprise-scale-ml

Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..

Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.

A list of tools for annotating data, managing annotations, etc.

The data-validation toolkit for enhanced dbt (data build tool) PR review

:sunglasses: A curated list of awesome DataOps tools

? dim: Manage the open data in your project like a package manager.

Beneath is a serverless real-time data platform ⚡️

Ingesting data with Pulumi, AWS lambdas and Snowflake in a scalable, fully replayable manner

Duke MIDS: Data Engineering and DataOps Course

DataOps framework for Machine Learning projects.