Popular GitHub Repositories for Data Pipeline

A Clojure machine learning library

Tools for ASR Corpus Generation from Online Video

Backtesting an algorithmic trading strategy using Machine Learning and Sentiment Analysis.

Splicing: Gen-AI Copilot for Data Engineering

Time Series Anomaly detection. The monitored signal is made-up of machinery vibration sensor measurements.

An Airflow-based pipeline that fetches data from a free API, cleans and transforms it, and saves it to a database—ready for downstream machine learning.

Sync your team's data to your LLM applications in real-time

End-To-End Data Engineering Project. Made to learn some common data engineering practices.

The TFX Automation Bot is a cutting-edge Python-based tool designed to streamline machine learning pipelines and optimize TensorFlow Extended (TFX) workflows. This bot automates model training, validation, deployment, and monitoring, making AI development seamless and efficient.

Churn Prediction with Kedro, Kedro-Viz, and Kedro-Mlflow ?? ?. PowerBI Dashboard ? also included. Kedro? https://kedro.org/

Generate lyrics with TensorFlow and the Genius API

Cryptocurrency prediction using LSTM (Long Short Term Memory) [ Hugging Face: https://huggingface.co/spaces/qywok/cryptocurrency_prediction ]

Latency Estimation for Neural Network Architecture

Pipeline-Genie is an intelligent data pipeline that processes CSV datasets, identifies their schema, and leverages LLaMA 2.0 to extract business insights. Users can select relevant business needs, triggering automated ETL transformations using Apache Spark. The final transformed dataset is stored in AWS S3 and made available for download.

Vector Database testing for semantic search capabilities for use in LLM applications

A pipeline that collects, preprocesses and labels raw user-item interaction data to build a recsys with collaborative filtering (SVD, ALS) and LTR (XGBoost ranking) methods

A cloud-based deployment and scaling for ML services (Docker, k8s, AWS S3/Lambda, Flask, GitHub Actions)

This repository contains the code, documentation, and datasets for a comprehensive exploration of machine learning techniques to address class imbalance. The project investigates the impact of various methods, like ADASYN, KMeansSMOTE, and Deep Learning Generator, on classification performance while effectively demonstrating benefits of pipelining.

A RAG-based pipeline used for context-based retrieval of queries related to my career portfolio

This project demonstrates a complete pipeline for building a machine learning model using XGBoost on sustainable energy data, integrated with a FastAPI-based REST API. The application allows users to upload data, train the model, and make predictions via API endpoints. Docker and PostgreSQL are used for containerization and data management.