Popular GitHub Repositories for Low Rank Adaptation

[SIGIR'24] The official implementation code of MOELoRA.

Repository for Chat LLaMA - training a LoRA for the LLaMA (1 or 2) models on HuggingFace with 8-bit or 4-bit quantization. Research only.

LORA: Low-Rank Adaptation of Large Language Models implemented using PyTorch

The official implementation for MTLoRA: A Low-Rank Adaptation Approach for Efficient Multi-Task Learning (CVPR '24)

Easy wrapper for inserting LoRA layers in CLIP.

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

Fine tuning Mistral-7b with PEFT(Parameter Efficient Fine-Tuning) and LoRA(Low-Rank Adaptation) on Puffin Dataset(multi-turn conversations between GPT-4 and real humans)

Over 60 figures and diagrams of LLMs, quantization, low-rank adapters (LoRA), and chat templates FREE TO USE in your blog posts, slides, presentations, or papers.

A curated list of Parameter Efficient Fine-tuning papers with a TL;DR

Trying to train LoRAs for ESM-2

Efficient fine-tuned large language model (LLM) for the task of sentiment analysis using the IMDB dataset.

The simplest repository & Neat implementation of different Lora methods for training/fine-tuning Transformer-based models (i.e., BERT, GPTs). [ Research purpose ]

EDoRA: Efficient Weight-Decomposed Low-Rank Adaptation via Singular Value Decomposition

Long term project about a custom AI architecture. Consist of cutting-edge technique in machine learning such as Flash-Attention, Group-Query-Attention, ZeRO-Infinity, BitNet, etc.

In this we fine-tune MLP with LoRA and DoRA on CIFAR-10 Dataset

This repository contains the source code and related resources for R-LoRA.

Finetuning LLaMA 3.2-1B-Instruct model using qLoRA and LoRA quantization PEFT methods

FairFedMed: Group Fairness-aware Federated Learning Benchmark for Medical Imaging

Music-gen model fine-tuned to generate music in the style of the Violet Evergarden Original Soundtrack.

A Low-Rank Adaptation of a pretrained Stable Diffusion model that generates background scenery. Trained with PyTorch, and deployed with AWS EC2 and Ngrok.