LLM-RLHF

APOLLO: SGD-like Memory, AdamW-level Performance

Find secrets with Gitleaks ?

本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）

Efficient Triton Kernels for LLM Training

Litmus is a comprehensive LLM testing and evaluation tool designed for GenAI Application Development. It provides a robust platform with a user-friendly UI for streamlining the process of building and assessing the performance of your LLM-powered applications.

An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)

H2O LLM Studio - a framework and no-code GUI for fine-tuning LLMs. Documentation: https://docs.h2o.ai/h2o-llmstudio/

Code examples and resources for DBRX, a large language model developed by Databricks

dstack is a lightweight, open-source alternative to Kubernetes & Slurm, simplifying AI container orchestration with multi-cloud & on-prem support. It natively supports NVIDIA, AMD, TPU, and Intel accelerators.

MoBA: Mixture of Block Attention for Long-Context LLMs