Popular GitHub Repositories for Transformers

21 Lessons, Get Started Building with Generative AI ? https://microsoft.github.io/generative-ai-for-beginners/

??? 60+ Implementations/tutorials of deep learning papers with side-by-side notes ?; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), ? reinforcement learning (ppo, dqn), capsnet, distillation, ... ?

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

DocsGPT is an open-source genAI tool that helps users get reliable answers from knowledge source, while avoiding hallucinations. It enables private and reliable information retrieval, with tooling and agentic system capability built in.

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

State-of-the-art Machine Learning for the web. Run ? Transformers directly in your browser, with no need for a server!

Machine Learning Engineering Open Book

Ongoing research training transformer models at scale

This repository contains demos I made with the Transformers library by HuggingFace.

An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, DeepSpeed, Axolotl, etc.

[NeurIPS 2024 Best Paper][GPT beats diffusion?] [scaling laws in visual generation?] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction". An *ultra-simple, user-friendly yet state-of-the-art* codebase for autoregressive image generation!

BertViz: Visualize Attention in NLP Models (BERT, GPT2, BART, etc.)

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

Leveraging BERT and c-TF-IDF to create easily interpretable topics.

This repository is a curated collection of links to various courses and resources about Artificial Intelligence (AI)

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

OpenChat: Advancing Open-source Language Models with Imperfect Data