Transformers-Tutorials

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

《李宏毅深度学习教程》（李宏毅老师推荐?，苹果书?），PDF下载地址：https://github.com/datawhalechina/leedl-tutorial/releases

Natural Language Processing Tutorial for Deep Learning Researchers

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

? Scalable embedding, reasoning, ranking for images and sentences with CLIP

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

AI Code Completions

Pre-Training with Whole Word Masking for Chinese BERT（中文BERT-wwm系列模型）

大规模中文自然语言处理语料 Large Scale Chinese Corpus for NLP

An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.