TrAVis

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

《李宏毅深度学习教程》（李宏毅老师推荐?，苹果书?），PDF下载地址：https://github.com/datawhalechina/leedl-tutorial/releases

Natural Language Processing Tutorial for Deep Learning Researchers

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

? Scalable embedding, reasoning, ranking for images and sentences with CLIP

This repository contains demos I made with the Transformers library by HuggingFace.

Pre-Training with Whole Word Masking for Chinese BERT（中文BERT-wwm系列模型）

大规模中文自然语言处理语料 Large Scale Chinese Corpus for NLP

BertViz: Visualize Attention in NLP Models (BERT, GPT2, BART, etc.)

Leveraging BERT and c-TF-IDF to create easily interpretable topics.