Popular GitHub Repositories for Attention

??? 60+ Implementations/tutorials of deep learning papers with side-by-side notes ?; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), ? reinforcement learning (ppo, dqn), capsnet, distillation, ... ?

Natural Language Processing Tutorial for Deep Learning Researchers

A PyTorch implementation of the Transformer model in "Attention is All You Need".

该仓库主要记录 NLP 算法工程师相关的顶会论文研读笔记

Scenic: A Jax Library for Computer Vision Research and Beyond

My implementation of the original GAT paper (Veličković et al.). I've additionally included the playground.py file for visualizing the Cora dataset, GAT embeddings, an attention mechanism, and entropy histograms. I've supported both Cora (transductive) and PPI (inductive) examples!

Aspect Based Sentiment Analysis, PyTorch Implementations. 基于方面的情感分析，使用PyTorch实现。

This is a pytorch repository of YOLOv4, attentive YOLOv4 and mobilenet YOLOv4 with PASCAL VOC and COCO

Implementation of LambdaNetworks, a new approach to image recognition that reaches SOTA with less compute

Improving Convolutional Networks via Attention Transfer (ICLR 2017)

Quantized Attention that achieves speedups of 2.1-3.1x and 2.7-5.1x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various models.

Generative Adversarial Transformers

Replication of simple CV Projects including attention, classification, detection, keypoint detection, etc.

Implementation of various self-attention mechanisms focused on computer vision. Ongoing repository.

An implementation of Performer, a linear attention-based transformer, in Pytorch

My implementation of the original transformer model (Vaswani et al.). I've additionally included the playground.py file for visualizing otherwise seemingly hard concepts. Currently included IWSLT pretrained models.

list of efficient attention modules

A Tensorflow implementation of Spatial Transformer Networks.

Graph Transformer Architecture. Source code for "A Generalization of Transformer Networks to Graphs", DLG-AAAI'21.

Simple implementations of NLP models. Tutorials are written in Chinese on my website https://mofanpy.com