Popular GitHub Repositories for Masked Language Models

Transformer-based models implemented in tensorflow 2.x(using keras).

The source code used for paper "Empower Entity Set Expansion via Language Model Probing", published in ACL 2020.

Code to reproduce experiments from the paper "Continual Pre-Training Mitigates Forgetting in Language and Vision" https://arxiv.org/abs/2205.09357

Comparing Selective Masking Methods for Depression Detection in Social Media

[CHIL 2024] Interpretation of Intracardiac Electrograms Through Textual Representations

A Context Aware Approach for Generating Natural Language Attacks.

Using an LLM to discover the genetic causes of rare disease

This script utilizes Facebook AI's RoBERTa model for masked language modeling to predict the top-k most likely words for masked tokens in a given sentence. It generates variations of the input sentence by filling in the masked tokens with the predicted words.

Powerful Python tool for visualizing and interacting with pre-trained Masked Language Models (MLMs) like BERT. Features include self-attention visualization, masked token prediction, model fine-tuning, embedding analysis with PCA/t-SNE, and SHAP-based model interpretability.