TOFU

The TOFU dataset provides a benchmark for fictional forgetting tasks for large language models.

CommonProductProductivityLanguage ModelForgetting

The TOFU dataset contains question-answer pairs generated based on 200 fictional authors that do not exist. It is used to evaluate the forgetting performance of large language models on real-world tasks. The task aims to forget models fine-tuned on various forgetting set ratios. This dataset uses the question-answer format, making it suitable for popular chatbot models like Llama2, Mistral, or Qwen. However, it can also be used for any other large language model. The corresponding codebase is written for Llama2 chatbot and Phi-1.5 models but can be easily adapted to other models.

Provides a benchmark forgetting dataset
Supports the evaluation of forgetting performance in large language models
Uses a question-answer format suitable for chatbot models
Codebase supports multiple language models

Evaluate the forgetting ability of language models and train forgettable chatbot models.

Fine-tune a Llama model using the TOFU dataset and then forget the model on different sizes of forgetting sets to evaluate forgetting performance.
Build a chatbot based on the TOFU dataset
training a forgettable model to avoid the robot remembering or leaking sensitive information.
Use the forgetting functionality in the TOFU codebase to test the performance differences of various models when forgetting specific information.

Visit

TOFU Visit Over Time

Monthly Visits

488643166

Bounce Rate

37.28%

Page per Visit

5.7

Visit Duration

00:06:37

TOFU Visit Trend

TOFU Visit Geography

TOFU Traffic Sources

TOFU Alternatives

Grok-2 — A cutting-edge language model with advanced reasoning capabilities.

InternationalSelection•Language Model•Reasoning

504

LMSYS Chatbot Arena — An online chatbot arena where the performance of different language models is compared.

InternationalSelection•Chatbot•Language Model

324

Stable LM 2 1.6B — 160 million parameter stable language model

Productivity•Language Model•Chatbot

186

Bard Advanced — A paid language model service expected to be launched by Google

Productivity•Language model•Chatbot

1278

WikiChat — Utilizes Wikipedia data to mitigate the illusion effect of language models

chatting•chatbot•language model

2442

OuteTTS-0.1-350M — A text-to-speech synthesis model that operates through a pure language model.

Productivity•Text-to-speech•Voice synthesis

180

Crisp 4 — A brand new site search experience to boost customer service efficiency.

chatting•Customer Service•CRM

MobileLLM-1B — A sub-billion parameter language model developed by Meta, suitable for device-side applications.

Programming•Language Model•Transformer

MobileLLM-600M — An efficient and optimized 600M parameter language model designed for device applications.

Programming•Language Model•Transformer

MobileLLM-350M — An efficiently optimized language model with sub-billion parameters, specifically designed for device-side applications.

Programming•Language Model•Transformer

MobileLLM-125M — An efficient, optimized small language model designed for device-side applications.

Programming•Language Model•Device-side Applications

MobileLLM — Optimized small language model suitable for mobile devices

Productivity•Language Model•Mobile Devices

SimpleQA — A benchmark test for measuring the ability of language models to answer factual questions.

Others•Benchmark•Language Model

AudioLM — High-quality audio generation framework

Others•Audio Generation•Language Model

138

Spirit LM — Multimodal language model that integrates text and speech

Productivity•Multimodal•Language Model

ChatGPT (Early Version for Plus, Team, Enterprise, and Edu) — The early version of ChatGPT, designed specifically for Plus, Team, Enterprise, and Education users.

chatting•Artificial Intelligence•Chatbot

234

Zamba2-7B — High-performance small language model

Productivity•Language Model•Natural Language Processing

126

rag-chatbot — A chatbot that can interact with multiple PDF files locally.

Programming•chatbot•question-answering

o1 in Medicine — Preliminary Research of AI in Medicine

Education•Artificial Intelligence•Medicine

gptme — A personal AI assistant in the terminal with local tools.

Programming•cli•chatbot

102

WebLLM — High-performance in-browser language model inference engine.

Productivity•Browser•Language Model

AMD-Llama-135m — A high-performance language model trained by AMD

Programming•Language model•Text generation

SFR-Judge — An intelligent evaluation tool that accelerates model assessment and fine-tuning.

Productivity•Artificial Intelligence•Evaluation Tool

Llama-3.1-Nemotron-51B — An efficient and accurate AI language model

Productivity•Language Model•Neural Architecture Search

MemoryScope — A powerful and flexible long-term memory system for LLM chatbots.

Programming•LLM•Long Term Memory

Best AI Websites & Tools

TOFU

TOFU Visit Over Time

TOFU Visit Trend

TOFU Visit Geography

TOFU Traffic Sources

TOFU Alternatives

Grok-2 — A cutting-edge language model with advanced reasoning capabilities.

LMSYS Chatbot Arena — An online chatbot arena where the performance of different language models is compared.

Stable LM 2 1.6B — 160 million parameter stable language model

Bard Advanced — A paid language model service expected to be launched by Google

WikiChat — Utilizes Wikipedia data to mitigate the illusion effect of language models

Starling-7B — Enhancing the usability and safety of LLM

SymeChat — Cloud-based AI Chatbot Service

TalkGPT — Intelligent language model, interactive dialogue

Chai — Leading chatbot AI platform

OuteTTS-0.1-350M — A text-to-speech synthesis model that operates through a pure language model.

Crisp 4 — A brand new site search experience to boost customer service efficiency.

MobileLLM-1B — A sub-billion parameter language model developed by Meta, suitable for device-side applications.

MobileLLM-600M — An efficient and optimized 600M parameter language model designed for device applications.

MobileLLM-350M — An efficiently optimized language model with sub-billion parameters, specifically designed for device-side applications.

MobileLLM-125M — An efficient, optimized small language model designed for device-side applications.

MobileLLM — Optimized small language model suitable for mobile devices

SimpleQA — A benchmark test for measuring the ability of language models to answer factual questions.

AudioLM — High-quality audio generation framework

Spirit LM — Multimodal language model that integrates text and speech

ChatGPT (Early Version for Plus, Team, Enterprise, and Edu) — The early version of ChatGPT, designed specifically for Plus, Team, Enterprise, and Education users.

Zamba2-7B — High-performance small language model

rag-chatbot — A chatbot that can interact with multiple PDF files locally.

o1 in Medicine — Preliminary Research of AI in Medicine

gptme — A personal AI assistant in the terminal with local tools.

WebLLM — High-performance in-browser language model inference engine.

AMD-Llama-135m — A high-performance language model trained by AMD

SFR-Judge — An intelligent evaluation tool that accelerates model assessment and fine-tuning.

Llama-3.1-Nemotron-51B — An efficient and accurate AI language model

MemoryScope — A powerful and flexible long-term memory system for LLM chatbots.