LongLLaMA

A large language model designed to handle long-form text.

CommonProductProgrammingLanguage ModelNatural Language Processing
LongLLaMA is a large language model capable of processing long-form text. It is based on OpenLLaMA and fine-tuned using the Focused Transformer (FoT) method. It can handle text as long as 256k tokens or even more. We provide a smaller 3B base model (not instruction-tuned) and inference code with support for longer context on Hugging Face. Our model weights can be used as a replacement for LLaMA in existing implementations (for short contexts up to 2048 tokens). Additionally, we provide evaluation results and comparisons with the original OpenLLaMA model.
Visit

LongLLaMA Visit Over Time

Monthly Visits

488643166

Bounce Rate

37.28%

Page per Visit

5.7

Visit Duration

00:06:37

LongLLaMA Visit Trend

LongLLaMA Visit Geography

LongLLaMA Traffic Sources

LongLLaMA Alternatives