LongLLaMA

A large language model designed to handle long-form text.

CommonProductProgrammingLanguage ModelNatural Language Processing

LongLLaMA is a large language model capable of processing long-form text. It is based on OpenLLaMA and fine-tuned using the Focused Transformer (FoT) method. It can handle text as long as 256k tokens or even more. We provide a smaller 3B base model (not instruction-tuned) and inference code with support for longer context on Hugging Face. Our model weights can be used as a replacement for LLaMA in existing implementations (for short contexts up to 2048 tokens). Additionally, we provide evaluation results and comparisons with the original OpenLLaMA model.

Best AI Websites & Tools

LongLLaMA

LongLLaMA Visit Over Time

LongLLaMA Visit Trend

LongLLaMA Visit Geography

LongLLaMA Traffic Sources

LongLLaMA Alternatives

MobileLLM-1B — A sub-billion parameter language model developed by Meta, suitable for device-side applications.

MobileLLM-350M — An efficiently optimized language model with sub-billion parameters, specifically designed for device-side applications.

Zamba2-7B — High-performance small language model

Meta Llama 3.1-405B — Large multilingual pre-trained language model

DCLM-baseline — High-performance language model benchmark dataset

Arcee Spark — A highly efficient and compact 7B parameter language model

Multi-Token Prediction — A multi-token prediction model designed to boost the efficiency and performance of language models

MDLM — An efficient masked diffusion language model.

Samba — Official implementation of an efficient infinite context language model

MAP-NEO — MAP-NEO is an entirely open-source large language model offering advanced natural language processing capabilities.

Trustworthy Language Model (TLM) Playground — Try Cleanlab's Trustworthy Language Model (TLM) in your browser

LLaVA++ — LLaVA++ extends the LLaVA model by integrating Phi-3 and LLaMA-3, enhancing the interaction capability between visual and language models.

OpenELM — OpenELM is a family of efficient language models equipped with an open-source training and inference framework.

Cappy — A lightweight scoring model that enhances the performance of large, multi-task language models.

LaVague — LaVague is a large language model framework for automating browser interactions.

Twitter-Insight-LLM — A Large Language Model for Twitter Insights

H2O-Danube-1.8B — A 1.8B parameter language model, open-source and free.

Baichuan 3 — A large language model with over trillion parameters

Lepton Search — Lepton is an open-source language model search platform

MaLA-500 — A large language model covering 534 languages

Wiseses AI — Intelligent Content Creation Platform

TinyGPT-V — Efficient multimodal large language model

intfloat/e5-mistral-7b-instruct — A text embedding model improved by a large language model for better text representation.

PixelLLM — Pixel-Aligned Language Model

PPLX Online LLMs — The first online language model API designed for question-answering

BlueLM Large Model — An independently developed intelligent language understanding model by vivo

Ava PLS — Desktop Local Language Processing Tool

StreamingLLM — An efficient streaming language model with attention downsampling

Mistral — Mistral is an open-source natural language processing model